CNNS AND POLARIZATION SIGNATURES BASED NOVEL APPROACH FOR LAND

This thesis titled 'Development of Deep Learning Approaches to Extract Information from Satellite Images' incorporates the study and critical analysis of the performance of conventional convolutional neural network (CNN) and recurrent neural network (RNN) models, and further development of novel CNN and RNN models to exploit the spectral and/or spatial and/or temporal information present in satellite images for applications such as land cover classification, disaster monitoring and mitigation, and agriculture crop monitoring. The main objective of this thesis is to pursue these studies with unimodal or multimodal satellite images. Special focus during these studies is to develop novel deep neural network (DNN) models (common term for CNN and RNN based models) that can achieve better classification accuracy and generalization performance in mixed land cover class scenarios. Four tasks are set to pursue these studies. The tasks set are based on: type of satellite image used, application for which the task is realized, multi-modality images involvement, and presence of time series information. The tasks to be carried out are; i) Study and assessment of polarization signatures as potential features for PolSAR image based land cover classification and further development of novel polarization signatures and CNNs based approaches to achieve improved classification performances with PolSAR satellite images, ii) Development of a novel approach using polarization signatures and CNNs for detection of human settlements affected during floods, iii) Development of novel spectral and Chapter 3

The satellite image modalities used in these studies are PolSAR and multispectral images because these two satellite image modalities are most common and used popularly. Therefore, development of novel DNN models on these modalities is motivating and beneficial. ALOS PALSAR-2 and Sentinel-1 satellite PolSAR images and Landsat-8 and Sentinel-2 satellite multispectral images are considered in these studies. Since, SAR images are all weather images and are recently the popular satellite image modality, special emphasis is given on study and development of CNN and RNN models to extract information from PolSAR images, in standalone or in combination with multispectral images. Multiple study areas are also considered for testing of the developed models or approaches. The study areas are selected based on presence of diversity in land cover (mixed land cover) and ease of accessibility to ground truth data.

Extensive ground truth collection is carried out in the study areas for the development and testing xii of the novel models or approaches. Thousands of ground truth samples are collected for each task throughout the study period. The ground truth samples are either collected directly on the terrain using GPS or via visual inspection of the satellite images used in different studies and corresponding Google Earth images.

The thesis consists of seven chapters. Chapter 1 provides; a brief introduction to satellite images, major challenges involved with traditional satellite image processing techniques, motivation to develop deep learning and DNN based models for satellite image processing, major objectives to be carried out during the research, and framework & organisation of the thesis.

Chapter 2 provides a brief literature review on the several state-of-the art deep learning and DNN based models and techniques developed for various satellite image based applications.

The review includes literature on DNN based techniques developed for single senor/unimodal satellite images, multisensor/multimodal satellite images, and satellite time series images.

The study presented in Chapter 3 involves a critical analysis of polarization signatures (PSs) as potential features for PolSAR data based land cover classification and development of novel approaches which utilize polarization signatures to provide improved classific ation performance. The study is divided into three sub tasks i.e. sub task 1: critical analysis of PSs as potential PolSAR features, sub task 2: development of an optimal class boundary estimator algorithm to fully exploit PSs, and sub task 3: development of CNN model to exploit PSs. The first sub task of this study explores PSs and utilizes it for the computation of PSs based features from quad PolSAR data. A land cover (e.g. bare soil) behaves as a complex mixture of different standard scatterers or standard targets and therefore its PS is difficult to interpret. Therefore, there is a need to find correlations between PSs of land covers and PSs of standard targets. These correlations indicate the degree of similarity between the two scatterers. In the direction, polarization signature correlation features (PSCFs) are obtained from PSs. Further, the PSCFs are critically analysed as potential features for PolSAR image based LC classification. The analysis indicated to the significant potential of polarization signatures or PSCFs as features for PolSAR image based land cover classification. In the second and third sub tasks of the study, two different approaches are developed. In particular, both classification techniques are developed to improve classification between tall vegetation and built-up areas because the builtup areas also produce volume scattering response along with double bounce which makes segregation of the two classes challenging in a PolSAR image. In the second sub task of the study, i.e. sub task 2, a novel adaptive and optimal class boundary estimator based decision tree classification approach is developed. Image statistics are used to obtain class boundary values xiii which make the approach robust and adaptive. Genetic algorithms are used to optimize the class boundary values to achieve maximum overall classification accuracy. Finally, a decision tree (DT) classifier, which utilizes the optimal class boundaries obtained by using this novel estimator on PSCFs, is realized. At last, in the third sub task of the study, i.e. sub task 3, a second novel classification approach namely PSs-CNN which utilizes 2D-CNNs as classifier directly with polarization signatures to perform land cover classification is developed. Since, the PSs are two dimensional representation of target backscatter response, they can be treated as images of target response and thus can intuitively be used in 2D-CNN classification framework. The CNNs captures the spatial context present in a pixel PSs which helps in its classification. For experimentation, PALSAR-2 fully polarimetric SAR data over Roorkee (Uttarakhand, India) study area is considered. Five land cover classes are considered namely built-up, dense tall vegetation such as forests, bare soil, short vegetation such as agriculture crops, and water. Both novel approaches are compared with other popular methods. The optimal class boundary estimator based DT classifier reports highest classification accuracy of 75.3% whereas the PSs-CNN classifier reports 73.2.8% classification accuracy. Classification results from the novel PSs-CNN model are comparable to the optimal class boundary estimator based DT classifier.

However, the CNN technique is better than the optimal class boundary estimator based DT classifier because the PSs-CNN classifier is simple, less hand-crafted, and easily scalable. Satellite PolSAR images are the most reliable, useful, and consistent form of land cover information during a flood disaster because of its weather independent monitoring capabilities.

Chapter 4 presents an efficient approach to map human settlements, especially villages in the rural parts of India that are affected during flood using PolSAR imagery, CNNs, apriori knowledge, and image morphology. Monitoring human settlements in the rural parts of India during floods is difficult even with PolSAR data. The built-up structures (e.g. houses) in these human settlements, especially villages of rural India are closely packed and randomly oriented.

Therefore, these villages show volume scattering response along with double bounce scattering response in a PolSAR image. The mixed scattering response from these villages hinders their segregation from tall vegetation in a PolSAR image. This challenge is also discussed in chapter 3. Therefore, the detection and mapping of these villages during a flood using PolSAR image is challenging as there is high risk of false alarms, or missing out a flood affected village during rescue operations.

The approach involves three stages: -first, built-up area extraction from a PolSAR image, -second, detection of villages in a built-up area image, and -third, identification and mapping villages that are affected by flood and masking out villages that are not. In the first stage, a CNN xiv based novel classifier which utilizes PolSAR polarization signatures (PSs) as features (similar to as in chapter 3), is proposed to extract built-up areas from a PolSAR image. This novel classifier uses a window (rather than a pixel) of PSs as input image because the villages are usually large in area and therefore cover several pixels in a PolSAR image. The window approach helps suppress the volume scattering present in the backscatter response of the villages and thereby improves the detection of these villages. However, there is presence of false alarms and true negatives in the built-up areas extracted and there is a need to mask out these artefacts. Therefore, the second stage proposes a novel village detection methodology. The methodology detects whether a particular object in the built-up area image (obtained in the first stage) is a village or not. A series of morphological operations are developed based on apriori knowledge are employed to suppress false alarms and true negatives. Next, villages that are affected by flood are required to be mapped and villages that are not affected by flood are masked out. In the third stage, flood affected villages are identified and mapped and non-flooded villages are masked out using another image morphology based methodology.

Quantitative and qualitative assessment of the performance of the novel approach over two different study areas is carried out. Different artificial flood scenarios are created to evaluate the performance of developed approach. The developed approach is evaluated under these flood scenarios and flood affected villages are identified and mapped. Finally, number of villages successfully detected and false alarms are computed with the help of ground truth information (actual number of villages or towns in the study regions are known). Results indicate to high detection accuracy (above 90%), low false alarms and true negatives. The approach is scalable and potentially satisfactory for real time usage.

Chapter 5 presents study and analysis of conventional, and further development of novel multisensor satellite image based land cover classification approaches that are based on CNNs.

First, the conventional CNN models are analysed and evaluated for multisensor satellite image based land cover classification. It is observed from this analysis that since earth observation satellite images have usually larger spectral depths than normal camera images, conventional CNN models are not sufficient to exploit the high spectral information present in these images.

However, exploiting the spectral information is crucial. In fact, similar to spatial domain, convolutions in the spectral domain are intuitive with high spectral dimension data. Convolutions in the spectral domain might also be an alternative to the process of feature selection. Based on this idea, chapter 5 presents two novel multisensor satellite image classification approaches that utilize CNNs in both spatial and spectral domains. The frameworks employ localized spectral convolutions along with localized spatial convolutions on permuted and stacked high xv dimensional multisensor images. The study establishes the significance of performing localized spectral convolutions on permuted input in comparison to non-localized or non-permuted localized spectral convolutions. Two models, namely, a permuted local spectral CNN (Perm-LS-CNN) and a permuted local spectral-spatial CNN (Perm-LSS-CNN) are obtained.

For assessment of the performance of models (conventional and novel), PALSAR-2 PolSAR and Landsat-8 multispectral multisensor images over the Roorkee and Haridwar (Uttarakhand, India) study areas are considered. Since, PolSAR and multispectral images provide complementary information and they are widely used together in numerous applications. Five land cover classes are considered namely built-up, tall vegetation such as forests, bare soil, short vegetation such as agriculture crops, and water. The models are trained and tested on ground truth class data points measured directly on the terrain. Classification accuracy and generalization performance of key models are evaluated. Generalization performance is evaluated via visual inspection of classified images on selected well known regions in the study areas. In an attempt to quantize the generalization performance of classifiers, another parameter, AD, which the difference between precision and recall, averaged over all the land cover classes is computed.

Comparison with conventional CNN models and other popular machine learning classifiers shows that the Perm-LSS-CNN model although has classification accuracies in the order of 90% (similar to other classifiers), but provides better generalization performance. Both indicators i.e. visual inspection and AD support to the better generalization performance of the Perm-LSS-CNN.

Chapter 6 presents a study in which an attempt is made; to analyse the conventional CNN and RNN models, and to develop novel CNN and RNN (in standalone as well as synergistic modes) models for agriculture crop classification with multisensor/multimodal satellite time series (MSTS) imagery. For this purpose, two sub tasks are set in this study. In the first sub task i.e. sub task 1, conventional CNN and RNN models are analysed and evaluated for the task of multisensor satellite time series (MSTS) images based crop classification. 1D, 2D, and 3D CNNs, RNNs, and combinations of these are considered for analysis. Then, based on the critical analysis of the working and performance of these conventional models, few novel models are proposed and developed in sub task 2.

The novel models involve permuted spectral and/or spatial convolutions and/or bidirectional recurrent layers. The Bidirectional RNNs capture the temporal profile and the permuted spectral and/or spatial CNNs capture the spectral and/or spatial profile of a particular crop. Bidirectional strategy in RNNs enables the influence of the past, present, and future inputs on the current output. Bidirectional RNNs are beneficial in time series images based crop xvi classification because crop classification can utilize the bidirectional process i.e. classification result can be observed from any time node. Also, the permuted spectral band stacking strategy developed in chapter 5 is also used in this study and presumed to improve the significance of the spectral information present in multisensor satellite time series images. In particular, a synergistic utilization of permuted spectral-spatial CNNs and Bidirectional RNNs helps in better exploitation of the multisensor satellite time series images.

Overall, 12 models, 6 conventional models namely; 1D-CNN-v1, 3D-CNN-v1, RNN-v1, 1D-CRNN-v1, 2D-CRNN-v1, and 3D-CRNN-v1, and 6 novel models, namely; Perm-1D-CNN-v1, Perm-3D-CNN-v1, Perm-RNN-v1, Perm-1D-CRNN-v1, Perm-2D-CRNN-v1 (also termed as Perm-LS-Bi-LSTM), and Perm-3D-CRNN-v1 (also termed as Perm-LSS-Bi-LSTM) are developed. For analysis and performance evaluation of the conventional and novel models, multimodal satellite (Sentinel-1 dual-PolSAR and Sentinel-2 multispectral) time series images over the Roorkee study area is considered. Wheat and sugarcane are the major crops grown in the area during the study period. Multisensor time series satellite images over a period of one crop (wheat) cycle (based on heuristic knowledge) are used. Also, to evaluate the robustness of the models under mixed land cover scenarios, land cover classes other than crop types such as forests and built-up areas are also included as output classes. Addition of these classes in the classification process resembles the actual scenario on the ground. Training and testing samples are once again measured and collected directly from the ground. Conventional and novel DNN based crop classification models are evaluated on the basis of classification accuracy and generalization performance. In terms of classification accuracy, Perm-1D-CNN-v1 achieves an accuracy of 99.70% which is the best among the models considered for comparison. In contrast, the generalization performance evaluated via visual inspection, indicated to a superior performance of the proposed Perm-3D-CRNN-v1 and Perm-1D-CRNN-v1 models as these models are able to capture the spectral, spatial, and temporal identity of land cover features more efficiently which results to improved generalization performance of these models.

Finally, chapter 7 concludes the different studies reported in this thesis. The conclusions drawn from each study, contributions made by the studies and, future scopes of the studies are also listed in this chapter. In summary, based on the studies presented in chapter 3, 4, 5, and 6, it can be concluded that the novel CNN and RNNs models developed in these studies are high performance and scalable and are potentially automatable. The models have shown good generalization performance which makes the models robust and less prone to overfitting and also these models perform well in mixed class scenarios. Further studies are however required to establish these models in the satellite image processing community.

ACKNOWLEDGEMENTS

Now, at this stage I am overwhelmed by the feeling of completing my Ph.D. and don't know how to describe my long journey. I began this journey back in July, 2015 with lots of aims, hopes in my mind. Today, I say, it was roller coaster ride. A plethora of events and emotions went through me that I think I cannot outline in these few paragraphs. Let me start by stating that I could not have reached at this destination or even couldn't have walked this far without the support of so many persons. I wish to extend by warmest gratitude towards my supervisors, Dr Dharmendra Singh and Dr Hussein Yahia. Their immense knowledge and plentiful experience have encouraged me in all the time of my academic research and daily life. The one person to whom I admire, honour, acclaim, trust, rely and cherish is my honourable supervisor Dr Dharmendra Singh. He has been always there to help me both technically, personally, emotionally, and morally. He cares for everyone in his own way and by all his means. This is his forte that makes him unique in all scores. I am very much thankful for his guidance and inculcating research and personal skills in me. His intuitions and ability to resolve complex problems has always helped me excel my research findings. It is his hunger for research that I could have been able to accomplish my research work in an advanced topic of deep learning for satellite image processing with limited resources in hand. Without his untiring efforts, stimulating discussions, and motivational examples I would not have been writing this page for completion towards my thesis. In contrast, my honourable supervisor, Dr Hussein Yahia was a 'Polar Star' for me during my research journey. He always believed in my research intuitions & ideas and guided me through by appending his expertise, experience, and knowledge. He was as cool as mint irrelevant of the research outcome as he believed in learning rather than achieving. His unconditional support kept me going and timely inputs helped me breakthrough technical challenges. Let me assure you that he is a believer. I would like to extend my warmest gratitude and regards to Dr Dharmendra Singh and Dr Hussein Yahia for a synergistic (fire & ice) guidance during my Ph.D. I would like to take a minute to admire the efforts of Dr Nicolas Brodu, who played the role of a high speed wheels during my research journey. If not for him, I would have struggled a lot my way here. He is an all-rounder, may it be fundamentals of machine learning or programming languages or it be the state-of-the-art algorithms, he had notion of either. He advocated and recommended me to go for open-source scientific programming such as Python based development of satellite image processing deep learning algorithms. His vision helped me sail through my research goals and overcome conceptual and non-conceptual challenges. I will xx always be thankful for his all-round support. I would also like to forward my thanks to Dr Khalid Daoudi for his wonderful company at times and his virtues as a critic which helped me a lot to improve the understanding and representation of my work. This acknowledgement will be incomplete without mentioning the names of my research committee members; Dr M.V. Kartikeyan (chairman), Dr R. Balasubramanian (external), Dr A. Panaik (internal) for their valuable suggestions and healthy criticism during my research work presentations. I would also like to extend my thanks to all the faculty members and staff members of ECE department, IIT Roorkee for their support and suggestions. I would like to thank Shrawan and Satendra in the Remote Sensing Lab for his helping hand during the experimental work and his serving tea, snacks which comes as an energy drink during long, tiring study hours.

Nobody has been more important to me in the pursuit of this research than the members of my family. I extend my deepest gratitude to my father Mr. J. S. Phartiyal and my mother Ms. Manju Phartiyal for going through immense hardships to see me fulfil my dreams. I will be indebted to them forever. Their blessings, prayers, and values light my life. They made me believe that education is not only measured by academic titles but by ethics, integrity, kindness and diversity of interests. I wish to acknowledge my profound appreciation to my brother Mr. Himanshu Phartiyal, my sister Ms. Seema Phartiyal, my in-laws Mr. U. S. Trivedi, Ms. Anita Trivedi, Ms. Annapurna Dixit, and Mr. Shubham Trivedi for their anytime assistance and full support in various ways. It was their patience, encouragement and faith in me that I could complete this thesis. My journey xxi would not be possible or even instigated without the firm support of my lovely wife Ms. Deepmala Phartiyal who had been my friend and invisible guide, always caring and nurturing when I was stressed or tensed. I admire her for unconditional love and patience during all these years.

Writing these few lines give me utmost pleasure as these lines are dedicated to my friends; Akhil, Nikhil, Vishal, Deepmala, Prateek, Lokesh, Pankaj, Ganesh, Tarun, Avantika, Navin, Eshan, Rakesh, Alaknanda, Shilpi, Hitesh, Sachin, Ashutosh, Bharat, Ayush, Mandakini, Purnima, shambhaivi, Ruchi, Ayushi, Avantika. Without these guys, I would not have been able to withstand the pressure of this journey. Akhil, Nikhil, Vishal, Avantika, Deepmala were the sparkles who kept me cheered up all along. I would like to thank everyone whom I met and was part of journey during this period.

I would like to thank Prabhu, Bhagwaan, Ishwar, and God for keeping a blessing eye on me. This journey would not have been smooth if not for his showers of wisdom.

It would be impossible to acknowledge all the people who contributed either technically or by offering their friendship towards the completion of this work. I am thankful to all those who were associated with my research journey directly or indirectly and to whom I forgot to mention here unknowingly. My thesis has been a wonderful experience and I cherish the educational and personal growth it has provided to me for my future endeavors. Gopal

Background: Satellite Images

Satellite imagery depicts the earth's surface at various spectral, temporal, radiometric, and spatial resolutions, as determined by the sensing device (sensor) on board each collection system, and the orbital path of its reconnaissance platform [START_REF] Townsend | Modeling floodplain inundation using an integrated GIS with radar and optical remote sensing[END_REF]. Satellite sensors acquire information reflected or emitted from an object in the form of electromagnetic radiation and then the information about the state and condition of an object is derived by interpreting this received electromagnetic signal. The collected information (in the form of images) indicates about the physical, chemical, and biological properties of observed object which may be soil [START_REF] Singh | Spatial and temporal variability of soil moisture over India using IRS P4 MSMR data[END_REF], [START_REF] Maurya | A Step for Digital Agriculture by Estimating Near Real Time Soil Moisture with Scatsat-1 Data[END_REF], ice, snow [START_REF] Venkataraman | Fully polarimetric ALOS PALSAR data applications for snow and ice studies[END_REF], [START_REF] Singh | Categorization of the glaciated terrain of Indian Himalaya using CP and FP mode SAR[END_REF], water [START_REF] Gairola | Retrieval of precipitable water from bhaskara-II microwave measurements and its comparison with NOAA-7 and radiosonde data[END_REF], wind [START_REF] Sil | Response of OceanSat II scatterometer winds in the Bay of Bengal circulation[END_REF], vegetation [START_REF] Avtar | Characterization of forests and deforestation in Cambodia using ALOS/PALSAR observation[END_REF] or any other. The goal of collecting the information is to understand how electromagnetic energy interacts with the earth's surface objects. The information can further be used for crucial military and civil applications such as monitoring; vegetation coverage [START_REF] Pandey | An optimal estimation technique for increasing the accuracy of crop forecasts by combining remotely sensed and conventional forecast[END_REF], soil moisture [START_REF] Kim | A Time-Series Approach to Estimate Soil Moisture Using Polarimetric Radar Data[END_REF], disasters [START_REF] Altan | Spatially supported disaster management: Introduction to the special issue "gI for disaster management[END_REF], sea-ice fluctuation, sea level and sea surface temperatures [START_REF] Gohil | An algorithm for sea surface temperature estimation from ERS-1 ATSR using moisture dependent coefficients: A simulation study[END_REF], [START_REF] Basu | Empirical orthogonal function analysis of humidity profiles over the Indian Ocean and an assessment of their retrivability using satellite microwave radiometry[END_REF], ships in sea [START_REF] Chen | Ship detection using polarization cross-entropy[END_REF], concentrations of atmospheric gases, and many other parameters. These observations are applied to meteorological forecasting, environmental monitoring, disaster monitoring, and cartographic purposes.

Satellite images can be gathered using different ways of "sensing" the object: (i) by reflection, (ii) by emission, and (iii) by combined emission-reflection.

Satellite image: Characteristics

A satellite image has four different characteristics namely spatial resolution, spectral resolution, radiometric resolution, and temporal resolution. The concept of resolution embraces several aspects and most common of these is spatial resolution, but other aspects are also very important for image interpretation.

A. Spatial Resolution

Spatial resolution is the fineness of detail in the image and is measured in pixel per inch (ppi).

The more pixels per inch, better the resolution. Generally, an image with a higher resolution produces a better printed image quality. Spatial resolution helps to identify the minimum separation at which objects appear independent and isolated. It is measured in millimetres on the photograph or in meters on the ground, and it depends on the focal length of the camera and its height above the ground. The most common definition of sensor spatial resolution is the size of the projected instantaneous field of view (IFOV) on the ground (d), computed as in equation

(1.1). IFOV 2 tan( ) 2 dH  (1.1)
Where d is the distance on the ground for each information unit (pixel) and H the height of the observing sensor. IFOV is defined as the angular section observed by the sensor, in radians, at a given moment in time.

The spatial resolution of optical-electronic sensors depends on orbit height, speed of data collection, and the number of detectors. As for the antenna sensors such as SAR, their resolution depends on the aperture angle, height of the platform, and the wavelength at which they work.

The larger the aperture angle, the lower the height and the wavelength, and the more detailed the resolution. Figure 1.1 shows a subset of image acquired via the Landsat-8 sensor which has 30 m spatial resolution. It means that a single pixel represents an area of 900 square meters (30m ×30m) on the ground. The spatial resolution of imaging sensors improved significantly over time as shown in Figure 1.2 and is usually inversely related to acquisition period and coverage area.

B. Spectral Resolution

Spectral resolution refers to the number of bands provided by the sensor, as well as their spectral bandwidths. Generally speaking, a sensor will provide better discrimination capacity as more bands are acquired. Multispectral satellite imaging systems record energy in multiple bands of the electromagnetic spectrum. Table 1.1. briefly summarizes the categorization of the electromagnetic (EM) spectrum in terms of spectral bands and spectral bandwidths. The nominal size of a band may be large (i.e. coarse), as with the Landsat MSS near-infrared band 4 (800 -1,100 nm) or relatively smaller (i.e. fine), as with the Landsat MSS band 3 (700 -800 nm). Thus, Landsat MSS band 4 detectors recorded a relatively large range of reflected near-infrared radiant flux (300 nm wide) while the MSS band 3 detectors recorded a much reduced range of nearinfrared radiant flux (100 nm wide). A hyperspectral satellite imaging sensor typically acquires images in hundreds of spectral bands. For example, the earth observing-1 Hyperion has 220 bands in the region from 356 to 2,577 nm spaced just 10 nm apart based on the full width and half maximum (FWHM) criteria.

C. Radiometric Resolution

The radiometric resolution of an imaging system (including satellite sensors) is described as its ability to discriminate very slight differences in energy. The finer the radiometric resolution of a sensor, the more sensitive it is to detecting small differences in reflected, scattered, or emitted energy. Figure 1.3 demonstrates the concept of radiometric resolution. In Figure 1.3, the image represented with 1 bit has only 2 levels of intensities whereas the image represented with 16 bits has 65,536 levels of intensities. While the arrangement of pixels describes the spatial distribution of a structures present in an image, the radiometric characteristics represent information about the interaction of the structure with electromagnetic energy. Every time an image is acquired on film or sensor, its sensitivity to the change in magnitude of the electromagnetic energy determines its radiometric resolution.

D. Temporal Resolution

Temporal resolution is defined as the amount of time needed to revisit and acquire data for the exact same location. When applied to satellite imaging based earth observation, temporal resolution is controlled by a sensor's swath width alongside the orbital parameters of its sensing platform; together these parameters determine the periodicity (return interval) at which a fixed area on the ground is successively imaged by a space borne sensor. It is also important to bear in mind that the effective temporal resolution depends on the atmospheric conditions because optical and thermal sensors cannot detect surfaces under cloud cover. That is why in some areas the availability of useful images can be notably lower that the acquisition cycle of the sensor (tropical wet regions, for instance). To somehow alleviate this problem, modern sensors have the capacity to make observations pointing the sensor at different angles, facilitating the collection of data over a zone of interest from an adjacent orbit.

Temporal resolution is a primary consideration when selecting an image product for researchers studying time series behaviour. Sensors with higher temporal resolution or more extensive imaging archives are however, most likely to afford cloud-free and seasonally favourable images, making them attractive options for all investigators. Figure 1.4 depicts the temporal revisit period of an arbitrary satellite as a typical example. Table 1.1 Broad categorization of the electromagnetic spectrum in terms of spectral bands and their spectral bandwidths [START_REF] Jensen | Remote sensing of the environment: an earth resource perspective[END_REF].

EM Radiation Regions Wavelength Applications

Gamma-ray <10 -3 nm Food purification, medical hygienic and nuclear medicine X-ray 0.1 to 10 The temporal resolution of a satellite imaging system refers to how often it records imagery of a particular area. This example depicts the systematic collection of data every 30 days, presumably at approximately the same time of day.

Satellite images: Acquisition types and sensors

One of the most common ways to classify sensor systems is on the basis of the mechanism they use to detect electromagnetic energy. In this regard, two broad groups of sensors: i) Passive, which simply collect radiation derived from external sources, and ii) Active, in which the sensor system is able to provide both, the energy source and detect reflection of the energy from the observed surfaces. Obviously, active sensors are more flexible because they are independent of the sensor-earth illumination conditions. A further distinction among passive sensors is to divide them according to the method used to record the incoming energy.

Passive Imaging: Passive remote imaging works in one of the following three ways: i) By observing how radiation changes as it passes through objects (such as sunlight passing through a thin clouds), ii) By observing radiation emitted by an object of interest (such as the emission of infrared radiation by earth), or iii) By observing radiation reflected or scattered by an object of interest (such as a visible satellite image taken from space). These three methods take advantage of naturally occurring radiation (such as sunlight or infrared radiation emitted by objects) and as such, provide a wealth of information all around us. For example, human eyes serve as types of passive remote imaging devices -our eyes capture scattered and emitted visible light, which the human brain process into images. In contrast, satellite imaging devices such as the Landsat series satellites or the Sentinel-2 satellite constellation have aboard them sensors than can record electromagnetic radiation beyond visible light and provide a great deal of information about our environment.

Active Imaging: Active remote imaging devices work by, first emitting a signal, and then recording the "return" of the emitted signal from an object of interest. It includes, observing how long it took the signal to return, the magnitude of the returned signal, and changes in the signal properties such as frequency shifts or orientation (phase, polarization) of the signal. In nature, bats make extensive use of their natural active remote imaging system. They emit ultrasonic 'squeaks' which bounce off from objects around them including the 'moths'. The bats use this active imaging method to locate moths for food. A bat can 'see' (locate) a moth flying in the dark by processing to the changes in the returned ultrasonic signal from the moths. Whales and dolphins use a similar system to augment their other senses for communications and hunting.

Their system is termed as sound navigation and ranging or more popularly as 'SONAR'. Satellitebased active remote imaging systems include RADARS (an acronym for radio detection and ranging) and LIDARS (an acronym for light detection and ranging) which work by emitting electromagnetic radiation (either radio waves, in the case of radar, or a laser beam for the LIDAR system) in the direction of the object to be sampled, and then utilizing a parabolic dish to collect the return (called 'backscatter') radiation for processing by computers.

Popular satellite imaging modalities A. Optical imaging for earth observation

Imaging in the optical region of the electromagnetic (EM) spectrum implies use of visible, near A hyperspectral imaging system, also known as an "imaging spectrometer" collects information in about a hundred or more contiguous spectral bands. The goal is to collect precise spectral information about objects of interest such as minerals, oil, gas, forests or others. This information enables better characterisation and identification of objects. Hyperspectral images have potential applications in fields as coastal management (e.g. monitoring of phytoplankton, pollution, bathymetry changes) [START_REF] Mishra | A novel algorithm for predicting phycocyanin concentrations in cyanobacteria: A proximal hyperspectral remote sensing approach[END_REF], forestry, water spectroscopy, mineral exploration, precision agriculture (e.g. monitoring the types, health, moisture status and maturity of crops). An example of a hyperspectral system is: Hyperion which has 220 spectral bands. Hyperion sensor specifications are listed in Table 1.5.

B. Microwave imaging for earth observation

Electromagnetic radiation in the microwave region is also used in imaging to provide useful information about the earth's surface and objects on it. There are various types of microwave imaging systems developed to capture information in different forms.

A microwave radiometer is a passive sensing device which records the microwave emissions emitting naturally from the earth's surface. The sensing technology can be used in many applications such as measuring and monitoring sea surface temperature or land surface temperature etc.

A radar altimeter sends out pulses of microwave energy and records the energy scattered back from the object on the earth surface. The height of the object from a certain point on earth's surface can be measured based on the time taken by the energy signal to return.

A wind scatterometer can be used to measure wind speed and direction over the ocean surface. It sends out pulses of microwaves along several directions and records the magnitude of the signals backscattered from the ocean surface. The magnitude of the backscattered signal is related to the ocean surface roughness, which in turns is dependent on the sea surface wind condition, and hence the wind speed and direction can be derived from platforms to generate high resolution images of the earth surface using microwave energy.

Synthetic aperture radar

In synthetic aperture radar (SAR) imaging, microwave pulses are transmitted by an antenna towards the earth surface at an oblique angle and the microwave energy scattered back towards the antenna (backscatter) is measured and used to analyse earth surface features. The SAR imaging system form an image by utilising the magnitude and time delay of the backscattered signals. Mathematically, this backscattered signal is represented using radar cross section (RCS)  , which is defined as the ratio between the incident and received signal intensity. The microwave energy beam sent out by the antenna towards the earth's surface illuminates an area on the ground. The illuminated area is termed as the antenna's "footprint". The recorded or received signal strength depends on the microwave energy backscattered from the objects inside this footprint. In real aperture radar imaging, the ground resolution is limited by the size of the microwave beam sent out from the antenna. Finer details on the ground can be resolved by using a narrower beam. The beam width is inversely proportional to the size of the antenna i.e. the shorter the antenna, the wider is the beam or footprint and the longer the antenna, the narrower the beam or footprint. Increasing the length of the antenna will decrease the width of the footprint.

It is not feasible for a spacecraft to carry a very long antenna which is required for high resolution imaging of the earth surface. To overcome this limitation, a SAR capitalises on the motion of the space craft to emulate a large antenna from a small antenna it actually carries on board.

SAR polarimetry

In general, when an EM wave is emitted from a source (e.g. a radar antenna), it propagates in all directions (a spherical wavefront), with a specific field strength and a phase in each direction. At a long distance from the antenna, it can be assumed that the wavefront of the emitted EM wave lies on a plane, rather than on the surface of a sphere. Polarization is an important property of an EM wave. Polarization refers to the alignment and regularity of the electric and magnetic field components of the EM wave, in a plane perpendicular to the direction of propagation. In particular, EM wave can be characterized by the behaviour of the electric field vector as a function of time. The propagation of an EM wave is illustrated in Figure 1.5. A scatterer (object on the surface of earth in this case) can change the polarization of the EM wave incident upon it.

The polarization of the scattered EM wave can be different from the polarization of the incident EM wave. Usually, a radar antenna is often designed to receive a predefined set of polarization components of the scattered EM wave simultaneously. For example, the horizontal (H) and vertical (V) parts of an antenna can receive the two orthogonal components of the incoming EM wave, and the system electronics keep these two signals separate. Figure 1.5 Illustration of the propagation of an electromagnetic plane wave. The electric field vector has horizontal (green) and vertical (blue) components, which combine to yield the net electric field vector (red) [START_REF] Ulaby | Microwave remote sensing: active and passive. Volume I: microwave remote sensing fundamentals and radiometry[END_REF].

Denoting the transmit and receive polarizations by a pair of symbols, a radar system using H and V linear polarizations can thus have the following channels:

 HH -for horizontal transmit and horizontal receive,  VV -for vertical transmit and vertical receive,  HV -for horizontal transmit and vertical receive, and  VH -for vertical transmit and horizontal receive.

The first two of these polarization combinations are referred to as like-polarized or copolarized, because the transmit and receive polarizations are the same. The last two combinations are referred to as cross-polarized because the transmit and receive polarizations are orthogonal to one another [START_REF] Riegger | On the origin of cross-polarization in remote sensing[END_REF].

A radar system can have different levels of polarization complexity. Popular polarization configurations are listed here (though not limited to).

 Single polarized (single channel) -HH or VV or HV or VH  Dual polarized (dual channel) -HH and HV, VV and VH, or HH and VV  Four or full polarized (four channels) -HH, VV, HV, and VH A quadrature polarized (i.e. fully polarimetric) radar uses these four polarizations, and measures the phase difference between the channels as well as the magnitudes. Some dual polarized radars also measure the phase difference between channels, as this phase plays an important role in polarimetric information extraction.

When a horizontally polarized wave is incident upon a target, the backscattered wave can have contributions in both horizontal and vertical polarizations. The same applies to a vertically polarized incident wave. As the horizontal and vertical components form a complete basis set to describe an electromagnetic wave, the backscattering properties of the target can be described by a scattering matrix, S as.

 

HV VV S S

HH VH S S S     (1.2)
The four elements of the scattering matrix are complex, and can be obtained from the magnitudes and phases measured by the four channels of the receiving antenna of a polarimetric radar.

The different polarization channels record different scattering mechanisms, and hence offer different information of objects as mentioned in Table 1.6. HH, HV, VH, and VV polarizations provide double-bounce, volume, and surface scattering respectively. For example, HH polarization gives double-bounce scattering, which is usually obtained from buildings in a landscape consisting of a variety of land covers.

Historical, current and future SAR sensors:

Spaceborne SAR sensors have been in use for more than 40 years. The first SAR sensor was launched on June 28, 1978, on board the NASA's Seasat satellite. The SAR sensor was developed and deployed, aimed at monitoring oceanographic phenomena. Seasat carried an HHpolarized L-band SAR sensor mounted at a fixed angle to observe the global 'surface wave fields' and polar sea ice conditions as part of its sensor suite. Even though the Seasat's SAR sensor operated for only 106 days, the mission was deemed an extensive success, demonstrating a SAR's capability to monitor both ocean and land surface.

Since then, SAR imaging has made a long and successful journey. Several SAR sensors with ever-improving imaging capabilities have been used by organizations, communities, nations collectively ensuring continuous coverage of the earth and availability of SAR data [START_REF] Rao | Generation of dual beams from spherical phased array antenna[END_REF].

Unfortunately, this constellation of SAR systems comes with a downside. The SAR sensors launched by the various agencies vary widely in their sensor configurations. The data from the different sensors are not always directly comparable with each other. at C-or X-band. Hence, the applications supported by a SAR sensor depends on the frequency band used.

Table 1.7 also summarizes typical applications of SAR as a function of the frequency band. It shows that sensors at X-band predominantly are used for built-up and infrastructure monitoring. Due to the X-band's higher resolution capabilities, SAR sensors at this frequency find broad applications in surveillance and tracking. These SAR sensors are also often used in the monitoring of industrial installations. Due to its limited penetration into vegetation covers, X-band is rarely used to characterise forest canopies or monitor activity underneath vegetation.

With the predominant number of SAR sensors operating at this frequency range, C-band sensors have been the workhorse of SAR based monitoring over the last 30 years. C-band data can be seen as a good compromise between X-band and the longer wavelength (e.g. L-band)

sensor classes because of its moderate-to high-resolution capabilities and increased vegetation penetration. In comparison to X-band SARs in terms of swath widths, C-band SARs typically allow for wider swath imaging, lending its applicability to regional and global scale. While Cband has improved canopy penetration capabilities, its signals will typically not penetrate vegetation completely. C-band is of limited use for monitoring activity underneath vegetation canopy layers, especially in regions with dense vegetation.

While S-band SAR sensors were used rarely in earth observation in the past, this frequency will have increased usage soon. NovaSAR-S, a S-band SAR sensor, was launched in September 2018. NovaSAR-S will provide moderate-resolution SAR data. This data will help explore S-band data's performance for applications such as disaster monitoring, crop monitoring, forest monitoring (temperate and rainforests), and land cover mapping. More interesting to most users will be the upcoming NASA-ISRO SAR satellite, NISAR. In addition to an L-band SAR, NISAR will carry a fully polarimetric S-band SAR. While NISAR's S-band coverage will likely not be global, all data will be freely and openly available to the SAR science and applications community. A list of the most relevant past, current, and future SAR platforms is provided in Table 1.8. The sensors are sorted by their period of functioning. For each instrument, sensor's operating wavelength, supported polarization modes, resolution and size of image products, repeat cycle, and data access means are listed. or modality (spectral reflectance, backscatter, brightness-temperature) of satellite images.

Although, single sensor or unimodal satellite images are able to provide satisfactory quality spatial, spectral, or temporal information but a trade-off is always at play. For example, if a satellite image has very good spatial resolution, then the satellite sensor is ought to take more time to gather this detailed spatial information than the sensor which provides satellite image with lower spatial information. In short, higher the spatial resolution, lower the temporal resolution, keeping all other aspects same. Similar trade-off holds for spectral and spatial characteristics of satellite images. These trade-offs hinder the ability of a single sensor satellite image to be sufficient for every application.

Alternatively, use of multimodal and multisensor satellite images address the issues associated with the use of single sensor images. Use of multisensor images improves the spatial, spectral, and temporal characteristics simultaneously and therefore improving the application performance. Moreover, use of multisensor satellite images also address to some other issues associated with unimodal images. For example, multispectral images (modality: surface reflectance) are less noisy and easy to interpret but suffer from issues like obstruction due to cloud cover or no night-time imaging. On the other hand, PolSAR images (modality: backscatter) are weather-independent (can be captured at any time) but suffer from issues such as noise (speckle, layover, foreshortening etc.) and image interpretation. Use of multisensor images resolves these issues to some extent. Therefore, use of multisensor and/or multimodal satellite images for many applications is not uncommon. Also, with the availability of multiple sources of information for any particular application, it is important to exploit all the available sources efficiently and effectively. Therefore, this thesis also focuses to exploit multisensor satellite images for land cover classification, agriculture crop classification, and disaster monitoring.

Satellite time series images are set of images of the same area captured at different time stamps using the same satellite imaging sensor. Multisensor time series images include time series images from more than one satellite sensor. Both, single sensor and multisensor satellite time series images have proved potential in various applications such as change detection, agriculture crop monitoring, etc. This thesis focuses to exploit multisensor satellite time series images for agriculture crop classification.

In summary, unimodal or single sensor satellite images, multimodal or multisensor satellite images, and satellite time series images have advantages, disadvantages, and varied applicability for earth observation. This thesis focuses to exploit all the three categories for different applications.

Satellite Image Processing: Recent Challenges and Expectations

Processing satellite images for utilization in different application is challenging. In general, images acquired and utilized for earth observation are different from normal images because;first, earth observation images contain significantly higher spatial information. A single image may contain spatial information about from a few kilometres to few hundred kilometres of the earth's surface, and -second, earth observation images also contain relatively higher spectral information. It means unlike the normal camera images which have three spectral bands i.e. red, green, and blue, spectral bands in these images may vary from tens to hundreds. These unique characteristics of earth observation images make them challenging to handle and require special approaches to process. Satellite based earth observation images also possess these properties and are equally challenging to be utilized in desired applications such as land cover classification or agriculture crop monitoring. The challenges can be divided into three categories for the sake of clarity i.e. challenges with single sensor satellite image processing, challenges with multisensor satellite image processing, and challenges with satellite time series image processing. These challenges are briefly discussed here.

Single sensor and unimodal satellite images have been used in numerous earth observation applications. Among a wide variety of applications, land use land cover or land cover (LC) classification is one of the most important and mundane task. Traditional techniques have been successful in performing land cover classification with various unimodal satellite images [START_REF] Lu | A survey of image classification methods and techniques for improving classification performance[END_REF]. Few of the popular image classification techniques are decision trees, random forest [START_REF] Saraswat | Feature selection and classification of leukocytes using random forest[END_REF], maximum likelihood, support vector machines, artificial neural networks, k-nearest neighbours [1], logistic regression [2], distance measure [3], K-means, ISODATA, and more. These techniques have been widely used in satellite image based land cover classification for many years and have performed well [START_REF] Lu | A survey of image classification methods and techniques for improving classification performance[END_REF]. However, these techniques possess certain limitations such as requirement of a feature extraction process, high-dimensional data inoperability, sub-optimal classification, performance saturation with increase in availability of trainable data [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF], [START_REF] Li | Deep learning for remote sensing image classification: A survey[END_REF] or due to local minima, and low generalization performance [START_REF] Lecun | Deep learning[END_REF]. Generalization is the ability of a classifier or classification technique to perform well outside the training sample space.

Researchers in the satellite image processing community are always looking forward to techniques that can process the high dimensional satellite images and possess abilities like scalability, automatability, generalizability, and high-performance in mixed land cover scenarios.

For example, satellite polarimetric synthetic aperture radar (PolSAR) earth observation data is weather independent data but it suffers from issues such as overlapping of class observables Decomposition based [START_REF] Avtar | Characterization of forests and deforestation in Cambodia using ALOS/PALSAR observation[END_REF], [START_REF] Yonezawa | Polarimetric decomposition analysis of ALOS PALSAR observation data before and after a landslide event[END_REF]- [START_REF] An | Four-component decomposition of polarimetric SAR images with deorientation[END_REF], factorization based [START_REF] Ratha | A PolSAR scattering power factorization framework and novel roll-invariant parameters based unsupervised classification scheme using a geodesic distance[END_REF], and machine learning based techniques [START_REF] Gopal | Fuzzy neural network classification of global land cover from a 1° AVHRR data set[END_REF]- [START_REF] Feng | Machine learning-based integration of remotelysensed drought factors can improve the estimation of agricultural drought in South-Eastern Australia[END_REF] are most popular with satellite PolSAR image based land cover classification. Decomposition and factorization based methods mostly rely on exploring physical scattering properties [START_REF] Moreira | A tutorial on synthetic aperture radar[END_REF]- [START_REF] Zhou | Linear feature detection in Polarimetric SAR images[END_REF] and texture information [START_REF] He | Texture classification of PolSAR data based on sparse coding of wavelet polarization textons[END_REF], [START_REF] Chen | Nonlocal filtering for polarimetric SAR data: A pretest approach[END_REF] in SAR images. Also, these techniques are mainly based on specific problems and characteristics of SAR data sources. On the other hand, machine learning techniques are more generic in nature but they suffer from limitations such as overfitting and low generalization. Similar issues are observed with other modalities of satellite images. Therefore, more efficient (high performance), scalable and robust techniques are to be explored in order to achieve improved classification performances. Similar to land cover classification, other satellite image based applications such as crop and disaster monitoring can benefit from more advanced techniques. Moreover, traditional approaches with unimodal optical images also suffer from similar challenges.

Alternatively, synergistic exploitation of multimodal or multisensor satellite images is also a challenge for traditional techniques. Numerous multisensor satellite image processing techniques have been developed and have proved successful to some extent [START_REF] Pohl | Multisensor image fusion in remote sensing: Concepts, methods and applications[END_REF]- [START_REF] Mishra | Bathymetric mapping using IKONOS multispectral data[END_REF]. However, the popular multisensor satellite image processing techniques such as data fusion techniques are data and application intensive and tend to loose information during the fusion process [START_REF] Pohl | Multisensor image fusion in remote sensing: Concepts, methods and applications[END_REF], [START_REF] Pohl | Geometric aspects of multisensor image fusion for topographic map updating in the humid tropics[END_REF].

Machine learning techniques such as neural networks are also popular multisensor satellite image processing techniques but they suffer from overfitting and low generalization performance issues [START_REF] Zhu | Deep learning in remote sensing: A comprehensive review and list of resources[END_REF]. Therefore, there are still possibilities for improvements in the current multisensor satellite image processing techniques and there is scope for new ones.

Furthermore, efficient processing of satellite time series images is also critical for many applications [START_REF] Bargiel | Multi-temporal land-cover classification of agricultural areas in two european regions with high resolution spotlight TerraSAR-X data[END_REF]- [START_REF] Page | A novel cross-satellite based assessment of the spatio-temporal development of a cyanobacterial harmful algal bloom[END_REF]. For example, crop type identification and classification is a major step during agriculture crop monitoring [START_REF] De | Classification of crops, pastures, and tree plantations along the season with multi-sensor image time series in a subtropical agricultural region[END_REF], [START_REF] Skriver | Crop classification by multitemporal Cand L-band single-and dualpolarization and fully polarimetric SAR[END_REF]. Accurate crop classification requires synergistic exploitation of spatial, spectral and temporal profiles of crops [START_REF] Villa | In-season mapping of crop type with optical and X-band SAR data: A classification tree approach using synoptic seasonal features[END_REF]. Numerous techniques have been reported so far, for satellite time series image based crop classification such as in [START_REF] Mcnairn | A Review of Multitemporal Synthetic Aperture Radar (SAR) for Crop Monitoring[END_REF], but certain limitations still exist such as presence of other land covers (built-up, forests, or grasslands) in and around the croplands under study. Therefore, it is still a challenge for existing techniques to synergistically exploit the spatial, spectral, and temporal information of a crop present in unimodal/single-sensor or multimodal/multisensor satellite time series images and provide accurate and reliable crop classification results [START_REF] Gomez-Chova | Multimodal classification of remote sensing images: A review and future directions[END_REF].

In summary, literature reports plethora of techniques available for satellite image processing be it for unimodal or single sensor images, multimodal or multisensor satellite images, or satellite time series images. Overall, it is observed that development of techniques that can efficiently exploit different characteristics and/or different modalities offered by satellite images is tedious and crafty. On top of it, keeping track of more resourceful satellite images every day and improvising the techniques accordingly is not an easy task at all. Based on these observations, it is safe to say that any novel technique, approach, method, or model should prefer to possess qualities such as high performance, universal applicability framework, scalability, automatability, and generalization. Presence of these attributes will help any technique, approach, method, or model to keep up with the advancements in satellite images.

Motivation: Deep Learning in Satellite Image Processing

In this direction of improvements in satellite image processing techniques, deep learning with deep neural networks have shown substantial potential [START_REF] Penatti | Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?[END_REF]. In the recent years, many satellite image based applications have used deep neural networks and their popularity is still growing [START_REF] Ma | Deep learning in remote sensing applications: A meta-analysis and review[END_REF], [START_REF] Kamilaris | Deep learning in agriculture: A survey[END_REF]. The deep learning theory is a universally applicable approach which can map multiple levels of information to represent complex relationships between data [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF]. The deep neural network models are high performance, scalable, automatable, and can learn directly from raw data (less feature handling) [START_REF] Lecun | Deep learning[END_REF]. These are the abilities researchers were looking towards from new techniques as discussed in section 1. For example, Convolutional neural networks or CNNs are recently the most widely used deep network models in satellite image based land cover classification [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF]. CNNs have the ability to extract complex features directly from data [START_REF] Lecun | Deep learning[END_REF]. Another advantage of focusing on developing CNNs for satellite image processing is that CNNs can directly utilize multi-sensor high dimensional satellite images with efficient performance [START_REF] Kussul | Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[END_REF]. Unlike the conventional data fusion methods, this reduces the need of fusing the multisensor images together before using it further.

Furthermore, CNNs and recurrent neural networks or RNNs are the most recent and trending deep neural network frameworks for satellite time series image based agriculture crop monitoring applications. Since crop classification is the most challenging as well as important task in many crop monitoring applications, both CNN and RNN models have been developed in, standalone and synergistic modes for agriculture crop classification by many researchers [START_REF] Pelletier | Temporal convolutional neural network for the classification of satellite image time series[END_REF], [START_REF] Rubwurm | Temporal Vegetation Modelling Using Long Short-Term Memory Networks for Crop Identification from Medium-Resolution Multi-spectral Satellite Images[END_REF]. The aim of these models is to capture the spatial attributes of a particular crop type with CNNs and/or its temporal attributes with RNNs. The above mentioned research gaps are addressed (to some extent) in this thesis and various novel DNN models are sought to be proposed that focus on exploiting the information present in satellite images more carefully.

Thesis Objective

The main objective of this thesis is; the study and critical analysis of the performance of conventional CNN and RNN models and further development of the novel CNN and RNN models to exploit the spectral and/or spatial and/or temporal information present in unimodal/single senor or multimodal/multisensor satellite images. Special focus during the studies is to develop novel deep neural network (DNN) models (common term for CNN and RNN based models) that can achieve better generalization performance in mixed class scenarios.

In the attempt, four tasks are set to forward the stated objective. The tasks are based on; type(s) of satellite image(s) used, application for which the task is realized, multi-modality images involvement, and presence of time series information. The tasks and sub-tasks set to be carried out are:

Task 1: Development of CNNs and polarization signatures based novel approach for improved land cover classification with PolSAR satellite images.

Sub-tasks:

 Critical analysis of polarization signatures as potential feature for PolSAR image based land cover classification  Development of novel class boundary estimation algorithm for optimal utilization of polarization signature based features to improve land cover classification accuracy.  Development of novel CNNs based approach that directly utilizes polarization signatures to achieve land cover classification.

Task 2: Development of a novel approach using polarization signatures and CNNs for detection of human settlements affected during floods.

Sub-tasks:

 Development of a polarization signatures and CNN based model to extract builtup areas from PolSAR image.  Development of a detection approach to detect villages from built-up areas using image morphology and apriori information.  Mapping flood affected villages using image morphology.

Framework of the Research

Although each task follows a separate methodology, all the four tasks can be consolidated into a generic framework which is shown in Figure 1.9. The generic framework encompasses majorly five components namely: problem or objective identification, experiment setup, data preprocessing, model development and training, and model deployment, results, and discussion.

The order might differ for different tasks based on whether the solution is empirical or analytical.

Problem or objective identification

In this part of the research, either a problem at hand is explored or an objective is laid out. In the former, the challenges with current applications are critically analysed first, then new technologies, materials, and methods are searched in order to find motivation to solve the problem. Task 1, 3, and 4 listed in section 1.6 are set out in this manner. Whereas in the latter, first an objective or vision is set and then the goal is to attain the objective with best efficiency.

Task 2 is set out in this manner. In both the cases, knowledge of the background and extensive literature review is a prerequisite. Various challenges and objectives identified during the present research are listed in section 1.4 and 1.5 respectively.

Experiment setup

Once either the problem is identified or an objective is set, the next step is to set up an experiment.

Data preprocessing

The data considered in the present research are PolSAR and multispectral satellite image.

Preprocessing PolSAR data involves calibration, speckle filtering, and terrain correction whereas preprocessing multispectral data involves atmospheric correction, cloud removal, and resampling. Both the datasets are preprocessed before to be used for various applications. Data preparation such as rescaling and stacking of PolSAR and multispectral images for multisensor data analysis is also carried out under this component. Further, the various CNNs and RNNs models take input data in particular formats, the pre-processed data is restructured accordingly under this component.

Model development and training

The

Model deployment, results, and discussion

The Among other applications, land cover classification is one of the most popular application in the geoscience and satellite image processing community. Numerous techniques have been successfully developed for land cover classification with unimodal satellite images over the years [START_REF] Joshi | A review of the application of optical and radar remote sensing data fusion to land use mapping and monitoring[END_REF]. In context, DNN models have also been successful in satellite image based land cover classification applications.

Satellite SAR images are recently one of the most popular and most widely used modality among all modalities of satellite images. As first attempts in SAR, deep learning based methods have been adopted for a variety of tasks, including land cover or land cover classification [START_REF] Parikh | Classification of SAR and PolSAR images using deep learning: a review[END_REF], [START_REF] Lv | Urban land use and land cover classification using remotely sensed sar data through deep belief networks[END_REF], object detection [START_REF] Chen | SAR target recognition based on deep learning[END_REF], parameter inversion [START_REF] Wang | Sea Ice Concentration Estimation during Melt from Dual-Pol SAR Scenes Using Deep Convolutional Neural Networks: A Case Study[END_REF], despeckling [START_REF] Wang | SAR image despeckling using a convolutional neural network[END_REF], and specific application in InSAR [START_REF] Anantrasirichai | Application of Machine Learning to Classification of Volcanic Deformation in Routinely Generated InSAR Data[END_REF]. attempts to carry out HSI classification using 1D-CNN that contains four layers: one convolutional layer, one pooling layer and two fully connected layers. The study in [START_REF] Mei | Integrating spectral and spatial information into deep convolutional Neural Networks for hyperspectral classification[END_REF] exploits a similar 1D-CNN to classify hyperspectral images but relies on using additional techniques such as batch normalization and dropout. Among the three typical DL models (deep belief networks, CNNs, and stacked autoencoders), stacked autoencoders and deep belief networks take the data represented in a vector form as input, thereby satisfying the need to extract deep spectral feature from spectral vectors. In fact, stacked autoencoders and deep belief networks were introduced prior to CNNs to perform HSI spectral feature classification.

Land cover classification using

Spectral feature classification methods just take account of spectral information and fail to make use of spatial information. Spatial information (also termed contextual information) helps reveal useful relationships between adjacent pixels. Making full use of spatial information such as shape or size can significantly improve classification accuracy and efficiency. The most significant advantage of 2D-CNNs is that they are suitable to effectively learn features from raw 2D images directly. However, as hyperspectral image data contains more than 100 spectral bands, direct utilization of a 2D-CNNs to extract deep features from raw hyperspectral image data require a mass of learnable kernels. The learnable kernels are hard to train and will increase computational overheads. A randomized principal components analysis (PCA) is employed in [START_REF] Makantasis | Deep supervised learning for hyperspectral data classification through convolutional neural networks[END_REF] to condense the spectral depth of hyperspectral image data in order to improve computation cost. They then applied a 2D-CNN to extract deep features from the reduced spectral space before classification. A similar study is proposed in [START_REF] Yue | A deep learning framework for hyperspectral image classification using spatial pyramid pooling[END_REF] with the difference in consideration of principal components and window size. They used only 3 principal components and a window one is to extract spectral and spatial features separately first, and then to combine these features;

the other is to extract deep spectral-spatial features from hyperspectral image directly. The former strategy is employed in [START_REF] Yue | A deep learning framework for hyperspectral image classification using spatial pyramid pooling[END_REF], [START_REF] Liang | Hyperspectral imagery classification using sparse representations of convolutional neural network features[END_REF], [START_REF] Zhang | Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network[END_REF] whereas the later strategy is employed in [START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF] and [START_REF] Li | Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network[END_REF].

In case of multispectral satellite images or satellite images with low spectral depth, per pixel classification does not yield significant results with deep learning techniques. However, such images, when used in patch mode yield new applications and results. One application of low spectral satellite images with deep learning is scene or patch classification. Scene classification aims to automatically assign a semantic label to each scene image [START_REF] Hu | Unsupervised feature learning via spectral clustering of multidimensional patches for remotely sensed scene classification[END_REF]- [START_REF] Xia | AID: A benchmark data set for performance evaluation of aerial scene classification[END_REF].

Here, a scene image usually refers to a local image patch manually extracted from large-scale high-resolution aerial or satellite images that contain explicit semantic classes (e.g., residential area, commercial area, etc.). Due to high resolutions in such data, different scene images may contain the same types of objects or share similar spatial arrangements between objects. For example, residential and commercial areas may have buildings, roads, and trees, but they belong to two different build-up areas. Indeed, significant variations potentially existing in scene images in the spatial arrangements and structural patterns make scene classification a considerably challenging task [START_REF] Zhu | Deep learning in remote sensing: A comprehensive review and list of resources[END_REF]. Particular to land cover classification, a patch-based CNN which was more suitable than the pixel-based CNN for medium images is proposed in [START_REF] Sharma | A patch-based convolutional neural network for remote sensing image classification[END_REF].

In summary, enough evidence suggests the significance of using DL and DNNs for Multispectral images are mostly used in scene or patch classification.

However, certain aspects still need attention or are worth analysing. One aspect is, direct exploitation of PolSAR satellite images via DNN models for applications such as land cover classification. In particular, the use of 2D-CNNs to directly extract complex spatial feature from PolSAR data is motivating. More research gaps in DNN based unimodal satellite image processing is summarized in section 1.5.

Brief Review of Recent Developments in DNNs based Multisensor Satellite Image Processing

DNNs are not restricted to handling unimodal datasets. Instead, exploiting multisensor or multimodal data is one of the advantages of DNNs [START_REF] Gomez-Chova | Multimodal classification of remote sensing images: A review and future directions[END_REF]. It is important to note here that various satellite image applications require to focus on one, or more characteristics (spatial, spectral or temporal) of satellite images. Usually, a single sensor or unimodal satellite images cannot provide high-quality information in all three aspects. Therefore, the use of multisensor or multimodal satellite images for many applications is not uncommon. Pansharpening is the task of improving the spatial resolution of multispectral images by fusing these with high spatial resolution image such as panchromatic image. Traditionally, the field was dominated by works fusing multispectral data with panchromatic bands, but more recently it has been extended to hyperspectral images in [START_REF] Loncan | Hyperspectral pansharpening: A review[END_REF] and thermal images in [START_REF] Fasbender | Support-based implementation of bayesian data fusion for spatial enhancement: applications to ASTER thermal images[END_REF]. A first attempt in DNN based pansharpening is reported in [START_REF] Zhong | Remote sensing image fusion with convolutional neural network[END_REF] where the authors used a shallow [START_REF] Volpi | Dense semantic labeling of subdecimeter resolution images with convolutional neural networks[END_REF]. In contrast to stacking the input images directly, intermediate features (using a feature extraction process) have also been stacked and been used for classification [START_REF] Geng | Deep supervised and contractive neural network for sar image classification[END_REF]. The models proposed in [START_REF] Geng | Deep supervised and contractive neural network for sar image classification[END_REF] has been used in the classification of SAR images via learning through a stack of spatial features. Similarly, in [START_REF] Tuia | Multiclass feature learning for hyperspectral image classification: sparse and hierarchical solutions[END_REF], the authors present a hierarchical feature learning process that learns combinations of

Brief Review of Recent Developments in DNNs based Satellite Time Series Image Processing

This

Conclusions

This chapter presented a brief review on developments in DNN models and methodologies for satellite image processing, analysis, and applications. The review included unimodal satellite images, multisensor satellite images, and satellite time series images. In context to applications of satellite images, focus is kept on classification may it be land cover classification or crop classification. However, other applications are also touched upon. The review consolidates to certain observations  Most of the novel DNN models and methodologies have been explored for SAR, hyperspectral, and multispectral satellite images.  Spatial or contextual information present in satellite images is given preference in contrast to spectral information (in context to multisensor satellite images).  Both parallel and cascade DNNs architectures have been developed. Parallel architectures extract spatial and spectral information simultaneously with two different DNN modules whereas the cascade architectures used the same module to extract the spatial and spectral information (one after the other). Critical analysis of these architectures is worth investigating and interesting.  CNNs are more popular than other DNN models while handling spectral or spatial features of satellite images are whereas RNNs and 1D-CNNs are more popular while handling temporal features of satellite images. However, integration of CNN and RNN models for time series satellite image processing might yield high performance and consistency and also could lead to development of a general framework.  The main focus of most studies is on classification tasks for e.g. land cover classification, scene classification or, object detection. However, several other applications are worth mentioning including fusion, segmentation, change detection, and registration.

Prominent research gaps are identified and are briefly listed in chapter 1 (refer section 1.5).

Introduction

Use of polarimetry for radar applications has a long history. PolSAR target decomposition based techniques are used by various researchers for many earth observation applications such as land cover classification [START_REF] Singh | Seven-component scattering power decomposition of PolSAR coherency matrix[END_REF], [START_REF] Lee | Polarimetric radar imaging: From basics to applications[END_REF]- [START_REF] Cloude | A review of target decomposition theorems in radar polarimetry[END_REF]. A typical land cover scene includes both, natural (e.g. Observing the enlarged segments in Figure 3.1(c), it is evident that both the land cover classes are showing similar backscattering behaviour (high backscattering response in HV channel) and [START_REF] Turkar | Effect of different target decomposition techniques on classificatio accuracy for polarimetric SAR data[END_REF] found similar observation. Literature indicates few studies that aim to differentiate class pairs such as bare soli-agriculture or dense tall vegetation-rural settlements. Attempts are made in [START_REF] Ratha | A PolSAR scattering power factorization framework and novel roll-invariant parameters based unsupervised classification scheme using a geodesic distance[END_REF], [START_REF] De | A novel technique based on deep learning and a synthetic target database for classification of urban areas in PolSAR data[END_REF], [START_REF] Ratha | A generalized volume scattering model-based vegetation index from polarimetric SAR data[END_REF] and more recently in [START_REF] Ratha | A PolSAR scattering power factorization framework and novel roll-invariant parameters based unsupervised classification scheme using a geodesic distance[END_REF], [START_REF] Bi | An active deep learning approach for minimally supervised PolSAR image classification[END_REF] to segregate oriented built-up areas from tall vegetation using different decomposition and factorization features. However, the challenge still exists and new features needs to be explored which might have potential to segregate these class pairs and provide good land cover classification results.

In the context, PolSAR polarization signatures can be potential candidates. PolSAR polarization signatures (PSs) can characterize the scattering behaviour of target under study or in this case land cover under observation effectively and therefore can be considered for land cover classification applications. These signatures are 2D representations of scattering characteristics of a target. A method for generation of PSs with different polarization bases is proposed in [START_REF] Jafari | A new method for land cover characterization and classification of polarimetric SAR data using polarimetric signatures[END_REF].

They reported better classification accuracy than object based supervised Wishart classification method. Similar work proposed in [START_REF] Bielecka | SAR images analysis based on polarimetric signatures[END_REF] compared observed PSs to standard target or reference PSs based on norm and further, a pattern recognition using Kohonen networks was performed for LC classification in [START_REF] Bielecka | SAR images analysis based on polarimetric signatures[END_REF]. Results show that PSs not only allow to identify coherent scatters but also scattering mechanisms. Till date, few approaches have been developed for analysing PSs for land cover (LC) classification purposes [START_REF] Sadjadi | Improved target classification using optimum polarimetric SAR signatures[END_REF]- [START_REF] Strzelczyk | Identification of coherent scatterers in SAR images based on the analysis of polarimetric signatures[END_REF]. Therefore, PSs are considered here to be analysed as potential PolSAR features for land cover classification in mixed land cover scenario. In summary, this chapter presents a study in which PolSAR PSs are considered as potential PolSAR features for land cover classification.

However, the challenge with exploiting PSs for land cover analysis is its contextual nature. It means that PSs pixels are related with its neighbourhood. But, the current popular techniques that directly exploit PSs such as the normalized signature correlation mapper (NSCM)

and normalized Euclidean distance (NED) based classifiers requires the transformation of the two dimensional PSs into a one dimensional vector before classification. This results in a loss of contextual information present in the PSs. There is a need to develop classifiers which can exploit the contextual information present in the PSs. In the direction of novel classifiers, authors in [START_REF] De | A novel technique based on deep learning and a synthetic target database for classification of urban areas in PolSAR data[END_REF] and [START_REF] Bi | An active deep learning approach for minimally supervised PolSAR image classification[END_REF] Where σ represents backscattering coefficient, suffix i and j represent transmit and received combinations respectively. K is the Kennaugh matrix, which provides received power, given the polarization characteristics of the receiving antenna [START_REF] Kennaugh | Effects of type of polarization on echo characteristics[END_REF]. It is also known as the Stokes

Theoretical Background

Convolutional neural networks

CNNs are a popular class of deep feedforward artificial neural networks proposed in various studies [START_REF] Goodfellow | Convolutional Neural Networks[END_REF]- [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF]. Building blocks of a conventional CNN are: a convolutional layer, a pooling layer, a dropout layer, a fully connected layer, and a loss layer. A generic arrangement of the layers is discussed in [161] and [START_REF] Albelwi | A framework for designing the architectures of deep Convolutional Neural Networks[END_REF].

Imagine a CNN with M layers, the output state vector of the 𝑚 th layer is denoted by 𝑶 𝑚 , where  

mm mm g     W O O b (3.2)
Where,  indicates convolution operation and pool denotes a spatial aggregation, as defined in [START_REF] Boureau | A theoretical analysis of feature pooling in visual recognition[END_REF]. The matrix 𝐖 𝑚 , encodes the weights of the links connecting the (𝑚 -1) th convolutional layer to the 𝑚 th layer. The vector m b denotes a bias term. Subsequently, a non- linear pointwise activation operation g(.) is performed. Finally, a pooling layer is used which helps in selection of dominant features.

Several convolutional and pooling layers can be stacked to form a hierarchical feature extraction architecture. The resultant features are then combined to form a one-dimensional feature vector to be taken as input by the fully connected layer. The fully connected layer acts as a typical multi perceptron layer. The last layer is usually a linear (for regression scenarios) or softmax (for classification scenarios) layer, with the number of neurons matching the number of output variables or classes desired. This layer also specifies how training penalizes the deviation between the predicted and true labels and is normally the final layer.

The weights, {𝐖 1 , 𝐖 2 , … 𝐖 M } and the biases, {𝒃 1 , 𝒃 2 , … 𝒃 M } of the CNN compose the model parameters, which are iteratively and jointly optimized via overall accuracy maximization over the training set. Similar networks can be realized for 1D or 3D CNNs accordingly.

The whole network still expresses a single differentiable score function from the raw image pixels on one end to class scores at the other. 2D CNN architectures make the explicit assumption that the inputs are images. This allows the user to encode certain properties into the architecture. These make the following functions more efficient to implement and it also vastly reduce the amount of learnable parameters in the network. This property is explored and utilized for PSs in this study. CNNs with deep learning have found their application in PolSAR data as well [START_REF] Zhu | Deep learning in remote sensing: A comprehensive review and list of resources[END_REF].

Experiment Setup

Study area

The area selected for study includes Roorkee city at the foothills of the great Himalayas of the northern India and its neighbouring region. The area extends from [START_REF] Rao | Generation of dual beams from spherical phased array antenna[END_REF]

Dataset

The dataset used for study is the ALOS PALSAR-2 L band synthetic aperture radar (SAR) data.

PALSAR-2 is a polarimetric SAR sensor on-board the ALOS-2 spaceborne satellite. SAR data collected by PALSAR-2 is fully polarimetric with information in four polarization states i.e. HH, HV, VH, and VV. PALSAR-2 sensor and product specifications are listed in Table 1.8. The data acquisition date is March 13, 2015. The data is provided by JAXA, Japan for research purposes.

A subset of the whole image, nearly 400 km 2 is considered for algorithm development. ALOS PALSAR-2 data has been reported to be utilized successfully for many land cover applications such as geology mapping [START_REF] Pour | Structural geology mapping using PALSAR data in the Bau gold mining district, Sarawak, Malaysia[END_REF], land deformation monitoring [START_REF] Yastika | Monitoring of long-term land subsidence from 2003 to 2017 in coastal area of Semarang, Indonesia by SBAS DInSAR analyses using Envisat-ASAR, ALOS-PALSAR, and Sentinel-1A SAR data[END_REF], and mostly in land cover classification [START_REF] Jain | An optimal selection of probability distribution functions for unsupervised land cover classification of PALSAR-2 data[END_REF], [START_REF] Mittal | A critical analysis of em based fusion of different polarization data for effect on land cover classification[END_REF].

Ground truth

Preprocessing

The PALSAR-2 data considered in the study is a single look complex (SLC) data. Preprocessing is required before using the data for the study. The steps involved in preprocessing are similar to as discussed in [START_REF] Mishra | Land cover classification of PALSAR images by knowledge based decision tree classifier and supervised classifiers based on SAR observables[END_REF]. A Gamma MAP filter of window size 33  for the purpose of speckle removal is used as this filter retains polarimetric signature characteristics of SAR image [START_REF] Nezry | On the preservation of polarimetric signatures and polarimetric texture signatures by fully polarimetric map filters[END_REF],

and [START_REF] Mishra | A statistical-measure-based adaptive land cover classification algorithm by efficient utilization of polarimetric SAR observables[END_REF]. This filter also removes speckle noise from gamma distributed scenes such as forested areas and agricultural lands whereas keeping the original pixel values for non-gamma distributed scenes such as water. The window size is finalized on the basis of spatial resolution of input image and application. Geocoding is done using SRTM 3 arc-second global DEM model. Other tasks like radiometric calibration and terrain correction are performed similar to [START_REF] Mishra | Land cover classification of PALSAR images by knowledge based decision tree classifier and supervised classifiers based on SAR observables[END_REF]. The output of this step is a pre-processed backscattering or S-matrix. Four features namely HH, HV, VH, and VV are obtained. Since, HV and VH are similar due to the reciprocity theorem, only HH, HV, and VV are used further in the current study.

Sub Task 1: Polarization Signatures: Critical Analysis as Potential PolSAR Features for LC Classification

This section discusses the PSs based feature computation process and its utilization for land cover (LC) classification. Section 3.4 covers the sub task 1 of the study. Flowchart shown in Figure 3.4

summarizes the process of, computation of PSs, and its critical analysis as potential PolSAR features.

Polarization signatures computation: Observed and standard

Due to complex geometrical structure, natural targets (land covers) show complex scattering response [START_REF] Cao | Extraction of information on geology hazard from multi-polarization SAR images[END_REF], [START_REF] Maurya | An approach to use polarimetric signature for land cover classification[END_REF]. Scattering responses from natural targets are represented as a combination of simple point targets scattering responses which are well defined [START_REF] Phruksahiran | Backscatter power measurement of canonical targets under GBSAR environment[END_REF]. In this study, PSs from

Polarization signature correlation features or PSCFs

A land cover (e.g. bare soil) behaves as a complex mixture of different standard scatterers or standard targets and therefore its PS is difficult to interpret. Therefore, at first, there is a need to find correlations between polarimetric/ PSs of land covers and PSs of standard targets. These correlations indicate the degree of similarity between the two scatterers.

Correlation coefficient is a value, which varies from +1 and -1 and it is calculated to represent the linear inter-dependence of two variables or sets, of data. For two variables, say x and y, the correlation coefficient is computed using equation (3.3).

CC S xy SS x y  (3.3)

In this section, correlation is computed between each image pixel-standard target PS pair. Here, x is the observed (image pixel) PS, y is the standard target PS, S

x is standard deviation in x, S y is standard deviation in y, S xy is co-variance between x and y, and CC is correlation coefficient between x and y.

The following pairs are considered between which correlation is computed. A critical analysis is essential at this stage in order to identify the best feature for segregating a particular land cover class. For the same, a separability index based feature selection process is carried out in section 3.4.3.

Assessment and critical analysis of PSCFs

In this section, each PSCF obtained in section 3.4.2 are analysed and discussed. Class separation for each PSCF is evaluated by computation of class separability index (SI) proposed in [START_REF] Cumming | Feature utility in polarimetric radar image classification[END_REF].

Nomenclature used in section 3.4.2 is used in this section also.

The separability measure is computed using equation 3.4. For a given feature, , SI ij is the measure of separability between class i and j. In order to measure SI between different land cover/class pairs, correlation values are collected for all the ROI points (for Roorkee study area) mentioned in section 3.3.3. The SI values obtained for all the PSCFs as features are provided in Table 3.2.

Separability assessment is also conducted for decomposition features obtained using

Cloude decomposition [START_REF] Cloude | An entropy based classification scheme for land applications of polarimetric SAR[END_REF], modified Yamaguchi four component decomposition or m-Y4D [START_REF] Bhattacharya | An adaptive general fourcomponent scattering power decomposition with unitary transformation of coherency matrix (AG4U)[END_REF], generalized Freeman Durden decomposition or GFD [START_REF] Sen Lee | Generalized polarimetric model-based decompositions using incoherent scattering models[END_REF], and H-A-Alpha quad pol decomposition or H-A-Alpha [START_REF] Pottier | Application of the H/A/alpha polarimetric decomposition theorem for unsupervised classification of fully polarimetric SAR data based on the wishart distribution[END_REF] methods. SI values for class pairs are shown in Table 3.3.

A comparison between the PSCF features and the considered decomposition features on the basis of SI value for selected class pairs is done. The selected class pairs are, TV to built-up, TV to SV, and BS to Water. These three class pairs are difficult to segregate in a mixed class scenario and therefore are considered for comparison. From Table 3.2 and Table 3.3, it is evident that for TV to built-up class pair, corr_co_Di is the most significant feature (0.7125), followed by C 1 (0.439), H 3 (0.408), F 3 (0.372), and Y 1 (0.250). For TV to SV class pair, Y 2 is the most significant feature (0.502) followed by corr_co_FP (0.4657), C 2 (0.269), H 1 (0.364), and F 2 (0.337). For BS to Water class pair, F 2 is the most significant feature (1.943) followed by C 3

(1.499), Y 3 (1.392), corr_co_HD (0.8207), and H 1 (0.456). From the comparisons, it can be inferred that there is no clear indication of a set of features (from the same method) that are most significant for all class pairs. In short, there is no clear winner. Also, it is evident that the proposed PSCFs can be potential features to segregate land cover classes in mixed class scenarios.

Land cover classifications are performed using the PSCFs and other considered decomposition features in order to provide a comparative assessment based on LC classification performance. The classifier considered to be used with these features is support vector machines 3.4. It is evident from Table 3.4 that LC classification with PSCFs provides higher classification accuracy and kappa coefficient than the decomposition based features.

It is also evident from Table 3.4 that even if the proposed PSCF features performed better than the compared features, the classification accuracy can be further improved. For the proposed PSCF-SVM classification strategy, built-up (marker A) and TV (marker B) have low user accuracies (69.8% and 68.4% respectively) compared to other classes.

The classification performance can be further improved using more optimal class boundaries. Decision tree (DT) classifier provides the freedom to set optimal class boundary values during classification. Therefore, an optimal class boundary based DT classifier is considered and developed in this study. Details on the development of the optimal class boundary estimation algorithm are discussed in section 3.5 of this chapter. Also, to reduce the computational complexity of the proposed classifier, feature selection is carried out with PSCFs.

It is clear from Table 3.2 that built-up can be segregated from all other classes (except tall veg) using corr_co_Di as a feature. Corr_co_Di is showing highest SI value amongst all the PSCFs. Further, it can be observed that (bare soil + water) are clearly separable from other classes (except short vegetation) using corr_co_FP as feature. With SI value of 0.83, bare soil and water can be separated using corr_co_HD. Water is also easily separable from other classes using corr_co_VD. Since both corr_co_HD and corr_co_VD can separate water, either or both of them can be used. It is evident from Table 3.2 that the cross-polarization PSCFs do not contain any complementary information for segregation process and hence are not considered for classification. A water canal (Ganga canal) is visible in Figure 3.1(b). Water in the canal is not identifiable in corr_co_HD or corr_co_VD (refer to Figure 3.8(a) and Figure 3.8(b)). A complementary feature is required which can address the issue. In Figure 3.1(a), the water canal is appearing blue (VV polarimetric band). Hence, VV can be used for classifying water in the canal. It is also evident from Figure 3.1(a), that only VV is not a sufficient choice for the classification of water as "bare soil" is not separable from water using only VV as feature. Hence, corr_co_HD or corr_co_VD, and VV polarimetric band are used. In summary, PSCFs selected for segregating a particular class based on significance are listed in Table 3.5.

Table 3.5 indicates that no PSCFs have been selected for segregating vegetation (both SV and TV) because of low SI values for the class pair over all the PSCFs. Observing Table 3.2, it is clear that the best choice among all the PSCFs for segregating tall and short vegetation is corr_co_FP. In Table 3.2, corr_co_FP is having the highest SI value among all PSCFs for both short and tall vegetation. Table 3.6 provides the updated and final list of selected PSCFs to be used during classification. An alternate analysis of all the PSCFs as potential features for decision tree based land cover classification is presented in [START_REF] Phartiyal | Optimal use of polarimetric ssignature on PALSAR -2 data for land cover classification[END_REF]. Scatter plot and OTSU multi-thresholding methods are used for separability assessment and similar results are reported. The class boundary values obtained using methods in [START_REF] Phartiyal | Optimal use of polarimetric ssignature on PALSAR -2 data for land cover classification[END_REF] are rigid and do not change with the change in the PSCF statistics which in turn depends on the input PolSAR image. The changes captured in a PolSAR image can be via changes in landscape seasons or man-made. These changes needs to be considered while calculating class boundary thresholds. Therefore, in this study, a input image statistics based adaptive class boundary estimation approach is proposed to improve the results from [START_REF] Phartiyal | Optimal use of polarimetric ssignature on PALSAR -2 data for land cover classification[END_REF]. Development of proposed approach is discussed in section 3.5.

Sub Task 2: Optimal Class Boundary Estimator based Decision Tree Approach for LC Classification

The approach is developed and presented in a sequential manner. Section 3.5 is broken down to four sub-sections and each sub-section is discussed in detail. Details of dataset are provided in section 3.3.2. Flowchart in Figure 3.9 summarizes the proposed algorithm.

PSCF statistics

Making the algorithm adaptive means the algorithm estimates class boundaries based on PSCF statistics (mean, standard deviation, etc.). Statistics for selected PSCFs i.e. mean (M) and standard deviation (S) are computed and are shown in Table 3.7. These statistics are used for the computation of class boundaries.

Relationship between class boundaries and corresponding PSCF statistics

A relationship is formulated between class boundaries and corresponding PSCF statistics (the one selected for segregating that particular class (refer Table 3 n (i = 1 to 10).

Relating classification accuracy and class boundaries: Empirical modelling

In order to achieve the best overall accuracy (OA) with given feature space, the values of i n (i = 1 to 10) should be optimized. To get optimal i n (i = 1 to 10) values, an empirical function needs to be developed relating OA to i n (i = 1 to 10) which can be optimized. OA is defined in equation (3.10). total no of correctly classified pixels OA = Total no of pixels tested (3.10) PSCFs are used based on Table 3 Similar procedure is applied for all other classes and corresponding empirical relations are obtained. A coefficient of determination (R2 ) value is also provided in Table 3.8. An optimization method can now be used to provide optimal i n (i = 1 to 10) values that can maximize OA under constraints.

Optimizing the empirical model

Genetic algorithm (GA) [START_REF] Goldberg | Genetic algorithms with sharing for multimodal function optimization[END_REF] is used as an optimization tool for finding optimal i n (i = 1 to 10) values. GA is a metaheuristic inspired by the process of natural selection that belongs to the larger class of evolutionary algorithms [START_REF] Bansal | Self-adaptive artificial bee colony[END_REF]. Genetic algorithms are commonly used to generate The choice of GA as optimization method is because of the probability of more than one maxima/minima. Conventional methods like first derivative could be faster but can suffer from local maxima/minima problem.

The objective function of the optimization problem at hand is (refer equation (3. After optimizing equation 3.13, the optimal i n (i = 1 to 10) values obtained are shown in Table 3.9. These optimal i n (i = 1 to 10) values are used to obtain class boundaries which in turn will be used in decision tree classifier in section 3.5.5.

Optimal class boundaries based decision tree classifier

The optimal i n (i = 1 to 10) values obtained in section 3. The priority of a class to be segregated first is evaluated on how high is its SI value.

Results and discussion

The optimal class boundary estimator based decision tree classifier obtained in section 3.

Sub Task 3: Direct Utilization of PolSAR Polarization Signatures via Convolutional Neural Networks for LC Classification

The current section presents utilization and analysis of advanced pattern classification methods for PolSAR land cover classification. Convolutional neural networks (CNNs) are recently the most popular, efficient, and powerful techniques in image feature extraction and classification applications. Their parameter sharing and local connectivity properties makes them best tools for extracting information from images [START_REF] Lecun | Deep learning[END_REF]. CNNs in combination with deep learning technology are being widely used in PolSAR data applications [START_REF] Zhu | Deep learning in remote sensing: A comprehensive review and list of resources[END_REF]. In this study, CNNs are utilized to extract features from PSs and perform land cover classification. A CNN based PSs pattern classification technique is proposed here which considers the influence of the neighbourhood of a pixel and thereby utilizing the PSs directly.

Two dimensional convolution over polarization signatures: The concept

PSs-CNN model architecture design and training

A

Results and discussion

Overall Results and Discussion

Comparison between the two novel approaches developed in this study i.e. the optimal class boundary estimator based DT classifier and the PSs-CNN is made. Classification results from the two novel classifiers and the RBF-SVM are summarized in Table 3 Therefore, the study presented in this chapter concludes that; i) PSs have shown significant potential as PolSAR features for land cover applications, ii) segregation of built-up class and tall vegetation class in PolSAR data can be improved by using optimal class boundary estimators with decision trees for classification, and iii) 2D-CNNs have shown potential as classifiers with PSs for LC classification in terms of ease of implementation, and scalability.

However, the potential of PSs-CNN model needs to be further investigated for other important PolSAR data based earth observation applications.

Introduction

Floods are the most frequent, widespread, and impactful disasters in India. A major population of India lives in towns and villages and during the floods, rescue operations to these affected areas are both difficult and challenging. Rapid and reliable management and mitigation is key to minimize the loss of human lives and property during a flood disaster. Accurate and updated information of flood affected area can lead to an efficient disaster mitigation plan. In the context, satellite images based flood disaster monitoring is one of the most helpful and reliable technology to counter these flood disasters. Especially, synthetic aperture radar (SAR) imaging has proved to be a very successful and promising technology for both, monitoring floods or, planning rescue operations in flood affected areas [START_REF] Altan | Spatially supported disaster management: Introduction to the special issue "gI for disaster management[END_REF], [START_REF] Giustarini | A change detection approach to flood mapping in Urban areas using TerraSAR-X[END_REF]- [START_REF] Jennifer | Integration of SAR and multi-spectral imagery in flood inundation mapping-a case study on Kerala floods 2018[END_REF]. Polarimetric synthetic aperture radar (PolSAR) imaging sensors are developed to capture the changes in polarization states introduced by the interaction of EM waves with earth surface features (land covers). Among many other applications, PolSAR imaging has also proved its significance and uniqueness in flood monitoring and mitigation [START_REF] Giustarini | A change detection approach to flood mapping in Urban areas using TerraSAR-X[END_REF], [START_REF] Pradhan | A new semiautomated detection mapping of flood extent from TerraSAR-X satellite image using rule-based classification and taguchi optimization techniques[END_REF]- [START_REF] Dasgupta | Optimized GLCM ¬ based texture features for improved SAR ¬ based flood mapping[END_REF] [START_REF] Yamaguchi | Polarimetric SAR Imaging[END_REF]. However, the PolSAR data has been transformed, represented, and utilized in different other forms and bases in various flood monitoring studies [START_REF] Plank | Mapping of flooded vegetation by means of polarimetric sentinel-1 and ALOS-2/PALSAR-2 imagery[END_REF], [START_REF] Silvana | Adaptive sar image processing techniques to support flood monitoring from earth observation data[END_REF]. Each PolSAR data representation has its advantages and disadvantages and its selection depends on application and available processing methodologies.

Monitoring human settlements in the rural parts of India during floods is difficult even with PolSAR data. The built-up structures (e.g. houses) in these human settlements, especially villages of rural India are closely packed and randomly oriented. Therefore, these villages show volume scattering response along with double bounce scattering response in a PolSAR image.

The mixed scattering response from these villages hinders their segregation from tall vegetation in a PolSAR image. This challenge is also discussed in chapter 3 (refer section 3.1). Therefore, the detection and mapping of these villages during a flood using PolSAR image is challenging as there is high risk of false alarms, or missing out a flood affected village during rescue operations. The current study considers PolSAR polarization signatures (PSs) as features to improve the detection and mapping of villages that are affected by flood. Polarization signature (PS) is a two dimensional representation relating backscatter response from a target at a particular polarization state. Polarization signatures or PSs are introduced in [START_REF] Kennaugh | Effects of type of polarization on echo characteristics[END_REF] and later on represented graphically by many researchers in different ways [START_REF] Van Zyl | Imaging radar polarization signatures: Theory and observation[END_REF]. PSs have been proved to be useful for characterization of land covers such as built-up, forest, or water [START_REF] Jafari | A new method for land cover characterization and classification of polarimetric SAR data using polarimetric signatures[END_REF]. A comparison of PSs based features and decomposition based features for land cover characterization is also presented in section 3.4. The study demonstrates the significance of the utilization of PSs for land cover characterization and classification in mixed land cover scenarios which are similar to the scenario at hand in the current study. Therefore, the utilization of PSs as PolSAR features is considered in the current study. A detailed description on PSs is provided in section 3.2.1.

Moreover, it is also evident from most PSs based land cover studies that these studies utilized distance/correlation measures between standard target PSs and observed PS for detection, identification, and classification of land covers [START_REF] Jafari | A new method for land cover characterization and classification of polarimetric SAR data using polarimetric signatures[END_REF], [START_REF] Strzelczyk | Identification of coherent scatterers in SAR images based on the analysis of polarimetric signatures[END_REF], [START_REF] Maurya | A critical analysis of polarimetric signatures on PALSAR 2 data for land cover classification[END_REF] Normalized signature correlation mapper (NSCM) [START_REF] De Carvalho | Spectral correlation mapper (SCM): an improvement on the spectral angle mapper (SAM)[END_REF], and normalized Euclidean distance (NED) [START_REF] Maurya | A critical analysis of polarimetric signatures on PALSAR 2 data for land cover classification[END_REF] are the popular distance/correlation based classifiers used with PSs for various applications.

There is a lack in PSs based studies because of the fact that PSs are two dimensional features whereas the common processing techniques used with these PSs such as the NSCM or the NED, perform vector operations, and therefore are unable to consider the neighbourhood influence during target (land cover) identification or classification. Hence, the PSs are under-utilized and provide sub optimal results.

The current study also addresses this challenge of under-utilization of PSs and proposes a novel approach based on convolutional deep neural networks to fully exploit the PSs as land cover features. Convolutional deep neural networks (CNNs) are recently one of the most popular, efficient, and powerful techniques in image feature extraction and classification applications.

Their parameter sharing and local connectivity properties makes them best tools to extract complex and localized features from images [START_REF] Lecun | Deep learning[END_REF]. A brief introduction to CNNs is provided in The study also utilizes apriori knowledge and image morphology to build village detection filters and to mask out misclassifications. Image morphology has been extensively used in studies to detect built-up areas from satellite images [START_REF] Palmason | Classification of hyperspectral data from urban areas based on extended morphological profile[END_REF].

To summarize, the study in this chapter presents a novel approach to detect and map villages affected by a flood with the help of PSs, CNNs, apriori knowledge and image morphology. The approach presented here involves three stages. These are;

Proposed Approach, Implementation, and Discussion

This section discusses the development of a novel approach to detect, identify, and map flood affected villages in a PolSAR image. The overall approach is divided into three stages.

First Flowchart shown in Figure 4.4 depicts the overall proposed approach.

Land cover polarization signatures

According to the discussion in section 3.2.1, both, co-polarized and cross-polarized PSs are computed from pre-processed PolSAR data using equation (3.1). Co-polarized PS plots for considered land cover class (averaged over all class samples, refer to Table 4.1) are shown in These PSs are used as features for land cover characterization and classification in many studies. Classification schemes used in [START_REF] Jafari | A new method for land cover characterization and classification of polarimetric SAR data using polarimetric signatures[END_REF], [START_REF] Phartiyal | Optimal use of polarimetric ssignature on PALSAR -2 data for land cover classification[END_REF], and [START_REF] Maurya | A critical analysis of polarimetric signatures on PALSAR 2 data for land cover classification[END_REF] for

Stage 1: CNN and polarization signatures based classification model for built-up area extraction

In where per pixel PS (polarization signature of one pixel) is used for classification, a window of PSs is used as input here. The window based approach helps suppresses the random volume scattering responses from closely packed built-up structures in built-up areas and thereby improves the segregation of these built-up areas.

A 33  window of neighbouring PSs is used here as input. This corresponds to a 54 111  Signature correlation mapper or SCM is a cross correlogram constructed by calculating the cross-correlation between two polarimetric signatures. The SCM obtains from the Pearson correlation coefficient and varies from -1 to 1. The SCM between two signatures p1and p2 is given by equation (4.1).

1 2 1 2 1 1 1 ( , ) 12 22 22 1 1 2 2 1 1 1 1 m m m m p p p p SCM p p m m m m m p p m p p                                                         (4.1)
where m is the number of polarization states [START_REF] Jafari | A new method for land cover characterization and classification of polarimetric SAR data using polarimetric signatures[END_REF], [START_REF] De Carvalho | Spectral correlation mapper (SCM): an improvement on the spectral angle mapper (SAM)[END_REF]. NSCM is obtained by normalizing the range of SCM between 0 and 1 using equation (4.2). NSCM 0.5 (SCM +1)  

Euclidean distance or ED is the distance metric which provides a quantitative measure of the distance between vectors representing the two polarimetric signatures, and it reflects the dissimilarity between the polarimetric signatures [START_REF] Jafari | A new method for land cover characterization and classification of polarimetric SAR data using polarimetric signatures[END_REF], [START_REF] Gower | Properties of Euclidean and non-Euclidean distance matrices[END_REF]. The ED between two signatures p1 and p2 is given by equation (4.3).

2 1 2 1 2 1 ( , ) ( ) m ED p p p p   (4.3)
where m is the number of polarization states. Further, NED is the normalized version of ED. Both NSCM and NED have been utilized with PSs for land cover characterization in [START_REF] Jafari | A new method for land cover characterization and classification of polarimetric SAR data using polarimetric signatures[END_REF] and for land cover classification in [START_REF] Maurya | A critical analysis of polarimetric signatures on PALSAR 2 data for land cover classification[END_REF].

Detection and mapping of flood affected villages using apriori knowledge and image morphology

This section presents post classification methodologies based on apriori knowledge and image morphology to map the villages affected by flood from built-up area image obtained in stage 1.

This section covers stage 2 and stage 3 of the proposed approach. The methodology presented in stage 2 detects villages in the built-up area image obtained in stage 1 (refer section 4.

Stage 2: Village detection

The focus of this stage is to detect villages in the built-up area image obtained in stage 1 in section 4.3.2. In the methodology, at first, the built up area image __ a binary image __ is cleaned before detection because the image might contain objects (built-up areas) that are close to each other and could be part of same village or, there might be objects with holes in them which is undesirable. These artefacts could lead to inaccurate detection and mapping of villages.

Therefore, a series of morphological operations are utilized to clean the binary image. The objects or components which are close to each other, are merged together at first using a morphological image closing filter provided in equation (4.4) [START_REF] Gonzalez | Digital Image Processing[END_REF].

closed

I (I ) S S    (4.4) 92
Where, closed I is the closed image, I is the image on which the morphological closing operation is to be performed, S is the structuring element with which the closing operation is performed,  and  represents the morphological dilation and erosion operations respectively.

The vicinity, within which two objects can be merged together is set via the size of the structuring element parameter. The vicinity is set based on apriori knowledge of the approximate minimum distance between two villages and image pixel resolution.

The approximate minimum distance between two villages is considered here to 200 meters (based on ground surveys and experiment) and the ALOS PALSAR-2 PolSAR data considered in this study has 6-meter spatial resolution. Therefore, S is set as a disc of 15 pixels' diameter (100 meters) and two objects within this distance are merged as one. The effect of the closing operation is highlighted in Next, the holes in the objects in the closed I image are to be filled. The holes are filled using equation (4.5) [START_REF] Gonzalez | Digital Image Processing[END_REF].

filled closed k I I X  (4.5)
Where, filled I is the image with holes filled, closed I is the image with holes obtained from equation (4.4), is the union operator, and k X is obtained through iterative process governed by equation (4.6) [START_REF] Gonzalez | Digital Image Processing[END_REF],

1 complement ( ) k k X X S I    (4.6)
Where again, 0 X is the closed I image, S is the connectivity, and complement I is the negative of closed I . Figure 4.8(c) depicts the result of the hole filling operation.

Finally, an area limit is set to filter out objects that are not villages. The idea is that any object considered to be a village is to have at least 5000 square meters of ground area. The area limit is established based on apriori knowledge of the study region. is the th k object in the filled I image, AL is the area limit, and N is the total number of objects in filled I image. A quantitative analysis is therefore required to assess the performance of the classifiers in more detail. If the BBOR of a particular object pair is above a predefined threshold, then the corresponding object in the test image is counted a successful detection, otherwise not. The threshold is set to 0.1 based on experiment and critical analysis, and is set identical for both Roorkee and Haridwar study areas for consistency purpose.

True positives (villages successfully detected), true negatives (villages missed), and false alarms or false positives (objects detected as villages in the test image but are actually not) are computed using equation (4.9, 4.11, and 4.12).

1

True Positive(TP) logical(BBOR threshold)

m i   (4.9)
Where, m is the total number of objects (villages) in the reference image, and

1if threshold logical( ) 0if threshold x x x       (4.10)
True Negative(TN) Total number of villagesin referenceimage TP 

Stage 3: Mapping flood affected villages

Since, a PolSAR image acquired to monitor a flood might include both, flooded and not flooded regions. Therefore, it is necessary for any approach which maps only flooded villages to mask out the non-flood affected villages present in the PolSAR image. The village detection methodology proposed in section 4.3.3.1 does not separate out flooded and non-flooded villages.

Therefore, a new methodology is proposed to achieve the task. This methodology presented in this section encompasses stage 3 of the overall approach.

Intuitively, the villages that are flooded must be completely enclosed by water, otherwise not. This means that any village affected by flood should appear as a 'hole' in the water class (rest class in this case) image and any village that is not affected by flood should not. Based on this observation, a criterion is set i.e. if an object in the DV image obtained in section 4.3.3.1 is a subset of an object in the rest class image, then, that object in the DV image is a flooded village otherwise not. This criterion will separate the flooded village from non-flooded villages.

To realize the criterion, a bounding box overlap ratio (BBOR2) is computed over each pair of DV image object and the "holes filled" version of rest class image object. The holes filled rest class image is obtained using equation similar to (4.8) If the BBOR2 is 1, then the corresponding object in the DV image is a flooded village and if the BBOR2 is less than 1, the corresponding object in the DV image is not a flooded village. This methodology is depicted in

Conclusion

This chapter presented an efficient approach to map flood affected villages using PolSAR image.

The approach included three stages i.e. -first, built-up area extraction, -second, village detection, and -third, flood affected village mapping.

In the first stage, the approach utilized PolSAR polarization signatures (PSs) and CNNs to extract built-up areas from a PolSAR image. A novel PSs-CNN classifier is developed. The PSs-CNN model provided improved built-up area extraction performance in contrast to the NED and the NSCM classifiers. In the second stage, the approach introduced and implemented a novel village detection methodology. The methodology uses a detection filter to detect whether a particular object in the built-up area image (obtained in stage 1) is a village or not. The detection filter uses apriori knowledge and image morphology to detect villages in the built-up area image.

The detection methodology also significantly removes false alarms and misclassifications. In the third and final stage, flood affected villages are identified and non-flooded villages are masked out using another image morphology based methodology. This methodology computed overlap ratios between objects (successfully detected as villages) and flooded area and utilized it for identification of flood affected villages.

Quantitative and qualitative assessment of the performance of the novel approach over two different study areas is carried out. Different artificial flood scenarios are created to evaluate the performance of developed approach. The developed approach is evaluated under these flood scenarios and flood affected villages are identified and mapped. Number of villages successfully detected and false alarms are computed with the help of ground truth information (actual number of villages or towns in the study regions are known). Results indicate to high detection accuracy (above 90%). The approach may be scalable and potentially satisfactory for real time usage. "Generalization is the measure of the ability of machine learning techniques on how well these techniques perform on 'out of sample' data."

Introduction

The synthetic aperture radar (SAR) and multispectral (MS) image data modalities are crucial for earth studies and complement each other in many earth observation applications such as land cover classification, crop monitoring, or change detection. In general, multispectral image provides spectral reflectance information and PolSAR image provides scattering and polarizing behaviour information about the earth surface objects. These earth surface objects are more commonly termed as land covers. The basic idea behind utilizing the two image data modalities together is to acquire better spatial, spectral, and temporal information on the land covers.

Although, higher spectral information can be achieved through hyperspectral images but at a cost of spatial and temporal information. Also, since it is an earth observation strategy based on optical imaging, therefore it suffers from the classical challenges (clouds, night-time). Similarly, use of polarimetric SAR (PolSAR) in standalone fashion for land cover classification resulted in a maximum of 75% classification accuracy in mixed land cover class scenarios (refer section 3.7). For applications such as land use land cover classification or land cover monitoring, all three aspects are crucial and a multisensor data based approach is more beneficial. Using PolSAR and MS image data modalities together is shown to be a convenient and successful strategy.

Methodologies listed in many articles such as in [START_REF] Pohl | Multisensor image fusion in remote sensing: Concepts, methods and applications[END_REF], or more recently in [START_REF] Joshi | A review of the application of optical and radar remote sensing data fusion to land use mapping and monitoring[END_REF], [START_REF] Zhang | Multi-source remote sensing data fusion: Status and trends[END_REF], [START_REF] Chaudhri | Different Modality Based Remote Sensing Data Fusion Approach for Efficient classification of agriculture and urban subclasses[END_REF] have been used by the earth observation data processing community in processing multispectral and PolSAR data together. Authors in [START_REF] Joshi | A review of the application of optical and radar remote sensing data fusion to land use mapping and monitoring[END_REF] provide a meta-analysis on the methodologies in successful applications of PolSAR and MS image data fusion. The methodologies include traditional, machine learning, and knowledge based decision trees. In [START_REF] Joshi | A review of the application of optical and radar remote sensing data fusion to land use mapping and monitoring[END_REF], authors also remarked that the studies in multisensor data fusion based land cover classification mostly use MS and

PolSAR data with machine learning methods. Most popular machine learning based multisensor data fusion techniques include artificial neural networks (ANNs or NNs), and support vector machines (SVMs). In fact, studies in [START_REF] Joshi | A review of the application of optical and radar remote sensing data fusion to land use mapping and monitoring[END_REF]- [START_REF] Feng | Machine learning-based integration of remotelysensed drought factors can improve the estimation of agricultural drought in South-Eastern Australia[END_REF] and [START_REF] Zhang | Multi-source remote sensing data fusion: Status and trends[END_REF] have also recognized the significance of machine learning in various multisensor image data based applications. From the aforementioned review, two points are evident; -first, utilization of PolSAR and MS data together is a successful strategy, and -second, machine learning techniques are successful in utilizing the two i.e. PolSAR and MS, image data modalities together.

However, it is challenging for (NNs) with shallow depth (number of layers ≤ 2) to extract high level abstract/complex features from high dimensional earth observation image data without overfitting or losing generalization [START_REF] Lawrence | Overfitting and neural networks: conjugate gradient and backpropagation[END_REF]. Generalization refers to the ability of how well an algorithm performs on 'out of the seen samples' data points. If an algorithm is performing well outside training data, then the algorithm is showing good generalization performance and vice versa. Researchers in many studies [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF], [START_REF] Zhu | Deep learning in remote sensing: A comprehensive review and list of resources[END_REF], [START_REF] Hu | Deep convolutional neural networks for hyperspectral image classification[END_REF] are exploring new advanced machine learning approaches that are able to extract complex features from high dimensional earth observation image data. In the direction, deep neural networks with deep learning are the potential and powerful tools for extracting high level information from earth observation image data [START_REF] Zhong | Deep learning based multi-temporal crop classification[END_REF], [START_REF] Huang | Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery[END_REF] Convolutional neural networks (CNNs) are recently the most widely used deep network models in earth observation image data based land cover classification [START_REF] Zhu | Deep learning in remote sensing: A comprehensive review and list of resources[END_REF]. CNNs have the ability to extract complex features directly from data [START_REF] Lecun | Deep learning[END_REF], [START_REF] Rajput | CNN classifier based low-resolution Face recognition algorithm[END_REF], [START_REF] Bhadoria | Deep neural networks for biometric identification based on non-intrusive ecg acquisitions[END_REF]. Another advantage of developing CNNs for PolSAR-MS multisensor image data processing is that CNNs can directly utilize multisensor high dimensional earth observation image data with efficient performance.

Unlike the conventional data fusion methods, this reduces the need of fusing the PolSAR-MS images together before using it further.

In [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF], authors present a comparison of CNN based models with few popular earth observation image data classifiers in processing hyperspectral images. The CNN model in [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF], is designed to effectively capture global spectral features along with local spatial features. Similar studies are reported in [START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF], [START_REF] Mei | Integrating spectral and spatial information into deep convolutional Neural Networks for hyperspectral classification[END_REF] and [START_REF] Feng | CNN-based multilayer spatial-spectral feature fusion and sample augmentation with local and nonlocal constraints for hyperspectral image classification[END_REF]. The CNN models in these studies employed one, CNNs or, along with the spatial information using two, or three dimensional CNNs. Many studies [START_REF] Hu | Deep convolutional neural networks for hyperspectral image classification[END_REF], [START_REF] Mei | Integrating spectral and spatial information into deep convolutional Neural Networks for hyperspectral classification[END_REF], [START_REF] Chen | Deep learning-based classification of hyperspectral data[END_REF]- [START_REF] Yu | Convolutional neural networks for hyperspectral image classification[END_REF] and more have used one dimensional CNNs as spectral feature extractors for hyperspectral image based classification. In contrast, extracting spectral information along with spatial information from hyperspectral image using CNNs is a recently popular approach. The approach has however improved the results in land cover monitoring and classification applications. Because hyperspectral images have large spectral dimensions, extracting spectral information is computationally expensive and compounding with extraction of spatial information makes it worse. Therefore, some studies [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF], [START_REF] Makantasis | Deep supervised learning for hyperspectral data classification through convolutional neural networks[END_REF], [START_REF] Aptoula | Deep learning with attribute profiles for hyperspectral image classification[END_REF], [START_REF] Yue | Spectral-spatial classification of hyperspectral images using deep convolutional neural networks[END_REF] used principal component analysis (PCA) to reduce the high dimensionality of hyperspectral image and then used two dimensional CNNS for classification applications. While, alternate to this, some studies [START_REF] Zhang | Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network[END_REF], [START_REF] Yue | Spectral-spatial classification of hyperspectral images using deep convolutional neural networks[END_REF], [START_REF] Zhao | Spectral-spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach[END_REF] used one and two dimensional CNNs separately to extract the spectral and spatial information and combined the separately extracted features later. Further, some studies used three dimensional CNNs to extract local spectral and spatial information simultaneously. In [START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF], [START_REF] Li | Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network[END_REF], authors used three dimensional CNNs as spectral-spatial feature extractors and classifiers for hyperspectral satellite image data.

In summary, enough evidence suggests that; one, two, and three dimensional CNNs have been successfully used to extract spectral and/or spatial information from hyperspectral image data. However, the significance of spectral and/or spatial CNNs as land cover classifiers with high dimensional multisensor image data is still interesting to investigate. Therefore, the study presented in this chapter focusses on two aspects.

Theoretical Background

Convolutional neural networks

The fundamental concepts on CNNs are discussed in section 3.2.2 with an example of 2D CNNs.

Similar networks can be realized for 1D or 3D CNNs accordingly. However, the architectures of 1D, 2D, and 3D CNNs for remote satellite image data processing are as follows.

 For 1D CNNs, the satellite image data input is a vector (usually a vector of spectral features). m W is a matrix and m b is vector. In general, 1D CNNs are used as feature extractors in satellite image data, especially hyperspectral data processing.  For 2D CNNs, the convolution is performed on the two spatial dimensions (e.g. images). The input may contain extra dimensions (such as spectral information), but these are not used in the convolution. m W is a 3D matrix and m b is vector. It is important to note that in 2D CNNs, local convolutions are only performed in the spatial dimensions and not in the spectral dimension. 2D CNNs are used as spatial feature extractors and classifiers.  In 3D CNNs, convolutions are performed in the spatial dimensions and the temporal dimension simultaneously. However, in the absence of temporal information and presence of high spectral information, spectral dimension replaces the temporal dimension and local spectral convolutions are possible. m W is a 4D matrix and m b is vector. Therefore, 3D CNNs are used as spatial temporal or spatial spectral feature extractors and classifiers in satellite image data processing.

The present study utilizes and investigates the ability of 1D, 2D, and 3D CNNs as land cover classifiers on PolSAR-MS data. Also, the present study proposes novel LC classification models based on these CNNs.

Experiment Setup

Dataset

Datasets used in the study are multisensor i.e. multispectral (MS) and polarimetric synthetic aperture radar (PolSAR) data acquired over the study areas i.e. Roorkee and Haridwar.

Multispectral data used is the Landsat-8 multispectral data acquired on March 17, 2015 (band specifications are provided in Table 1.2. The PolSAR data used is the ALOS PALSAR-2 L band fully polarimetric single look complex PolSAR data acquired on March 13, 2015 (common from chapter 3 and chapter 4).

Preprocessing is required with both MS and PolSAR raw data. Multispectral data mostly suffers from atmospheric perturbations whereas PolSAR data suffers from speckle noise.

Preprocessing is done separately on MS and PolSAR data. Preprocessing of PolSAR PALSAR-2 data is similar to as explained in chapter 3 (refer section 3.3.4). Derived PolSAR features are also used in this study. Polarization signatures (PSs) and further, polarization signatures correlation features (PSCFs) are computed from the single look complex PolSAR data with procedure similar to as in chapter 3 (refer section 3.4). Based on the analysis of chapter 3 (refer section 3.4), four PSCFs namely "corr_co_Di", "corr_co_FP", "corr_co_HD", and "corr_co_VD" are used as derived PolSAR features in the current study. Overall, three basic (HH, HV, and VV) and four derived (corr_co_Di, corr_co_FP, corr_co_HD, and corr_co_VD) PolSAR feature are used in the current study. Landsat-8 operational land imager (OLI) multispectral data is atmospherically corrected first. Out of the 11 bands shown in Table 1.2, band 2 (blue) through 7 (short wave infrared) are selected for the study. These selected bands are pan sharpened as explained in [START_REF] Gillespie | Color enhancement of highly correlated images. II. Channel ratio and "chromaticity" transformation techniques[END_REF] and then resampled to the PALSAR-2 data spatial resolution of 6 meters using the "bicubic spline" interpolation method [START_REF] Rajput | Multisensor fusion of satellite images for urban information extraction using pseudo-Wigner distribution[END_REF]. The experiment setup i.e. the study areas, the datasets, or the ground truth data points, is used for, first, the critical analysis of the performance of conventional CNN frameworks, and second, the development of novel CNN classification frameworks for land cover classification.

The flowchart shown in Figure 5.3 depicts the experiment setup.

1D, 2D, and 3D CNNs for Land Cover Classification with PolSAR-MS Data

1D-CNNs for PolSAR-MS data.

One dimensional CNNs (1D CNNs) have been successfully used with unimodal satellite image data in the recent years. In [START_REF] Chen | Deep learning-based classification of hyperspectral data[END_REF], authors used stacked auto encoders for classification with hyperspectral satellite image data. They also used PCA for hyperspectral feature reduction before classification as the number of input features is very high (say 176 for KSC dataset). A range i.e. [100] The present study focuses on the analysis of the performance of 1D CNNs; both 1D-CNN-v1 and 1D-CNN-v2 with PolSAR-MS data.

The selected hyper-parameter settings are listed in Table 5. For example, a first filter (in yellow) convolves with bands 1, 2, and 3 whereas a second filter (in blue) convolves with bands 5, 6, and 7. This provides an improvement in classification accuracy and robustness compared to other 3D-CNN models [START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF], [START_REF] Feng | CNN-based multilayer spatial-spectral feature fusion and sample augmentation with local and nonlocal constraints for hyperspectral image classification[END_REF] Configuration and learning parameters are the same as in section 5.4.1 and are summarized in Table 5.2.

Proposed Permuted Local

Results and Discussions

A. Quantitative analysis

The models 1D-CNN-v1, 1D-CNN-v2, 2D-CNN-v1, 3D-CNN-v1, Perm-LS-CNN, and Perm-LSS-CNN are trained and cross validated on ground truth class data points provided in Table 5.1.

Classification results are shown in Figure 5.6 and Table 5.5 for the Roorkee data set, and in Figure 5.7 and Table 5.6 for the Haridwar data set. Authors in [START_REF] Heydari | Meta-analysis of deep neural networks in remote sensing: A comparative study of mono-temporal classification to support vector machines[END_REF]

B. Qualitative analysis

A detailed analysis of the classification performance of all the classifiers considered here is however required to support these tabular results. A visual analysis of the classified images from RF, RBF-SVM, 2D-CNN-v1, 3D-CNN-v1, and Perm-LSS-CNN is performed. Figure 5.8

shows the classification results on selected areas for the five models. The selected areas are marked as A, B, C, D, and E in Figure 5.6(a). They contain the points that are taken as ground truth.

In as compared to other classifiers. In Figure 5.8(t, and u), most of the ground areas (bare soil or short vegetation) is wrongly classified as water. In Figure 5.8(v), the golf ground is approximately well classified, although the shape is a partially deteriorated. In summary, the analysis of the results in Table 5.5, Table 5.6, and Figure 5.

Conclusion

This chapter presents a critical analysis of the potential and performance of the one, two, and three dimensional CNNs, where dimensionality refers to the convolved features: spectral, spatial or both. This chapter presents a study in which an attempt is made; to analyse conventional, and to develop novel, CNN and RNN models based agriculture crop classification models with multisensor/multimodal satellite time series imagery. This chapter is an extension of chapter 5 in a sense that the concept of permuted spectral input is used here with time series images and how DNN models exploit the satellite images with spectral, spatial, and temporal information.

Introduction

Recent reports indicate the initiatives taken by authorities worldwide towards development of sustainable agriculture science and management strategies and policies [START_REF] Garnett | Sustainable intensification in agriculture: Navigating a course through competing food system priorities[END_REF]- [START_REF]Global indicator framework for the Sustainable Development Goals and targets of the 2030 Agenda for Sustainable Development[END_REF]. Satellite image based crop monitoring is a key technology in acquiring crop information such as crop type, or crop phenology [START_REF] Pandey | An optimal estimation technique for increasing the accuracy of crop forecasts by combining remotely sensed and conventional forecast[END_REF], [START_REF] Godbole | Remote Sensing: Applications in Agriculture[END_REF]- [START_REF] Mishra | Remote Sensing Applications in Agriculture[END_REF]. Satellite image based crop monitoring is easy, fast, reliable, techno-economical, and efficient [START_REF] Geosys | Understanding and Evaluating Satellite Remote Sensing Technology in Agriculture[END_REF]- [START_REF]Bureau, '7 Benefits of Remote Sensing & GIS in Agriculture[END_REF]. In the recent years, India is strongly leaning towards the use of satellite images for crop monitoring and crop information extraction [START_REF]Crop Insurance/PMBFY = Remote Sensing Play Major Role in Near Future[END_REF], [START_REF]Agriculture[END_REF] but still needs to go far.

Optical and PolSAR satellite images have been successfully used for agricultural applications in India such as crop mapping/classification [START_REF] Wardlow | Analysis of time-series MODIS 250 m vegetation index data for crop classification in the U.S. Central Great Plains[END_REF], crop yield forecasting [START_REF] Kastens | Image masking for crop yield forecasting using AVHRR NDVI time series imagery[END_REF], crop stress and drought stress monitoring [START_REF] Balint | Drought monitoring with the combined drought index[END_REF], mapping of crop type/cycle changes [START_REF] Galford | Wavelet analysis of MODIS time series to detect expansion and intensification of rowcrop agriculture in Brazil[END_REF], crop phenology assessment [START_REF] Sakamoto | A crop phenology detection method using time-series MODIS data[END_REF], and biomass estimation [START_REF] Rembold | Using low resolution satellite imagery for yield prediction and yield anomaly detection[END_REF] in the last two to three decades. The use of satellite images in these applications is increasing day by day and new applications such as precision agriculture [START_REF] Mulla | Twenty five years of remote sensing in precision agriculture: Key advances and remaining knowledge gaps[END_REF] are being discovered as well. In particular, with the availability of open source high spatial and/or spectral and/or temporal resolution multisensor satellite images via programs such as the Copernicus Sentinel satellite series, the performance and efficiency of the aforementioned applications has increased. Also, new and advanced studies in crop monitoring have surfaced [START_REF] Kussul | Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[END_REF], [START_REF] Van Tricht | Synergistic use of radar sentinel-1 and optical sentinel-2 imagery for crop mapping: A case study for Belgium[END_REF], [START_REF] Lavreniuk | Crop classification strategies using hybrid sentinel-1, sentinel-2 and landsat-8 data series in Ukraine[END_REF]. Recent studies on Indian croplands are also using Sentinel series data for crop monitoring [START_REF] Maurya | Development of fusion approach for estimation of vegetation fraction cover with drone and sentinel-2 data[END_REF], [START_REF] Kumar | Estimation of winter wheat crop growth parameters using time series Sentinel-1A SAR data[END_REF].

Moreover, time series satellite images are rich source of crop phenology information and have become popular, efficient, and crucial for agriculture crop monitoring including crop identification and classification. For agriculture, it has been shown that the temporal evolution of the signal measured via satellite imaging enables a better separation between crop types classes [START_REF] Whitcraft | A framework for defining spatially explicit earth observation requirements for a global agricultural monitoring initiative (GEOGLAM)[END_REF]. Satellite time series image based agriculture crop classification is one of the most costeffective, reliable, and successful framework. Also, different image modalities have been successfully used in crop classification. Satellite PolSAR imaging is a weather independent technology and time series PolSAR images have been successfully used in crop classification applications [START_REF] Skriver | Multitemporal C and L band polarimetric signatures of crops[END_REF]- [START_REF] Bargiel | Capabilities of high resolution satellite radar for the detection of semi-natural habitat structures and grasslands in agricultural landscapes[END_REF]. On the other hand, time series optical imagery provides crop phonology information via spectral-temporal features. Time series optical imagery has also been successfully used for crop classification in the recent past [START_REF] Tatsumi | Crop classification of upland fields using Random forest of time-series Landsat 7 ETM+ data[END_REF], [START_REF] Dara | Mapping the timing of cropland abandonment and recultivation in northern Kazakhstan using annual Landsat time series[END_REF] The reduced number of optical data based crop classification studies is because of the effect of clouds on optical satellite images. PolSAR and optical image modalities have also been used synergistically for crop classification to improve the performance of methodologies [START_REF] Zhu | Fusion of multisensor multitemporal satellite data for land cover mapping[END_REF], [START_REF] Villa | In-season mapping of crop type with optical and X-band SAR data: A classification tree approach using synoptic seasonal features[END_REF], [START_REF] Mcnairn | Integration of optical and Synthetic Aperture Radar (SAR) imagery for delivering operational annual crop inventories[END_REF]- [START_REF] Skakun | Efficiency assessment of multitemporal c-band radarsat-2 intensity and landsat-8 surface reflectance satellite imagery for crop classification in ukraine[END_REF].

In context to satellite time series image processing techniques for crop classification, numerous techniques have been proposed and/or utilized in the past. The popular techniques are maximum likelihood classifier [START_REF] Bargiel | A new method for crop classification combining time series of radarimages and crop phenology information[END_REF], [START_REF] Mcnairn | The contribution of ALOS PALSAR multipolarization and polarimetric data to crop classification[END_REF], [START_REF] Mcnairn | Integration of optical and Synthetic Aperture Radar (SAR) imagery for delivering operational annual crop inventories[END_REF], [START_REF] Chust | Land cover discrimination potential of radar multitemporal series and optical multispectral images in a Mediterranean cultural landscape[END_REF], neural networks [START_REF] Mcnairn | The contribution of ALOS PALSAR multipolarization and polarimetric data to crop classification[END_REF], [START_REF] Mcnairn | Integration of optical and Synthetic Aperture Radar (SAR) imagery for delivering operational annual crop inventories[END_REF], [START_REF] Skakun | Efficiency assessment of multitemporal c-band radarsat-2 intensity and landsat-8 surface reflectance satellite imagery for crop classification in ukraine[END_REF],

decision trees [START_REF] Mcnairn | The contribution of ALOS PALSAR multipolarization and polarimetric data to crop classification[END_REF], [START_REF] Mcnairn | Integration of optical and Synthetic Aperture Radar (SAR) imagery for delivering operational annual crop inventories[END_REF], random forest [START_REF] Villa | In-season mapping of crop type with optical and X-band SAR data: A classification tree approach using synoptic seasonal features[END_REF], [START_REF] Bargiel | A new method for crop classification combining time series of radarimages and crop phenology information[END_REF], [START_REF] Tatsumi | Crop classification of upland fields using Random forest of time-series Landsat 7 ETM+ data[END_REF], [START_REF] Dara | Mapping the timing of cropland abandonment and recultivation in northern Kazakhstan using annual Landsat time series[END_REF], [START_REF] Inglada | Improved early crop type identification by joint use of high temporal resolution sar and optical image time series[END_REF], [START_REF] Forkuor | Integration of optical and synthetic aperture radar imagery for improving crop mapping in northwestern Benin, West Africa[END_REF], temporal fusion classification [START_REF] Zhu | Fusion of multisensor multitemporal satellite data for land cover mapping[END_REF], and hierarchical classification strategy [START_REF] Blaes | Efficiency of crop identification based on optical and SAR image time series[END_REF] This chapter is organized as follows. Section 6.2 provides conceptual background on CNNs and RNNs. Further, section 6.3 provides the information on the experiment setup i.e. study areas, datasets, and ground truth. Section 6.5 and 6.7 provides the implementation details on the conventional and novel CNN-RNN models Section 6.8 discusses on results obtained using these models and its analysis. Finally, section 6.9 concludes the study.

Theoretical Background

Basic knowledge of CNNs and RNNs is required in this chapter as CNNs can exploit the spectral and/or spatial information and RNNs can exploit the temporal information, present in the MSTS images. The brief concepts of CNNs are presented in chapter 3 (refer section 3.2.2) and chapter 5 (refer section 5.2.1). In addition, the basic concepts on RNNs are discussed in this section.

Recurrent neural networks

Recurrent neural networks, also known as RNNs, are a class of neural networks that allow previous outputs to be used as inputs while having hidden states. The RNNs have the ability to process input of any length while keeping the model size unchanged. RNNs also share weights across time along with space. Therefore, the output depends on historical information. A typical architecture of a RNN is shown in Figure 6.1.

For each timestep t, the activation t a  and the output t y  are expressed as follows [START_REF] Elman | Finding Structure in Time[END_REF].

 

A. Recurrent cell

A recurrent cell is the smallest unit in RNNs which is able to memorize activation representing historical information. There are different variants of recurrent cells. The popular recurrent cell architectures are discussed here.

Simple recurrent cell

Architecture and signal flow of a simple recurrent cell is shown in Figure 6.2. The cell takes t

x 

i.e. current input and 1 t a    i.e. activation from previous cell, as inputs and outputs t a  i.e. activation for next RNN cell and t y  i.e. prediction from current state. Although the cell is able to capture short historical information but is unable to retain longer historical information. There are two popular variants of the basic RNN cell that are able to capture longer historical relations within a signal.

Long short term memory (LSTM) recurrent cell

Recurrent neural networks with simple recurrent cell have problems associated with vanishing and exploding gradients [START_REF] Hochreiter | Gradient flow in recurrent nets: the difficulty of learning longterm dependencies[END_REF]. This is a common problem in neural network updates where successive multiplication by the weight matrix W is inherently unstable; it either results in the gradient disappearing during backpropagation, or in it blowing up to large values in an unstable way. In RNNs, this type of instability is the direct result of successive multiplication with the (recurrent) weight matrix at various time-stamps. One way of viewing this problem is that a neural network that uses only multiplicative updates is good only at learning over short sequences, and is therefore inherently endowed with good short-term memory but poor longterm memory [START_REF] Hochreiter | Gradient flow in recurrent nets: the difficulty of learning longterm dependencies[END_REF]. This problem can be addressed with the use of a long short-term memory (LSTM) cell for longer memory. The operations inside the LSTM cell are designed to have finegrained control over the data written into this long-term memory. Figure 6.3 show the internal architecture and signal flow in a LSTM recurrent cell. The LSTM recurrent cell has a new variable t c  , which stands for memory cell at time t. The memory cell will provide memory to remember activations which can be used during predictions at later time steps.

The governing equations inside a LSTM recurrent cell are as follows [START_REF] Hochreiter | Long Short-Term Memory[END_REF]. [START_REF] Hochreiter | Long Short-Term Memory[END_REF].

  1 tanh , t t t c r c c W a x b              (6.3) 1 t t t uf c c c              (6.4) 0 tt ac        (6.

Gated recurrent unit (GRU) recurrent cell

The Gated Recurrent Unit (GRU) can be viewed as a simplification of the LSTM, which does not use explicit cell states. Another difference is that the LSTM directly controls the amount of information changed in the hidden state using separate forget and output gates. On the other hand, a GRU uses a single reset gate to achieve the same goal. The architecture and signal flow in a GRU recurrent cell is shown in Figure 6.4.

The governing equations inside a GRU cell are as follows [START_REF] Cho | Learning phrase representations using RNN encoder-decoder for statistical machine translation[END_REF].

  1 tanh , t t t c r c c W a x b              (6.6) 1 ( 1
) t t t uu c c c               (6.7) tt ac      (6.8)
Where, all the parameters have usual meanings. The GRU cell replaces the forget gate

B. Learning in RNNs

The key concept behind learning in RNNs is the 'backpropagation through time' algorithm. In backpropagation through time, the negative logarithms of the softmax probability of the correct predictions at the various time-stamps are aggregated to create the loss function. If the softmax probability at time t is represented as t p  , the loss L is defined as follows [252].

 

C. RNN architectures

RNN models are mostly used in the fields of time series signal processing. Overall, there are four variants of RNN architectures.

i. One to one ii. One to many iii. Many to one iv.

Many to many

The many to one and many to many versions of RNNs shown in Figure 6.5 are popularly used for crop classification in many studies.

Experiment Setup

This section provides information about the resources collected and utilized during the study. It includes information of study areas, datasets, and ground truth selected or collected for the study.

Study area

The study area selected for study is a cropland region in the Haridwar district of the Uttarakhand, India where common seasonal crops are practiced. The region is considered based on accessibility to ground truth collection, crop season, and generic land cover distribution. The geographical extents of the region are provided in Table 6.1.

Dataset

Multisensor satellite images are considered for crop classification in this study. Sentinel-

Ground truth

Based on the study regions and crop season selection, wheat is the major crop grown in the area.

However, the region also grows sugarcane throughout the year. The region also has evergreen forests, deciduous commercial plantations such as Salicaceae (Popular Eucalyptus), grasslands, and villages which must be considered during crop classification as it is a usual landscape (mixed

Preprocessing

Both Sentinel-1 and Sentinel-2 datasets require preprocessing before classification. Sentinel-1 GRD datasets are terrain corrected and calibrated. Terrain correction is performed using the SRTM 1 arc-second digital elevation model (DEM) available for public use. Speckle filtering is not performed as speckle noise is already suppressed in GRD products. Sentinel-2 L1C datasets are atmospherically corrected using the 'sen2cor' processor [START_REF] Main-Knorn | Sen2Cor for Sentinel-2[END_REF]. Further the 20 m spatial resolution bands in the bottom of atmosphere (BOA) Sentinel-2 datasets are resampled to 10 m using the 'super-resolution' approach proposed in [START_REF] Brodu | Super-resolving multiresolution images with band-independent geometry of multispectral pixels[END_REF]. Overall, 2 polarimetric channels i.e.

VH, and VV from the terrain corrected and calibrated Sentinel-1 and 10 multispectral bands from resampled bottom-of-atmosphere Sentinel-2 datasets are selected and stacked together to form a PolSAR-multispectral multisensor dataset. Overall, 7 multisensor datasets are created corresponding to the 7 time steps. Further, the 7 multisensor datasets are stacked along the temporal dimension creating a multisensor time series dataset. This PolSAR-multispectral multisensor satellite time series (MSTS) image dataset is utilized for crop classification.

Generalized DNN Framework for Satellite Time Series Images based Crop Classification

Although, numerous studies reported on satellite time series image processing have developed and utilized different DNN models, these models can be consolidated into a generalized framework. Figure 6.9 depicts a generalized DNN framework for time series satellite image based crop classification. The framework is composed of CNNs (1D, 2D or, 3D), RNNs (Simple, LSTMs or, GRUs), and fully connected (FC) modules. Different DNN architectures can be created using either any one, more than one or, all three modules. For example, 1D-CNNs can be used as spectral or temporal feature extractors with or without FC layers. Further, these 1D features can be utilized with a classifier (e.g. a softmax layer, SVM or any other) to achieve classification. Alternatively, the 1D CNNs can also be integrated with RNNs as, the former acts as spectral feature extractors and the latter acts as temporal feature extractors. The classification stage can be a softmax layer.

Similarly, 2D-CNNs can replace 1D-CNNs in the aforementioned framework. The only difference is that 2D-CNNs provide spatial features in contrast to 1D-CNNs (which provide only spectral features). If 2D-CNNs are used with RNNs then spatial-temporal features are generated which can be used with a classifier to achieve classification. 1D-and 2D-CNNs can also be used in parallel and cascade fashion to obtain both spectral and spatial or, spectral-spatial features respectively. These spatial and spectral or spectral-spatial features can be used with RNNs to obtain, spectral, spatial, and temporal features, spectral-spatial-temporal features or, any combination of these. These strategies lead to full exploitation of time series images with good spatial and spectral information.

However, the cascade fashion of 1D-and 2D-CNNs can be replaced with 3D-CNNs. 3D-CNNs have the ability to extract spectral and spatial features simultaneously from satellite images. 3D-CNNs can be used with or without FC layers prior to classification. Also, 3D-CNNs can be utilized with RNNs to capture all three i.e. spectral, spatial, and temporal information from satellite time-series images. The classification stage is usual.

The 1D-CNNs can be employed at pixel level for classification whereas the 2D-and 3D-CNNs can only be employed at image patch or image window level. Per-pixel classification is not an option with 2D-and 3D-CNNs. Also, the 1D-, 2D-, and 3D-CNNs have to be employed in time distributed fashion before using RNNs as temporal feature extractors. RNNs can also be The learning rate and the number of epochs are set to 0.01 and 200 respectively. It is to be noted here that the models (conventional or novel) discussed in this chapter have similar hyperparameters.

2D-CNN model for MSTS images based crop classification

The input MSTS image data has 1 spectral, 2 spatial and 1 temporal dimension. It is not possible to realize 2D-CNNs, in standalone mode, to process this 4-dimensional data directly. Therefore, 2D-CNN model is not explored further.

Therefore, in this chapter, the study and analysis of the performance of synergistic use of CNNs with permuted local spectral convolution and RNNs for MSTS based crop classification is carried out. It is postulated here that the RNNs might take advantage of permuted local spectral convolutions in MSTS image applications. The more generalized features obtained using the permuted local spectral convolutions are more abstract in nature and the CNN-RNN models built on top of this concept might improve the performance of these models.

Sub Task 2: Novel CNN and RNN Models with Permuted Local Spectral Convolutions for MSTS Image based Crop Classification

In this section, various novel CNN and RNN models are developed for MSTS image based crop classification which use the concept explained in the previous section. This section provides information about the architecture, and training parameters of the novel models developed.

1D-CNN model with permuted local spectral convolution for MSTS Images based crop classification

In this 1D-CNN model namely Perm-1D-CNN-v1, first, a permutation layer which generates 10 permuted versions of spectral bands as explained in chapter 5 (refer section 5.5). Then, the 10 permuted versions are stacked together to form an ensembled spectral band stack. This stack of bands (120, 10 permutations with 12 bands in each permutation set) is used as input in the further models.

After the ensembled input is generated, one dimensional convolution layers are used.

Since, the temporal depth is unaltered, one dimensional convolution in the temporal dimension will yield similar results as 1D-CNN-v1, it is insignificant to replicate the model. However, the spectral depth has now increased from 12 to 120, local spectral convolutions should yield more and may be different spectral features. Therefore, a series of spectral convolutions are performed over the spectral dimension with the help of one dimensional convolutional filters.

Three convolutional layers in time-distributed fashion are employed. The kernel size of the filters in the three convolutional layers i.e. C1, C2, and C3 layers are set to 7, 21, and 63

respectively. The kernel sizes are set based on experiments. Then, a FC layer with 50 nodes is used. Finally, an output layer with 6 classes is used. The FC and the output layer are also employed in time-distributed fashion.

CNN-RNN models with permuted local spectral convolution for MSTS images based crop classification

In this section, CNN and RNN architectures are used synergistically to form a novel CNN-RNN model for crop classification with MSTS satellite images. These CNN-RNN models are similar to CNN-RNN models discussed in section 6.5.5 with the difference of addition of a permuted layer. The ensemble of permuted sets of spectral bands help emphasize the spectral characteristic during MSTS image based classification as discussed in section 6.6.

1D-CNN-RNN model with permuted local spectral convolution for MSTS images based crop classification

In this 1D-CNN-RNN model namely Perm-1D-CRNN-v1, first, a permutation layer which generates 10 permuted versions of spectral bands. Then, the 10 permuted versions are stacked together to form an ensembled spectral band stack. This stack of bands (120, 10 permutations with 12 bands in each permutation set) is used as input.

After the ensembled input is generated, one dimensional convolution layers are used.

Since, the temporal depth is unaltered, one dimensional convolution in the temporal dimension will yield similar results as 1D-CRNN-v1, it is insignificant to replicate the model. However, the spectral depth has now increased from 12 to 120, local spectral convolutions should yield more and may be different spectral features. Therefore, a series of spectral convolutions are performed over the spectral dimension with the help of one dimensional convolutional filters.

Three convolutional layers in time-distributed fashion are employed. The kernel size of the filters in the three convolutional layers i.e. C1, C2, and C3 layers are set to 7, 21, and 63 respectively. The kernel sizes are set based on experiments.

Then, a conventional RNN model is considered. Since, 1D-CNNs or RNNs cannot accommodate spatial contextual information, per-pixel approach is considered here. A simple LSTM unit is selected as recurrent cell. Since the temporal depth is 7, seven LSTM units constitute the R1 recurrent layer. Here again, the RNN architecture is used in a many-to-many fashion. Also, the flow of activation from each recurrent cell in the recurrent layer is considered in a bidirectional manner. Then, a FC layer with 20 nodes is used. Finally, an output layer with 6 classes is used. The FC and the output layer are also employed in time-distributed fashion.

The architecture and hyper-parameter settings are listed in Table 6.

2D-CNN-RNN model with permuted input for MSTS images based crop classification

In this 2D-CNN-RNN model namely Perm-2D-CRNN-v1, first, a permutation layer which generates 10 permuted versions of spectral bands. Then, the 10 permuted versions are stacked together to form an ensembled spectral band stack. This stack of bands (120, 10 permutations with 12 bands in each permutation set) is used as input.

After the ensembled input is generated, a two dimensional convolution layer is used.

Since, two dimensional convolutions can accommodate spatial contextual information, patch based convolution approach is considered here. The kernel size of the filter in the convolutional layer i.e. C1 layer is a 33  window. The convolutional layer is employed in time-distributed fashion.

Then, a conventional RNN model is considered identical to the RNN model used in Perm-1D-CRNN-v1. The architecture of the Perm-2D-CRNN-v1 is listed in Table 6.10. The hyperparameter settings are similar to the Perm-1D-CRNN-v1 model and are listed in Table 6.10.

3D-CNN-RNN model with permuted local spectral convolution for MSTS images based crop classification

In this 3D-CNN-RNN model namely Perm-3D-CRNN-v1, first, a permutation layer which generates 10 permuted versions of spectral bands. Then, the 10 permuted versions are stacked together to form an ensembled spectral band stack. This stack of bands (120, 10 permutations with 12 bands in each permutation set) is used as input.

After the ensembled input is generated, a series of three dimensional convolution layers are used. The three dimensional convolution layers are used here to extract spectral-spatial features as the temporal features are capture by the RNN architecture. Since, three dimensional convolutions can also accommodate spatial contextual information, patch based convolution approach is considered here. The kernel size of the filters in the three convolutional layers i.e. Models with only convolutional layers and no recurrent layers such as 1D-CNN-v1 or 3D-CNN-v1 focus on a particular time stamp and does not exploit the temporal pattern present in the MSTS data. For example, in Figure 6.12, 3D-CNN-v1 (in both, standard and permuted input case) classifies the temporary fallow lands (unused land in between two crop cycles) as grasslands (shown in yellow). On the contrary, models with recurrent layers such as RNN-v1 or 3D-CRNN-v1 are able to ignore the temporary fallow lands and classified the areas as either sugarcane or wheat. It is important to emphasize here that even though the Perm-1D-CNN-v1 provides highest classification accuracy, it is unable to capture the temporal pattern in the MSTS data. Therefore, it is obvious to infer that standalone convolutional models are overfitting.

It can also be observed from Figure 6.12 that all the standard input based models have ). Hence, it is not obnoxious to assume that the Perm-3D-CRNN-v1 model is the best classifier among all the developed and compared classifiers.

Conclusion

This chapter presents a comprehensive study and analysis of the performance of conventional

Future Scope

The research presented in this thesis has extensive scope and possibilities. A few major possibilities are:

 The PSs-CNN model presented in chapter 3 uses PSs in H-V basis. However, other basis such as in [START_REF] Jafari | A new method for land cover characterization and classification of polarimetric SAR data using polarimetric signatures[END_REF] may provide complementary information which can lead to improved land cover classification performance.  In chapter 4, the approach which utilizes CNNs, apriori information, and image morphology to detect human settlements affected by flood disaster has potential to be developed into an operational system.  In chapter 5, local spectral and local-spectral-spatial convolutions on ensembled input provides better classification results in terms of generalization. However, the ensembled input increases the size of the input which increases the processing time. This increase in the input space can be optimized via more efficient permutation strategies.  The CNN and RNN models developed for MSTS images based crop classification developed in chapter 6 can also be optimized further in a manner similar to as for models in chapter 5.

APPENDIX-A

A1. Kennaugh Matrix

Kennaugh matrix or K-matrix is the representation of backscattered power from target in terms of the receiving antenna. The kennaugh matrix represents the received power, given the receiving antenna polarization characteristics. It is real quantity unlike scattering matrix. It is similar to

Muller matrix but explains for the radar backscattered case instead of optics. Kennaugh matrix is power matrix rather than amplitude and phase measure in scattering and coherency matrix respectively. Hence, all element of Kennaugh matrix are real quantities. Given Where, x is the transmit polarization state and y is the receive polarization state. Modified K-matrix calculated from scattering parameters given in equation (A.1) as follows: The K-matrix is represented as 164 This K-matrix is used in the calculation of co-and cross-polarization signatures (refer equation (3.1)).

  spectral-spatial CNN models for multisensor (PolSAR and multispectral) images based land cover classification, and iv) Development of novel CNN, RNN, and CNN-RNN models for multisensor (PolSAR and multispectral) satellite time series images based crop classification.

  My deepest regards extend to Department of Electronics & Communication Engineering, for providing me the working platform and well equipped laboratory to carry out my research tasks. I want to convey my heartfelt obligations to Dr Dharmendra Singh for providing me a space in his research lab with an ideal research environment. I will never forget the company I had from my fellow researchers and friends. In particular, I am thankful to my research colleagues from RailTel-IITR Centre of Excellence in Telecom (RICET), Remote Sensing Lab (RSL), Microwave Imaging and Space Technology Applications Lab (MISTAL), and INRIA (Bordeaux, France) labs; Dr Ankita Jain, Dr Shruti Gupta, Dr Neeraj Gill, Dr Varsha Mishra, Dr Akanksha Garg, Dr Bambam Kumar, Dr Prabhat Sharma, Dr Naina Narang, Deepak, Ankush, Arun, Jaydeep, Arun Kant, Ajay, Mandar, Ardhya, Rakesh, Jyothi, Ekta, Vyomika, Priyanka, Anshika, Udeshwari, Kundan, Rohit, Tushar, Kangana, Rishikant and Anas (INRIA, Bordeaux) for the congenial cheering work environment and for the round of discussions altogether sometimes regarding research, sometime social, political and cinematic reviews.

  ............................................................................................................................. XI xxiv 3.1. INTRODUCTION .................................................................................................................. 3.2. THEORETICAL BACKGROUND ............................................................................................3.2.1. Polarization signatures ......................................................................................................................... 3.2.2. Convolutional neural networks .............................................................................................................3.3. EXPERIMENT SETUP ..........................................................................................................

3. 3 . 1 .

 31 Study area ............................................................................................................................................. 3.3.2. Dataset .................................................................................................................................................. 3.3.3. Ground truth ......................................................................................................................................... 3.3.4. Preprocessing ....................................................................................................................................... 3.4. SUB TASK 1: POLARIZATION SIGNATURES: CRITICAL ANALYSIS AS POTENTIAL POLSAR FEATURES FOR LC CLASSIFICATION ........................................................................................

3. 4 . 1 .

 41 Polarization signatures computation: Observed and standard ............................................................ 3.4.2. Polarization signature correlation features or PSCFs ......................................................................... 3.4.3. Assessment and critical analysis of PSCFs...........................................................................................3.5. SUB TASK 2: OPTIMAL CLASS BOUNDARY ESTIMATOR BASED DECISION TREE APPROACH FOR LC CLASSIFICATION ..........................................................................................................

3. 5 . 1 .

 51 PSCF statistics ...................................................................................................................................... 3.5.2. Relationship between class boundaries and corresponding PSCF statistics ........................................ 3.5.3. Relating classification accuracy and class boundaries: Empirical modelling ..................................... 3.5.4. Optimizing the empirical model ............................................................................................................ 3.5.5. Optimal class boundaries based decision tree classifier ...................................................................... 3.5.6. Results and discussion .......................................................................................................................... 3.6. SUB TASK 3: DIRECT UTILIZATION OF POLSAR POLARIZATION SIGNATURES VIA CONVOLUTIONAL NEURAL NETWORKS FOR LC CLASSIFICATION ............................................ 3.6.1. Two dimensional convolution over polarization signatures: The concept ............................................ 3.6.2. PSs-CNN model architecture design and training ................................................................................ 3.6.3. Results and discussion ..........................................................................................................................

3. 7 .

 7 OVERALL RESULTS AND DISCUSSION ................................................................................ 3.8. CONCLUSIONS ................................................................................................................... CHAPTER 4 NOVEL APPROACH FOR DETECTION OF HUMAN SETTLEMENTS AFFECTED DURING FLOODS USING POLARIZATION SIGNATURES AND CNNS ..................................................................................................................................................... 4.1. INTRODUCTION .................................................................................................................. 4.2. EXPERIMENT SETUP ..........................................................................................................

4. 2 . 1 .

 21 Study area ............................................................................................................................................. 4.2.2. Satellite data ......................................................................................................................................... 4.2.3. Ground truth ......................................................................................................................................... 4.3. PROPOSED APPROACH, IMPLEMENTATION, AND DISCUSSION ............................................ xxv 4.3.1. Land cover polarization signatures ....................................................................................................... 4.3.2. Stage 1: CNN and polarization signatures based classification model for built-up area extraction .... 4.3.3. Detection and mapping of flood affected villages using apriori knowledge and image morphology ....4.4. CONCLUSION ..................................................................................................................... CHAPTER 5 PERMUTED SPECTRAL AND PERMUTED SPECTRAL-SPATIAL CNN MODELS FOR POLSAR-MULTISPECTRAL DATA BASED LAND COVER CLASSIFICATION ................................................................................................................ 5.1. INTRODUCTION ............................................................................................................... 5.2. THEORETICAL BACKGROUND .......................................................................................... 5.2.1. Convolutional neural networks ........................................................................................................... 5.3. EXPERIMENT SETUP ........................................................................................................

5. 4 . 1 .

 41 1D-CNNs for PolSAR-MS data. .......................................................................................................... 5.4.2. 2D-CNNs for PolSAR-MS data ........................................................................................................... 5.4.3. 3D-CNNs for PolSAR-MS data ........................................................................................................... 5.5. PROPOSED PERMUTED LOCAL SPECTRAL CNN (PERM-LS-CNN) AND PERMUTED LOCAL SPECTRAL-SPATIAL CNN (PERM-LSS-CNN) ....................................................................... 5.6. RESULTS AND DISCUSSIONS ............................................................................................ 5.7. CONCLUSION ................................................................................................................... CHAPTER 6 NOVEL CNN, RNN, AND CNN-RNN MODELS FOR MULTISENSOR SATELLITE TIME SERIES IMAGE BASED CROP CLASSIFICATION .................... 6.1. INTRODUCTION ............................................................................................................... 6.2. THEORETICAL BACKGROUND .......................................................................................... 6.2.1. Recurrent neural networks .................................................................................................................. 6.3. EXPERIMENT SETUP ........................................................................................................ 6.3.1. Study area ............................................................................................................................................ 6.3.2. Dataset ................................................................................................................................................ 6.3.3. Ground truth ........................................................................................................................................ 6.3.4. Preprocessing ...................................................................................................................................... 6.4. GENERALIZED DNN FRAMEWORK FOR SATELLITE TIME SERIES IMAGES BASED CROP CLASSIFICATION .................................................................................................................... 6.5. SUB TASK 1: CONVENTIONAL CNN AND RNN MODELS FOR MULTISENSOR SATELLITE TIME SERIES IMAGE BASED CROP CLASSIFICATION .......................................................................... xxvi 6.5.

6. 8 .Figure 1 . 1 4 Figure 1 . 2 4 Figure 1 . 3 7 Figure 1 . 4 7 Figure 1 . 5 14 Figure 1 . 6 27 Figure 1 . 7 27 Figure 1 . 8 A 28 Figure 1 . 9 33 Figure 3 51 Figure 3 . 2 53 Figure 3 . 3 56 Figure 3 . 4 58 Figure 3 . 5 Figure 3 . 6 60 Figure 3 . 7 : 64 Figure 3 . 8 65 Figure 3 . 9 70 Figure 3 . 70 Figure 3 . 11 : 72 Figure 3 . 12 : 73 Figure 3 . 74 Figure 3 .

 81141241371471514162717271828193335132533356345835366037643865397037031172312733743 Figure 1.1 Landsat-8 pixel of 30m spatial resolution. .................................................................. 4 Figure 1.2 Improvements in image spatial resolutions over time. ................................................ 4 Figure 1.3 Demonstration of the concept of radiometric resolution. The larger the bit depth, the higher the radiometric resolution (more number of intensity levels). .................................. 7 Figure 1.4 The temporal resolution of a satellite imaging system refers to how often it records imagery of a particular area. This example depicts the systematic collection of data every 30 days, presumably at approximately the same time of day. .............................................. 7 Figure 1.5 Illustration of the propagation of an electromagnetic plane wave. The electric field vector has horizontal (green) and vertical (blue) components, which combine to yield the net electric field vector (red) [27]. ...................................................................................... 14 Figure 1.6 Comparison of performances of traditional machine learning and deep learning techniques with respect to amount of training data availability. ........................................ 27 Figure 1.7 Figure illustrates that deep learning models can directly learn from raw data. The feature selection/extraction process is therefore redundant and sub-optimal in these models [71]. ..................................................................................................................................... 27 Figure 1.8 A general framework in DNN based models for satellite image analysis and processing. .......................................................................................................................... 28 Figure 1.9 Framework of the research ........................................................................................ 33 Figure 3.1 (a) False colour composite generated from PALSAR-2 data. Colour scheme: red = HH, green = HV, and blue = VV. (b) Panchromatic image collected from Google Earth dated Feb 19, 2015. Symbol 'A' marks built-up area in both the images and symbol 'B' marks the dense vegetation area in both the images. (c) The enlarged segments are shown in middle in the outlined boxes. The built-up area is showing volume scattering (high backscatter response in HV, refer) along with double bounce scattering. .......................... 51 Figure 3.2 Co-polarized polarization signature of an arbitrary land cover. ................................ 53 Figure 3.3 Sample images of different land covers collected during ground truth survey. (a) Tall vegetation, (b) bare soil, (c) agriculture cropland, and (d) sugarcane field. ....................... 56 Figure 3.4 Flowchart showing polarization signature correlation features (PSCFs) computation using fully polarimetric SAR data. ..................................................................................... 58 Figure 3.5 Geometrical structures, scattering matrices, and co-polarization signatures of different standard targets. .................................................................................................................. 58

Figure 4 .

 4 Figure 4.1 (a) Google Earth imagery of the Roorkee with markers indicating major land cover classes. Markers highlighted in dots: red = built-up, green = TV, blue = water, light green = SV, and yellow = BS (Courtesy: Google Earth). (b) False colour representation of preprocessed SAR data. ...................................................................................................... 83

Figure 4 . 84 Figure 4 . 3 84 Figure 4 . 4 Figure 4 . 5 86 Figure 4 . 6 88 Figure 4 . 7 90 Figure 4 . 8

 48443844445864688479048 Figure 4.2 (a) Google Earth imagery of the Haridwar with markers indicating major land cover classes. Markers highlighted in dots: red = built-up, green = TV, blue = water, light green = SV and yellow = BS (Courtesy: Google Earth). (b) False colour representation of preprocessed SAR data ....................................................................................................... 84Figure 4.3 Areas highlighted in white are villages in the (a) Roorkee, and (b) Haridwar study areas. Village areas are extracted from Google Earth imagery of the corresponding regions. (Courtesy: Google Earth). ................................................................................................... 84 Figure 4.4 Flowchart of the proposed approach.......................................................................... 86

Figure 4 94 Figure 4 . 97 Figure 4 . 98 Figure 4 . 98 Figure 5 . 1 107 Figure 5 . 2 107 Figure 5 . 3 108 xxxFigure 5 . 4 110 Figure 5 . 5 : 112 Figure 5 . 6 118 Figure 5 . 7 119 Figure 5 . 8 121 Figure 6 . 1 127 Figure 6 . 2 128 Figure 6 . 3 A 130 xxxiFigure 6 . 4 A 130 Figure 6 . 5 131 Figure 6 . 6 132 Figure 6 . 7 133 Figure 6 . 8 134 Figure 6 . 9 A 136 Figure 6 . 151 Figure 6 . 152 Figure 6 . 153 Figure 6 . 154 Figure 6 .

 494497498498511075210753108541105511256118571195812161127621286313064130651316613267133681346913661516152615361546 Figure 4.9 (a) False colour composites for Roorkee using

10 : 132 Table 6 . 3 133 Table 6 . 4 134 Table 6 . 5 139 Table 6 . 6

 1013263133641346513966 Classification performance indicators for the proposed method and RBF-SVM classifier with PSCFs as features. ....................................................................................... 71 Table 3.11 Summary of PSs-CNN model configuration, hyperparameters and training ........... 74 Table 3.12 Classification performance indicators for the PSs-CNN, NSCM, and NED classifiers with PSs as features.. BS = bare soil, TV = Tall vegetation, SV = Short vegetation, BU = Built-up. .............................................................................................................................. 75 Table 3.13 Classification performance indicators for the optimal class boundary estimator based DT, PSs-CNN, and RBF-SVM classifier. .......................................................................... 76 Table 4.1. Summary of ground truth samples collected for the study. *BS = bare soil, TV = dense tall vegetation, SV = short vegetation. All points are acquired directly on the field by visual xxxiv inspection at selected GPS coordinates. Also, the total number of villages in both the study regions are listed here. ......................................................................................................... 84 Table 4.2 Summary of PSs-CNN model configuration, hyperparameters and training .............. 88 Table 4.3 Summary of the classification performance of the considered classifiers considered for Roorkee and Haridwar datasets. OA = overall accuracy. Rest = BS + Water + SV. .......... 91 Table 4.4 summary of quantitative assessment of performance of PSS-CNN, NED, and NSCM ............................................................................................................................................. 96 Table 5.1 Summary of the ground truth collected for study. *BS = Bare soil, TV = Tall vegetation, SV = Short vegetation. All points are acquired directly on the field by visual inspection at selected GPS coordinates. ................................................................................................. 108 Table 5.2 1D-CNN-v1, 1D-CNN-v2, 2D-CNN-v1, and 3D-CNN-v1 CNN model configurations, hyperparameter and training settings. ............................................................................... 111 Table 5.3 Perm-LS-CNN and Perm-LSS-CNN model configurations, hyperparameters and training settings. ................................................................................................................ 115 Table 5.4 Separability index (SI) values for all the possible class pairs for the 2D CNN-v1, 3D CNN-v1, and 'Perm-LSS-CNN filters. ............................................................................. 116 Table 5.5 Classification performance of various classifiers considered for PolSAR-MS Roorkee dataset. *AD is the difference between precision and recall, averaged over all the land cover classes considered. BS = bare soil, TV = tall vegetation, SV = short vegetation, BU = builtup, W = water. ................................................................................................................... 120 Table 5.6 Classification performance of various classifiers considered for PolSAR-MS Haridwar dataset. *AD is the difference between precision and recall, averaged over all the land cover classes considered. BS = bare soil, TV = tall vegetation, SV = short vegetation, BU = builtup, W = water. ................................................................................................................... 120 Table 6.1 Geographical extent of study area. ............................................................................ 131 Table 6.2 Sentinel-1 and Sentinel-2 data acquisition dates. ..................................................... Band specifications of Sentinel-1 and Sentinel-2. .................................................... Summary of ground truth samples collected during ground truth survey. ................ Architecture and hyperparameter settings of the 1D-CNN-v1, 2D-CNN-v1, and 3D-CNN-v1 models for MSTS images based crop classification. .......................................... Architecture and hyperparameter settings of the RNN-v1 model for MSTS images based crop classification. .................................................................................................. 139

Figure 1 . 1

 11 Figure 1.1 Landsat-8 pixel of 30m spatial resolution.

Figure 1 . 2

 12 Figure 1.2 Improvements in image spatial resolutions over time.

Figure 1 . 3

 13 Figure 1.3 Demonstration of the concept of radiometric resolution. The larger the bit depth, the higher the radiometric resolution (more number of intensity levels).

Figure 1 . 4

 14 Figure 1.4 The temporal resolution of a satellite imaging system refers to how often it records imagery of a particular area. This example depicts the systematic collection of data every 30 days, presumably at approximately the same time of day.

  While most historical SAR systems operated in C-band, future SAR sensors are focused on the L-band frequency range. Although L-band SARs do not provide the high-resolution capabilities of shorter wavelength SARs, their ability to penetrate vegetation holds several advantages for earth observation. With a higher likelihood of seeing the ground underneath vegetation cover, L-band SARs are useful for mapping activity underneath canopies such as flooding or troop movements. Due to the high penetration into vegetation covers, L-band SAR can also characterise canopy structure, especially in dense vegetation. The higher canopy penetration ability of L-band SARs is also advantageous to interferometric SAR (InSAR) users in terms of achieving higher interferometric coherence and better deformation tracking. P-band SAR sensors are currently under development. The major issue with P-band SAR's data is 'ionospheric distortions'. These ionospheric distortions can adversely hamper the quality of data. However, recent developments in ionospheric correction strategies have allowed spaceborne P-band SAR missions to go forward. The first spaceborne P-band SAR-ESA's BIOMASS mission is planned to launch in 2021 and will focus on mapping the status and dynamics of earth's forests.

(

  backscatter) due to weak sensor isolation or speckle noise and therefore often results in suboptimal land cover classification with traditional classifiers in mixed class scenarios. Mixed class scenarios refer to study regions where land cover classes considered for classification are small, unorganized, and are scattered randomly. Due to this landscape two or more different land covers appear similar in a satellite image. For example, built-up areas having building structures close to each other and are randomly oriented (with respect to SAR sensor flight path), might show volume scattering along with double bounce scattering thus behaving as mixed class. Numerous techniques are available to perform land cover classification with satellite PolSAR images.

3 .

 3 For example, Figure 1.6 indicates the power of deep learning to learn directly from data. The deep learning based models improve their performance with increase in availability of training data in contrast to other machine learning techniques that are unable to learn beyond a certain point no matter how much training data is available. This means that, the more the training data, the better the deep learning based model performs. Another advantage of deep learning is its independency from hand-crafted features. Traditional machine learning techniques require hand-crafted features (not strictly) for learning. Figure 1.7 illustrates this advantage. These attributes or abilities of deep learning based models placed them in prominent position to be widely used for satellite image processing.Although numerous techniques and methods are available that are based on deep learning and DNNs, there is a general framework at play. Figure1.8 illustrates the general framework of DNN based models for satellite image analysis and processing. The framework includes three main components; an input data, a core deep networks, and expected output. In practice, the input-output pairs are dependent on a particular application. When the input-output pairs are properly defined, the intrinsic and natural relationship between the input and output is then constructed by a deep network composed of multiple levels of linear and nonlinear operations and a deep learning algorithm. When the core deep network has been well trained, it can be employed to predict the expected output of an input (test) whose output is never seen by the core deep network.

Figure 1 . 6

 16 Figure 1.6 Comparison of performances of traditional machine learning and deep learning techniques with respect to amount of training data availability.

Figure 1 . 7

 17 Figure 1.7 Figure illustrates that deep learning models can directly learn from raw data. The feature selection/extraction process is therefore redundant and sub-optimal in these models [71].

Figure 1 .

 1 Figure 1.8 A general framework in DNN based models for satellite image analysis and processing.

Task 3 :Task 4 :

 34 Development of novel spectral and spectral-spatial CNN models for multisensor (PolSAR and multispectral) images based land cover classification. Sub-tasks:  Study of conventional CNN models for multisensor (PolSAR and multispectral) image based land cover classification.  Development of novel CNN models to achieve better generalization performance by efficiently utilizing spectral and spatial information from multisensor image. Development of novel CNN, RNN, and CNN-RNN models for multisensor satellite time series images based crop classification. Sub-tasks:  Study of conventional CNN and RNN models for multisensor satellite time-series images based crop classification.  Development of novel CNN and RNN models with improved performance in multisensor satellite time-series images based crop classification.by synergistically exploiting spectral, spatial, and temporal profiles of crops.

  final component of any research is the deployment of developed model and evaluation of results obtained. Accuracy and generalization are key evaluation parameters considered in the present research. Results are evaluated on multiple study areas. Generalization performance is evaluated via visual inspection of the results especially in task 3, and 4.

Figure 1 . 9 Framework of the research 1 . 8 .

 1918 Figure 1.9 Framework of the research

of 42 42 

 42 for 2D convolution. Similar studies utilizing feature reduction prior to DL based feature extraction or classification are reported in [104]-[107]. Three-dimensional hyperspectral image simultaneously involves rich spectral information and spatial information. Neither spatial feature classification or spectral feature classification frameworks takes full advantage of this intertwined character of hyperspectral imagery. Deep learning based spectral-spatial classification strategies are more suitable for hyperspectral images and can lead to the improvement of classification accuracy. There are two main strategies to combine the spectral and spatial information present in hyperspectral images;

  unimodal satellite image processing. Most DL based studies focussed to either PolSAR, hyperspectral, and multispectral satellite images. DNN models have been developed and utilized with PolSAR images for land cover classification, sea ice monitoring, speckle filtering, and change detection applications. On the other hand, spectral and spectral-spatial characteristics based hyperspectral image classification seems to be popular recently. Initially, feature reduction is opted before the DNN model based classification process but later on, hyperspectral images are used directly for classification. 1D-, 2D-, and 3D-CNNs are popularly used with hyperspectral images in various applications including land cover classification. In case of multispectral satellite images, spatial and temporal characteristics are more focussed and are exploited with the help of 2D-CNNs (spatial), and 3D-CNNs and RNNs (spatial, temporal).

  This section presents a brief review of the developments in DNNs based multisensor/multimodal satellite image processing and analysis. Multisensor here refers to a set of at least two modalities of satellite images. Multimodal satellite image processing falls under the umbrella of multisensor satellite image processing with image sources from two different modalities. The most popular multi-modal satellite image set is a stack of optical-PolSAR satellite images. Studies on multisensor satellite image processing with DNNs are reviewed in this section with a focus on land cover classification application.Land cover classification with multisensor satellite images is achieved via two modes. In the first mode, the images from multiple sources are fused together via a fusion process (data fusion), and then the fused image is used as input during the classification process. In the other mode, images from multiple sources are directly used in a stacked manner during the classification process. The development in DNNs based methodologies in both these modes is briefly reviewed here.Data fusion is one of the fast-moving areas of satellite image processing. Due to recent increases in multisensor data availability, using heterogeneous and big data for earth observation has become more tangible. In this regard, DNNs are natural candidates to tackle the challenges of modern satellite image data fusion. Pansharpening, feature level, and decision level information fusion are the three major strategies of data fusion.

  network to upsample the intensity component obtained after the intensity, hue, and saturation transformation of the red, green, and blue images. Once the multispectral bands have been upsampled with the CNN, a traditional Gram-Schmidt transform is used to perform the pansharpening. The authors simply performed the interpolation operation with a CNN instead of using a nearest neighbour or bicubic interpolation function. Around the same time, authors in[START_REF] Masi | Pansharpening by convolutional neural networks[END_REF] used a three hidden layered CNN framework to learn the pansharpening operation end to end. In short, the CNN performs the complete pansharpening process. in this model, authors stacked the panchromatic band with the upsampled spectral bands and then, for each patch, the high resolution values of the central pixel are learned. This method eliminated the element of external intervention in the process and the requirement of alternative methods in the pansharpening process. They used the architecture on GeoEye-1 (panchromatic-multispectral pair), IKONOS-HRV (panchromatic-multispectral pair), and WorldView-2 and achieved satisfactory results. Moreover, the authors in[START_REF] Yuan | Hyperspectral image superresolution by transfer learning[END_REF] fine-tuned a superresolution CNN model pretrained on natural images for a set of hyperspectral images. They attempted transfer learning[START_REF] Tuia | Recent advances in domain adaptation for the classification of remote sensing data[END_REF] between the spectral domains of natural images (three bands, large bandwidths) and the spectral domains of hyperspectral images (many bands, narrow bandwidths).Most of the current literature dealing with DNNs based satellite image processing, addresses the problem of image classification (e.g. land cover classification). In the case of multisensor or multimodal images, land cover classification is performed by fusing information from multiple sources at either feature level or at decision level. A very simple way of using multisensor images in DNN models for land cover classification or any other application for that matter is to stack or concatenate the multisensor images and process them further as one. The filters learned by the first layer of the DNN, in this case therefore, depends on the order of stack of different sources. Such models have been used in satellite image patch classification. Fully convolutional and de-convolutional models are good examples of such models and have been used in multisensor satellite image classification (patch level)

  spatial filters extracted from hyperspectral image bands and digital surface model (DSM) image.. Another series of studies which include an explicit decision level fusion layer between land cover class label images. The studies either perform the decision fusion post classification or the decision fusion process is learned directly. If the classification is performed with two different classifiers one of which is a DNN based model, the different class labels are fused posterior to classification. Two studies are particularly notable in this regard. The authors in[START_REF] Paisitkriangkrai | Semantic labeling of aerial and satellite imagery[END_REF] fused classification maps obtained from CNNs with classification maps obtained from random forest (RF). The RF classifier is trained on handcrafted features. The two maps are fused by multiplication of the posterior probabilities. An edge-sensitive conditional random field is also learned on top to improve the quality of the final labelling. Whereas the authors in[START_REF] Marcos | Geospatial correspondences for multimodal registration[END_REF] learned an ensemble of CNNs and then averaged their predictions. Their proposed processing framework has two main streams: -first, processing the colour infrared data from the Vaihingen dataset, and -second, processing the digital surface model. They trained an ensemble of CNNs and further used them as inputs for each layer of the primary model's activation maps. They obtained a series of land cover maps to nourish the ensemble. This strategy improved the performances by considering classifiers issued from different data sources and levels of abstraction. This model has the advantage of being entirely learned in an end-to-end fashion. However, it incurs a heavy computational load and involves a complex architecture with numerous fusion layers and skip connections. Data fusion is not only about fusing image data with the same viewpoint.Multimodal satellite image data that exceed these restrictive boundaries and approaches to tackle new, exciting problems with satellite image processing are beginning to appear in the literature. In summary, DNNs based multisensor satellite image processing, analysis, and applications have significant potential and future. Notably, multisensor satellite image based land cover classification with DNNs provides higher accuracy, less fine-tuning, high dimensional data operability, and scalability. A model developed for a particular multisensor satellite image set can be used with other multisensor satellite image sets with slight modifications. Although, data fusion approach for multisensor satellite image applications seems to be popular right now but direct utilization of multisensor satellite images with DNNs is gaining attention. Few research gaps identified in multisensor satellite image processing with DNNs are discussed briefly in section 1.5.

  section presents a brief review of the developments in DNNs based satellite time series image processing techniques. Satellite time series images refer to a set of satellite images collected over time observing the same area of interest with the same sensor. If more than one sensor is considered, then the sensors may belong to same modality (unimodal) or different ones(multimodal). The review includes studies on both, unimodal and multisensor satellite time series images. However, in this section, the review is presented in the order of DNN technologies used in satellite time series image processing. The popular techniques include CNNs, RNNs and combined CNNs-RNNs.CNNs are the most popular DNN technology in satellite time series image processing. In the recent years, numerous techniques have been developed for satellite time series image processing using 1D, 2D, and 3D CNNs. 1D CNNs are first utilized for Landsat-8 satellite time series image processing in[START_REF] Mauro | End-to-end learning of deep spatio-temporal representations for satellite image time series classification[END_REF]. They used 1D-CNNs with multilayer perceptron inspired from[START_REF] Zheng | Time series classification using multichannels deep convolutional neural networks[END_REF] to perform land cover classification. Zhong et al. in[START_REF] Zhong | Deep learning based multi-temporal crop classification[END_REF] developed two types of deep learning models; first, a 1D CNN based satellite time series crop classification model with inception blocks within the classification framework and second, a long short term memory (LSTM) RNN based crop classification model. The 1D-CNN with inception blocks model performed well in contrast to basic LSTM-RNN model for satellite time series image based crop classification. Pelletier et al. in [73] proposed a temporal CNN model to perform satellite time series image based crop classification. The model focuses the significance of temporal convolution in time series image classification. They also evaluated the effects of appending hand crafted features such as vegetation indices into the classification model. Their model performed satisfactorily over random forests and RNNs. Further, 3D-CNNs have also been utilized for satellite time series image processing in many studies. 3D CNNs have the ability to capture spectral, spatial, and temporal features simultaneously whereas 1D and 2D CNNs lack such ability. A satellite multi-temporal images based crop classification model is proposed and developed in[START_REF] Ji | 3D convolutional neural networks for crop classification with multi-temporal remote sensing images[END_REF] which uses 3D-CNNs as spatiotemporal feature extractors. It is validated from the study that 3D-CNNs are better spatiotemporal feature extractors than 2D or 1D CNNs for satellite time series images. A 3D CNN-transformer hybrid approach for crop classification using multitemporal Landsat-8-Sentinel-2 images is developed in[START_REF] Li | A CNN-transformer hybrid approach for crop classification using multitemporal multisensor images[END_REF]. The transformer architecture utilizes multi-head self-attention module to represent the sequence pattern which improves the classification performance in contrast to conventional RNN based crop classification modules.RNNs are also popular DNN architectures to utilized with satellite time series images.Along with CNNs, RNNs have also gained interest in satellite time series image applications.The first study which utilized RNNs with satellite time series images is proposed in 2016[START_REF] Lyu | Learning a transferable change rule from a recurrent neural network for land cover change detection[END_REF].They developed a transferable change rule algorithm based on LSTM-RNNs for detecting change in land cover. Their comparison with SVM-CNN and decision tree signified the better performance of the proposed model. Later, a LSTM-RNN based model is developed in[START_REF] Rubwurm | Temporal Vegetation Modelling Using Long Short-Term Memory Networks for Crop Identification from Medium-Resolution Multi-spectral Satellite Images[END_REF] for crop classification. The model is evaluated on Sentinel-2 multispectral time series images and performed well in comparison to CNNs and SVMs. A similar model is proposed in[START_REF] Ienco | Land cover classification via multioral spatial data by recurrent neural networks[END_REF] for pixel as well as object based land cover classification. This model is evaluated over two different multispectral datasets and results indicated improved and stable performance. Moreover, RNNs have also been utilized for PolSAR time series image based crop classification in[START_REF] Ndikumana | Deep recurrent neural network for agricultural classification using multitemporal SAR Sentinel-1 for Camargue, France[END_REF] and[START_REF] Tong | Deep recurrent neural networks for winter vegetation quality mapping via multitemporal SAR sentinel-1[END_REF].Both, LSTMs and gradient recurrent unit (GRU) based RNN models have been developed and tested with Sentinel-1 time series images. Both studies indicate to a slightly better performance of GRU over LSTMs.Sun et al. in [133] used ancillary time series cropland data layer along with Landsat-8 time series images and further used LSTM-RNN model over the combined dataset to perform land cover classification. Another study which uses a patch or window of pixel instead of per-pixel or object based in classification process is proposed in[START_REF] Sharma | Land cover classification from multi-temporal, multispectral remotely sensed imagery using patch-based recurrent neural networks[END_REF]. The study is developed on Landsat-8 time series images and LSTM-RNN architecture is used. The patch based land cover classification model benefits from the contextual information present in the patch which otherwise is absent in pixel based classifications.CNNs and RNNs have also been used synergistically and successfully in satellite time series image processing. A 2D-CNN-RNN architecture is developed in[START_REF] Benedetti | M3 fusion: A deep learning architecture for multiscale multimodal multitemporal satellite data fusion[END_REF] to perform data fusion with multimodal, multiscale, and multitemporal satellite images. The satellite time series images are Sentinel-2 whereas a very high spatial resolution SPOT-6/7 image is also used. In another study, a temporal attention network is developed for multisensor and multitemporal satellite time series image based crop classification. Attention mechanism is a brain mechanism of human signal processing. The study utilizes attention network in the temporal dimension to reinforce the phenological differences in crops and restraining their phonological similarity. Further, in[START_REF] Gadiraju | Multimodal deep learning based crop classification using multispectral and multitemporal satellite imagery[END_REF], CNNs are used with RNNs in a parallel architecture to extract spectral-spatialtemporal features from a combination of MODIS time series and NAIP high spatial resolution dataset. Finally, a SVM and multilayer perceptron models are used to perform crop classification.CNN-RNN models have also been used in change detection applications. In[START_REF] Mou | Learning spectral-spatialoral features via a recurrent convolutional neural network for change detection in multispectral imagery[END_REF], authors used a recurrent convolutional cascade model for learning spectral-spatial and temporal features from Landsat -7 time series multispectral images. The learned features are then used to detect changes with a binary classification module. In summary, three major DNN model categories i.e. CNNs, RNNs, and CNNs-RNNs have been successful in analysing and processing satellite time series images for various applications. The applications include land cover classification, change detection and most importantly, agriculture crop classification. Although CNNs and RNNs have performed well with satellite time series images in standalone modes, synergistic utilization of both models either in parallel or in cascade fashion, results in superior performance. The use of CNN-RNN architecture in feature extraction and classification with multisensor satellite time series images for crop classification is very motivating. Research gaps in DNN based multisensor satellite time series image processing are discussed briefly in section 1.5.

  forests, water bodies) and man-made (e.g. buildings, bridges, roads) targets. These land covers show complex scattering response and poses a challenge in classification using satellite PolSAR data. Target decomposition based techniques aims at analysing the received scattering matrix from polarimetric data to extract information about the scattering processes exhibited by various land covers. These techniques compute particular features (e.g. decomposition features) from the backscattering response of land covers (recorded by the PolSAR sensor) and further use these features to detect, identify or, classify land covers. An extensive study on the performance of various decomposition based features in land cover classification is done in[START_REF] Mishra | Critical analysis of model-based incoherent polarimetric decomposition methods and investigation of deorientation effect[END_REF]. They observed an absence of dominant decomposition based features for different land covers in mixed land cover scenario. Mixed class scenarios refer to landscape (study regions) where land cover classes considered for classification are small, unorganized, and are scattered randomly. Due to this landscape two or more different land covers appear similar in a satellite image. For example, built-up areas having building structures close to each other and are randomly oriented (with respect to SAR sensor flight path), might show volume scattering along with double bounce scattering thus behaving as mixed class. Researchers are also developing new multi-component decomposition and factorization based features that are able to segregate land covers more efficiently but the challenge still needs more attention[START_REF] Ratha | A PolSAR scattering power factorization framework and novel roll-invariant parameters based unsupervised classification scheme using a geodesic distance[END_REF],[START_REF] Kumar | Three-component decomposition technique for hybrid-pol SAR data[END_REF]-[START_REF] Wei | A multi-component decomposition method for polarimetric SAR data[END_REF] Impact of various target decomposition base features on land cover/use classification performance is analysed in[START_REF] Turkar | Effect of different target decomposition techniques on classificatio accuracy for polarimetric SAR data[END_REF].They merged land cover/use classes that are not separable with given dataset and features. For example, agriculture and bare soil are combined and treated as one class. Similarly, rural settlements and dense tall vegetation are combined and treated as one class. This indicates to the challenge to segregation various land covers in a mixed land cover scenario. An example of mixed land cover scenario is shown in Figure 3.1. For example, in Figure 3.1(b), marker A indicates rural settlements (built-up area) and marker B indicates dense tall vegetation patches.

  utilized deep learning techniques for land cover (LC) classification with the help of synthetic databases for learning. Their attempt is motivating and points to the potential of deep learning techniques in extraction of information form PolSAR images. These studies motivate to explore the use of deep learning and DNNs with PolSAR data for various applications. The study presented in this chapter to propose a CNN based approach for PolSAR PSs based land cover classification.

Figure 3 .

 3 Figure 3.1 (a) False colour composite generated from PALSAR-2 data. Colour scheme: red = HH, green = HV, and blue = VV. (b) Panchromatic image collected from Google Earth dated Feb 19, 2015. Symbol 'A' marks built-up area in both the images and symbol 'B' marks the dense vegetation area in both the images. (c) The enlarged segments are shown in middle in the outlined boxes. The built-up area is showing volume scattering (high backscatter response in HV, refer) along with double bounce scattering.

  scattering matrix. Computation of K matrix parameters is presented in Appendix-A. k is the propagation constant. Co-polarized signature is obtained by transmit and received combination of ij   and ij   , and cross polarized signature of 90 ij   , and ij   .Ellipticity angle defines the polarization behaviour (linear, circular or elliptical polarization), while orientation or tilt angle in combination with ellipticity angle, defines polarization states, for example, horizontal or vertical polarization[START_REF] Lee | Polarimetric property preservation in SAR speckle filtering[END_REF]. For the sake of visualization, co-polarized PS of an arbitrary land cover is shown in Figure3.2.

Figure 3 . 2

 32 Figure 3.2 Co-polarized polarization signature of an arbitrary land cover.

Figure 3 .

 3 1(b) depicts the Google Earth imagery of the study area. This topography makes the study area a mixed land cover scenario in terms of land cover classification.

  Ground truth sample points are collected for training and validation. Five land cover classes are considered namely bare soil (BS), tall vegetation (TV), short vegetation (SV), built-up (BU), and water. Ground truth class sample points are measured directly on the terrain based on visual inspection. Approximately 1200 data points are collected for both the study areas. Details of the ground truth collected for both study areas are shown in Table 3.1. Also, Figure 3.3 depicts samples of ground truth collected during the study.

Figure 3 . 3

 33 Figure 3.3 Sample images of different land covers collected during ground truth survey. (a) Tall vegetation, (b) bare soil, (c) agriculture cropland, and (d) sugarcane field.

Figure 3 .

 3 Figure 3.1(a) show a false colour composite image of the pre-processed Roorkee study area. It is observed from the false colour composite image that some parts of built-up areas are showing volume scattering (represented by green colour) behaviour similar to dense vegetated areas such as tree plantations. For example, in Figure 3.1(a), built-up area (marked as A) is showing backscatter behaviour similar to backscatter behaviour of tree plantations (marked as B). It can be further observed from enlarged patches shown in Figure 3.1(c). It is because of the randomly oriented, densely packed and non-similar structures constitute the built-up area and hence is showing volume scattering as dominant scattering. The main objective of the current study is to develop features and classification techniques that can overcome this challenge and achieve higher classification performances.

  standard targets namely Flat Plate (FP), horizontal dipole (HD), vertical dipole (VD), and dihedral (Di) are considered as standard targets. Geometrical structures, scattering matrices, and PSs of these standard targets are shown in Figure 3.5. The PSs of these standard targets are further utilized in representing PSs of natural targets (observed or image-pixel PSs). Further details about standard targets are provided in [174]. PSs, both observed (image pixel) and standard target are obtained using equation (3.1). The PSs are obtained for ellipticity angle  ranging from 45   to 45   and orientation angle  ranging from 90   to 90   with a step size of 5  in each direction. Each obtained PS is a 18 37  two dimensional matrix.

Figure 3 . 4

 34 Figure 3.4 Flowchart showing polarization signature correlation features (PSCFs) computation using fully polarimetric SAR data.

Figure 3 . 5

 35 Figure 3.5 Geometrical structures, scattering matrices, and co-polarization signatures of different standard targets.

Figure 3 . 6

 36 Figure 3.6 Figure illustrates the computation of polarization signature correlation features (PSCFs) between observed (image pixel) and standard targets PSs.



  are standard deviation for class i and j respectively.

(

  SVM) with radial basis function (RBF) kernel because of its inherent optimum classification capability. All the classified images are shown in Figure 3.7. Overall classification performance indicators are provided in Table

Figure 3 . 7 :

 37 Figure 3.7: Classified Image; (a) M-Y4D-SVM, (b) GFD-SVM, (c) Cloude-SVM, (d) H-A-Alpha-SVM, (e) PSCF-SVM (proposed), and (f) Google Earth image for reference purpose. Colour scheme; red = built-up, light green = agriculture or SV, green = TV., blue = water, and yellow = bare soil. Marker A indicates built-up area and marker B indicates tall vegetation area.

Figure 3 . 8

 38 Figure 3.8 Correlation images (a) corr_co_HD and (b) corr_co_VD.

  high-quality solutions to optimization and search problems by relying on biologically inspired operators such as mutation, crossover and selection. They imitate the biological processes of reproduction and natural selection to solve for the 'fittest' solutions. Like evolution, many of a genetic algorithm's processes are random, however this optimization technique allows one to set the level of randomization and the level of control. These algorithms are far more powerful and efficient than random search and exhaustive search algorithms yet require no extra information about the given problem.

𝑛 5 =

 5 -2.0, -2.0 ≤ 𝑛 6 ≤ 1.0 (3.13d) -1.5 ≤ 𝑛 7 ≤ -1.0, -0.5 ≤ 𝑛 8 ≤ 0 (3.13e) -2.0 ≤ 𝑛 9 ≤ -0, -1.0 ≤ 𝑛 10 ≤ -1.5 (3.13f) The upper and lower bounds of i n (i = 1 to 10) values are obtained via experimental study during developing the empirical relation formulation as discussed in section 3.5.3 and are provided in equation 3.13(a-f). The internal parameters to GA are finalized using trial and error method.

  5.4 are used to compute optimal class boundary values (using equation (3.5) through (3.9)) and henceforth, these class boundary values are used to develop a decision tree classification scheme. The decision tree based on the optimal class boundary values, in turn based on optimal i n (i = 1 to 10) values is illustrated in Figure 3.10.

Figure 3 . 9

 39 Figure 3.9 Flowchart illustrates the development process of the optimal class boundary estimator based decision tree classifier.

Figure 3 .

 3 Figure 3.10 Decision tree based on optimal class boundaries values.

5 . 5 (

 55 refer Figure 3.10) is utilized to perform LC classification on Roorkee dataset. The classified image is shown in Figure 3.11(a). The proposed optimal class boundary estimator based DT classifier is compared with the RBF-SVM classifiers. The same PSCFs are used as features with the RBF-SVM. Classified image the RBF-SVM is shown in Figure 3.11(b) respectively. Overall classification performance indicators for the proposed method and RBF-SVM are provided in

Figure 3 .

 3 Figure 3.1 are clearly separated using the proposed optimal class boundary approach (refer figure 3.11(a)).

Figure 3 . 11 :

 311 Figure 3.11: Classified image: (a) Optimal class boundary estimator based DT classifier, (b) RBF-SVM, and (c) Google Earth image for reference purpose. Colour scheme; red = built-up, light green = SV, green = TV, blue= water, and yellow = BS. Marker A indicates built-up area and marker B indicates tall vegetation area.

Figure 3 .

 3 Figure 3.12 demonstrates the utilization of two dimensional convolutions on PolSAR PSs.Figure

Figure 3 .

 3 Figure 3.12 demonstrates the utilization of two dimensional convolutions on PolSAR PSs.Figure 3.12(a) and Figure 3.12(b) depicts isometric view and top view respectively of the PS of a PolSAR image pixel. A 2D convolution filter as shown in Figure 3.12(d) convolves over the PS in a sliding window manner and provides convolutional features. These features can be used in classification, regression or recognition.

Figure 3 . 12 :

 312 Figure 3.12: Two dimensional convolution over polarization signature. (a) Polarization Signature of a PolSAR image pixel. (b) Top view. (c) Polarization signature subset. (d) Convolutional filter.

Figure 3 .

 3 Figure 3.13 Architecture of the proposed PSs-CNN model.

Figure 3 .

 3 Figure 3.14(a) is the classified image for using the PSs-CNN model as classifier. To assess the classification performance of proposed PSs-CNN model, the PSs-CNN model performance is compared with two popular PSs based land cover classifiers i.e. the normalized signature correlation mapper (NSCM) based minimum distance classifier and the normalized Euclidean distance (NED) based minimum distance classifier.Figure 3.14(b) is classified image obtained

Figure 3 .

 3 14(b) is classified image obtained using the NSCM classifier whereas Figure 3.14(c) is the classified image obtained using the NED classifier. It is evident form Figure 3.14 that PSs-CNN outperforms the NSCM and NED based classifiers. Table3.12 summarizes the quantitative assessment of classification performance of PSs-CNN, NED, and NSCM classifiers. The PSs-CNN classifier performs land cover classification with good overall accuracies, i.e. 72.8% with kappa coefficient of 0.69. In contrast, the NED performed the classification with 62.3 % and the NSCM performed the classification with 64.1%. Both NED and NSCM showed low kappa values (refer Table3.12). Because of the point-wise operability, the NED and NSCM classifiers are not able to capture and exploit the neighbourhood relationship of a pixel and thereby are not able to efficiently address the classical classification challenges with PolSAR data such as, Built-up vs Tall veg, where volume scattering is dominant[START_REF] Garg | Development of an efficient contextual algorithm for discrimination of tall vegetation and urban for PALSAR data[END_REF]. The proposed PSs-CNN classifier resolves the issue significantly by extracting and utilizing the subtle differences in the PSs of built-up and forest. Markers A and B (refer Figure3.1(a)) are clearly separated using the proposed approach (refer Figure3.14(a)).

Figure 3 .

 3 Figure 3.14 Classified Image: (a) PSs-CNN, (b) NSCM, (c) NED, and (d) Google earth image. Colour scheme; red = built-up, light green = SV, green = TV, blue= water, and yellow = BS. Marker A indicates built-up area and marker B indicates tall vegetation area.

chapter 3 .

 3 In the recent years, CNNs in combination with deep learning technology are being widely used in PolSAR data applications [58]. The utilization of CNNs with PSs for land cover classification has been introduced in chapter 3. The study in chapter 3 indicates the significance of CNNs as PSs feature extractors. The study also reports improvement in classification performance with utilization of CNNs with PSs for land cover classification. Therefore, the study presented here in chapter 4 explores and extends the utilization of CNNs with PSs in order to extract built-up areas from a PolSAR image.

__first,

  design and development of PSs based CNN model for extraction of built-up areas from PolSAR image which contains the flooded region, __ second, development of novel methodology for design of village detection filters based on apriori knowledge and image morphology, and __ third, development of novel methodology for identification, and mapping of flood affected villages using image morphology.Second, the total number of villages (a common term opted here to represent all types of human settlements in the study region) in each study area are counted for validation of the proposed approach. Table4.1 also lists the actual number of villages in the Roorkee and Haridwar study areas. Further, Figure 4.3(a, and b respectively) highlights (in white colour) the villages in the Roorkee and Haridwar study regions captured from Google Earth imagery via a digitization process. Any village in the rural parts of India consists of a minimum of 20 houses and is spread over a minimum area of 7600-meter square. These physical dimensions help in detection of villages in a PolSAR image. It is important to note here that in this chapter; different terminologies have been used at different stages to represent different versions of built-up areas. Basic terminologies have been defined here.  Built-up structure: Man-made structures.  Built-up area: A collection of built-up structures.  Village or town: Built-up area larger than a predefined size.  Object: A connected group of image pixels.  Detected villages: Villages that have been successfully detected.  Flooded villages: Villages that are affected by flood. These terminologies help to better understand the different stages of the study. Also, in order to evaluate the performance of proposed approach, artificial flood scenarios are considered and used during the study. Details of the process of creating artificial flood scenarios are provided in section 4.3.3.2.

Figure 4 .

 4 Figure 4.1 (a) Google Earth imagery of the Roorkee with markers indicating major land cover classes. Markers highlighted in dots: red = built-up, green = TV, blue = water, light green = SV, and yellow = BS (Courtesy: Google Earth). (b) False colour representation of preprocessed SAR data.

Figure 4 .

 4 Figure 4.2 (a) Google Earth imagery of the Haridwar with markers indicating major land cover classes. Markers highlighted in dots: red = built-up, green = TV, blue = water, light green = SV and yellow = BS (Courtesy: Google Earth). (b) False colour representation of preprocessed SAR data

Figure 4 . 3

 43 Figure 4.3 Areas highlighted in white are villages in the (a) Roorkee, and (b) Haridwar study areas. Village areas are extracted from Google Earth imagery of the corresponding regions.(Courtesy: Google Earth).

Figure 4 . 5 . to 90   with a step size of 5 

 455 Figure 4.5. Each PS plot in figure 4.5 shows backscatter power response for ellipticity angle  ranging from 45   to 45   and orientation angle  ranging from 90   to 90   with a step size of 5  in each direction resulting in a 18 37  two dimensional matrix. It is evident from Figure 4.5 that PSs of considered land covers are different from each other.

  PolSAR PSs based land cover classification are distance/correlation based which do not consider the neighbourhood relationship of a pixel during classification and therefore PSs remain under-utilized. There is a need to explore and develop advanced classifiers that can capture local spatial relationship of a PS pixel during classification which might improve the classification. A CNN based PSs classification technique is introduced in chapter 3 which considers the influence of the neighbourhood of a PS pixel and thereby utilizes the PSs efficiently. The current study extends the utilization of PSs with CNNs to extract built-up areas from PolSAR image.

Figure 4 . 4 Figure 4 . 5

 4445 Figure 4.4 Flowchart of the proposed approach

(refer section 4 . 3 . 1 )

 431 image as input to the PSs-CNN model. This 54 111  input image has two channels i.e. copol and crosspol. Therefore, the overall size of input to the PSs-CNN is 54 111 2  . The PSs-CNN used five convolutional layers. The number of convolution filters and filter sizes used in each layer are listed in Table 4.2. ReLU activation function and dropout fraction of 10% is used in each convolutional layer. The dropout regularization strategy is employed to reduce overfitting and achieve better generalization. Further, two Fully Connected or FC layers, first with 50 nodes and second with 10 nodes are used. Both FC layers used the ReLU activation function and a dropout fraction of 10% . Finally, a Softmax classification layer with 3 output labels is used. The first output class is the built-up class which needs to extracted, the second is 'rest' class (flood), and the third output class is TV class because TV and built-up are the only classes that are not completely submerged in water during a flood. A batch normalization process is carried out before the convolutions to speed up the training process and to reduce the network sensitivity. Overall, an eight layered PSs-CNN classification model is realized as shown in Figure 4.6. mapper (NSCM) based minimum distance classifier and the normalized Euclidean distance (NED) based minimum distance classifier.

Figure 4 .

 4 Figure 4.7(e) and Figure 4.7(g) shows classified images using the NSCM classifier (minimum distance is considered as classifier) for Roorkee and Haridwar respectively whereas

Figure 4 .

 4 Figure 4.7(i) and Figure 4.7(k) shows the classified images using the NED classifier (minimum distance is considered as classifier) for Roorkee and Haridwar respectively. It is evident form

Figure 4 . 1 ,

 41 Figure 4.1, Figure 4.2, and Figure 4.7(a, c, e, g, i, and k) that the PSs-CNN outperforms the NSCM and the NED on both, the Roorkee, and the Haridwar datasets. Because of the point-wise operability, the NED and NSCM classifiers are not able to capture and exploit the neighbourhood relationship of a PS pixel and thereby are not able to efficiently segregate the built-up areas as discussed in section 4.1. The developed PSs-CNN classifier resolves this issue significantly by extracting and utilizing the subtle differences in the PSs of built-up and TV and successfully

Figure 4 . 7

 47 Figure 4.7 Roorkee classified image obtained using: (a) the PSs-CNN classifier, (e) the NED classifier, and (i) the NSCM classifier. Areas classified as built-up in Roorkee study area using: (b) the PSs-CNN classifier, (f) the NED classifier, and (j) the NSCM classifier. Haridwar classified image obtained using: (c) the PSs-CNN classifier, (g) the NED classifier, and (k) the NSCM classifier. Areas classified as built-up in Haridwar study area using: (d) the PSs-CNN classifier, (h) the NED classifier, and (l) the NSCM classifier. Colour scheme: red = built-up, green = TV, and blue = rest (Water + BS + SV).

  3.2). The methodology utilizes apriori knowledge and image morphology to build village detection filters. These filters detect villages in the built-up area image and significantly remove false alarms or misclassifications. Development of this methodology is discussed under stage 2 in section 4.3.3.1. Stage 3 presents a novel methodology to identify and map villages that are affected by flood and mask out villages that are not affected by flood. This methodology also utilizes image morphology to map flood affected villages. Development of this methodology is discussed in section 4.3.3.2.

Figure 4 . 8 .

 48 A segment of built-up area image shown in Figure 4.7 is shown in Figure 4.8(a).

  Figure 4.8(b) depicts the effects of closing operation on the segment. It can be observed from Figure 4.8(b) that some objects are merged together.

Figure 4 .

 4 Figure 4.8(d) depicts output of the area based filtering. It is observed from Figure 4.8(d) that many unwanted objects have been removed successfully. The objects remained in the filtered image are successfully detected as villages.The closing, hole filling, and the area filter operations in sequence act as a 'detection filter' which detects villages in a built-up area image and mask out false alarms or misclassifications. This village detection filter is applied to the built-up area images of the PSs-CNN, NSCM, and NED.Output images after applying the village detection filter are shown in Figure4.9. These output image(s) which contains only the successfully detected villages is termed as "DV" image(s) hereafter in this chapter.The DV images corresponding to the PSs-CNN, NSCM, and NED classifiers are assessed using reference data. The reference images for both Roorkee and Haridwar are provided in section 4.2.3 (refer Figure4.3). The reference images contain all the villages in the corresponding study areas. Figure4.9 shows false colour composites (FCC) of the DV images of the PSs-CNN,

Figure 4 .

 4 [START_REF] Venkataraman | Fully polarimetric ALOS PALSAR data applications for snow and ice studies[END_REF] shows false colour composites (FCC) of the DV images of the PSs-CNN, NSCM, and NED, and reference images for Roorkee and Haridwar areas.

Figure 4

 4 In Figure 4.9(a), red colour objects are reference objects (actual villages), green colour objects are villages as per the PSs-CNN DV , and blue colour objects are villages as per the NED DV . Further, white colour objects are objects which are present in all three images i.e. the reference image, the the object is actually a village and is correctly extracted by both, the PSs-CNN and the NED. Yellow colour objects are villages as per the reference and the PSs-CNN but not as per the NED. Cyan colour objects are villages as per the reference and the NED but not as per the PSs-CNN. The abundance of yellow colour in Figure 4.9(a) indicates to the efficient extraction of built-up areas by the PSs-CNN classifier. Similarly, the PSs-CNN NSCM reference-DV -DV false colour composite for Roorkee shown in Figure 4.9(c) also evidences the better performance of the proposed PSs-CNN classifier. Haridwar in Figure 4.9(b) and Figure 4.9(d) show numerous green colour objects. These objects are falsely detected as villages as per PSs-CNN and indicate false alarms.

Figure 4 . 8

 48 Figure 4.8 Results from different morphological operations considered during village detection: (a) Segment of image shown in Figure 4.7(a). Corresponding segments of: (b) Image after the closing operation, (c) Image after the holes filling operation, and (d) image after the filtering operation.

Figure 4

 4 Figure 4.9 (a) False colour composites for Roorkee using

(4. 11 )

 11 FalsePositive(FP) Total number of objectsin test image TP (4.12) True positives (TP), true negatives (TN), and false positives (FP) are computed for Roorkee and Haridwar study areas for PSs-CNN, NED, and NSCM. Table 4.4 summarizes the quantities assessment of the DV images of the PSs-CNN, NED, and NSCM classifiers. The PSs-CNN successfully extracts 75 (out of 93) villages in the Roorkee study area and 47 (out of 60) villages in the Haridwar study area. Whereas, NED successfully extracts 26 villages in Roorkee and 20 villages in Haridwar, and NSCM successfully extracts 22 villages in Roorkee and 27 villages in Haridwar. The low detection rate in case of the NED and the NSCM resonates with their low classification accuracies for built-up class (please refer Table 4.2). The false alarms are also high for the NED and the NSCM. The low values of kappa for NED and NSCM (please refer Table4.2) indicate to the high false alarms. The PSs-CNN offers higher detection rate and low false alarms for both Roorkee and Haridwar study areas.

Figure 4 . 10 .

 410 Figure 4.10. The methodology is evaluated on three different scenarios on the DV image of Roorkee corresponding to the PSs-CNN. The three different scenarios are shown in Figure 4.11. In the first scenario i.e. scenario 1, flood has affected the whole study region (already discussed in section 4.3.2) whereas, in the second and third scenarios i.e. scenario 2 and 3, flood has affected parts of the study region (partial flooding). When the DV image of Roorkee corresponding to the PSs-CNN is subjected to these three scenarios, the resulting flood situations are shown in Figure 4.12(a, b, and c). It is clear from Figure 4.12(a) that all the villages are affected by flood whereas in Figure 4.12(b) and Figure 4.12(c), only few villages are affected. Employing the proposed methodology to the three flood situations results in identification and mapping of flood affected

Figure 4 .

 4 Figure 4.10 Methodology flowchart for flood affected village mapping.

Figure 4 .

 4 Figure 4.11 Different scenarios: (a) Scenario 1, (b) Scenario 2, and (c) Scenario 3. Colour scheme: blue = flooded region.

Figure 4 .

 4 Figure 4.12 Flood maps: (a) Affected villages under flood condition 1. (b) Affected villages under flood condition 2. (c) Affected villages under flood condition 3. Colour scheme: red = villages not affected by flood, yellow = villages affected by flood, and blue = flooded region.

Chapter 5 PERMUTED

 5 SPECTRAL AND PERMUTED SPECTRAL-SPATIAL CNN MODELS FOR POLSAR-MULTISPECTRAL DATA BASED LAND COVER CLASSIFICATION In contrast to chapter 3 and chapter 4 where novel DNN models are proposed and developed for unimodal satellite image, this chapter explores the performance of conventional DNN models with multisensor/multimodal satellite images. Also, based on the performance of these conventional models, novel DNN models are proposed here. In context to satellite based multisensor earth observation data, today, both synthetic aperture radar (SAR) and multispectral (MS) image data are easily available with good spatial and temporal resolutions. The two data modalities with different spectral characteristics complement each other in many earth observation data applications such as land cover classification, crop monitoring, and change detection. However, it is still a challenge to develop techniques, methods, or approaches which can process the two data modalities together efficiently and without losing information from either. This chapter presents a study which attempts to introduce deep learning and DNN based novel models for multisensor (, PolSAR-MS) satellite image data processing that utilizes localized convolutions in both spatial and spectral domains to perform land cover (LC) classification. The study also attempts to establish the significance of performing ensembled localized spectral convolutions in comparison to nonlocalized or localized spectral convolutions. Two novel models namely; permuted local spectral convolutional neural network (Perm-LS-CNN), and permuted local spectral-spatial convolutional neural network (Perm-LSS-CNN) are proposed in order to achieve better classification results in terms of both accuracy and generalization.

  two, and three dimensional convolutions to extract spectral and spatial information from unimodal earth observation image data. Moreover, CNNs have also been used with multisensor image data. For example, CNNs have been utilized with PolSAR-MS data in earth observation image data applications. In[START_REF] Kussul | Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[END_REF], authors used one and two dimensional CNNs on Landsat8-Sentinel1 multisensor image data for land cover and crop type classification and further compared the results with random forest (RF) classifier. Their model reports better classification performance by capturing local contextual (spectral and/or spatial) features whereas the RF classifier captured only the global dominant features. The study in[START_REF] Kussul | Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[END_REF], indicates potential of CNNs with PolSAR-MS multisensor image data for earth observation applications and motivates for similar studies. However, one important aspect which comes to consideration while processing this PolSAR-MS data using CNNs is the efficient exploitation of the spectral information present in the high dimensional multisensor (PolSAR-MS) data. The exploitation of spectral information is crucial for CNN based multisensor data applications. Exploitation of spectral information present in hyperspectral image has been done in past, either exclusively with one dimensional



  To critically analyse the one dimensional (1D), two dimensional (2D), and three dimensional (3D) CNN capabilities to perform land cover classification using multi-sensor (PolSAR-MS) data.  Development of novel classification frameworks based on 1D (spectral) or 3D (spectral-spatial) CNNs; in order to improve the generalization ability of the classifier during land cover classification with PolSAR-MS satellite data. 1D, 2D, and 3D CNN frameworks for PolSAR-MS data based land cover classification are discussed in section 5.4. The idea of extracting the spectral information efficiently from PolSAR-MS data using CNNs is achieved by performing a permuted localized spectral convolution alone, and along with localized spatial convolution on PolSAR-MS data. Spectral permutation before localized spectral convolution is proposed to increase the generalization ability of the CNN classifier. Based on this idea, two models are proposed, namely permuted local spectral CNN (Perm-LS-CNN) and permuted local spectral-spatial CNN (Perm-LSS-CNN). Both models are discussed in section 5.5. The remaining of the chapter is organized as follows; Section 5.2 provides a brief description on popular CNN models for earth observation image data processing. Section 5.3 provides information on the experimental setup i.e. study areas, datasets, ground truth, and other resources considered and used during the study. Section 5.4 discusses the development of conventional 1D, 2D, and the 3D CNN models for processing PolSAR-MS image data and presents internal configurations of these models. Section 5.5 presents two novel CNN based PolSAR-MS data processing frameworks for land cover classification. Results from the developed methods and compared classifiers are discussed in section 5.6. Finally, section 5.7 concludes the chapter.

5. 3 . 1 .

 31 Study areaTwo areas are considered for the analysis, development and testing of the proposed methodologies. The first study area is common from chapter 3 and chapter 4 (Roorkee study area). Details of this study area are presented section 3.3.1. This study area is termed as 'Roorkee study area' in this chapter. The second study area is also common from chapter 4 (Haridwar study area). This second study area is termed as 'Haridwar study area' in this chapter. Details of this study area are presented in section 4.2.1. Both study areas have mixed class type landscape, which is beneficial for the evaluation of classifiers performance in mixed class scenarios.

Figure

4. 1

 1 (a) and Figure 4.2(a) shows a Google Earth images of the Roorkee and Haridwar study areas respectively.

  Figure 5.1(a) and Figure 5.2(a) depicts a false colour composite from the pre-processed PolSAR image data for Roorkee and Haridwar study areas respectively.

Figure 5 .

 5 1b) and Figure 5.2(b) depicts a false colour composite from the pre-processed MS image data for Roorkee and Haridwar study areas respectively. The resampled, pre-processed MS bands are co-registered with the pre-processed PolSAR channels and the stacked 13 bands (six from multispectral and seven from PolSAR) are used as input data for further analysis and study. It is clear from Figure 5.1 or Figure 5.2 that PolSAR and MS satellite image modalities complement each other. For example, built-up areas and forests in the Roorkee study area exhibit similar backscattering response (light green colour in PolSAR image, Figure 5.1(a)) whereas both the land covers are clearly separable in the MS image (refer Figure 5.1(b)).107 Similarly, built-up areas are not easily separable from bare soil in MS image whereas the two land covers are easily separable in a PolSAR image.5.3.3. Ground truthFive land cover (LC) classes are considered namely bare soil (BS), tall vegetation (TV), short vegetation (SV), built-up, and water. Ground truth class data points are measured directly on the terrain based on visual inspection. Ground truth class data points collected for study in chapter 4 (refer section 4.2.3) are utilized here as well since the class labels are same and the study areas are also same. Details of the ground truth collected for both study areas are shown in Table 5.1. Also, samples images of land covers collected as ground truth are shown in Figure 3.3.

Figure 5 . 1

 51 Figure 5.1 Datasets for Roorkee study area: (a) False colour composite (red = HH, green = HV, and blue = VV) of ALOS PALSAR-2 PolSAR data, and (b) False colour composite (red = NIR, green = Red, and blue = Green) of Lansat-8 multispectral data.

Figure 5 . 2

 52 Figure 5.2 Datasets for Haridwar study area: (a) False colour composite (red = HH, green = HV, and blue = VV) of ALOS PALSAR-2 PolSAR data, and (b) False colour composite (red = NIR, green = Red, and blue = Green) of Lansat-8 multispectral data.

Figure 5 . 3

 53 Figure 5.3 Flowchart of the experiment setup and CNN based land cover classification methodologies proposed with PolSAR-MS data

from 4 to 8

 8 principle components are considered as features during classification. PolSAR-MS data used in the present study only have 13 spectral bands and do not require feature reduction.In[START_REF] Hu | Deep convolutional neural networks for hyperspectral image classification[END_REF], authors used different 1D-CNNs for two popular hyperspectral satellite image data (the Salimas Valley and the Indian Pines dataset) classification. In[START_REF] Mei | Integrating spectral and spatial information into deep convolutional Neural Networks for hyperspectral classification[END_REF], authors used 1D-CNN for hyperspectral satellite image data classification. All these references report that 1D-CNNs are good feature extractors for high-dimensional hyperspectral satellite image data. Most of these studies used some form of feature reduction technique before feeding data to the 1D-CNN in order to reduce the computational complexity of the model. The reduced number of bands are on the same order as the number of bands present in the PolSAR-MS data considered here in this study. This argument justifies the direct use of 1D-CNNs for PolSAR-MS satellite image data.Two variants of 1D CNNs are considered here. In the first 1D CNN model i.e. 1D-CNN-v1, the kernel size of the filter in the first convolutional layer i.e. C1 layer is equal to the spectral depth of PolSAR-MS data i.e. 13. One dimensional spectral convolution is shown in Figure5.4(b). In the second 1D CNN model, i.e. 1D-CNN-v2, the kernel size is less than the spectral depth of PolSAR-MS data. One dimensional local spectral convolution is visualized in Figure5.4(c). Models similar to 1D-CNN-v2 has been successfully tested with hyperspectral data[START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF],

2 .

 2 Apart from kernel size, the rest of the parameters are similar for both models. Rectified linear unit (ReLU) and Softmax activation functions are used in the intermediate and output layers respectively. A dropout strategy is considered for regularization and dropout fraction values of 0.2 for C1 and 0.1 for C2 and FC layer are set. Pooling is not considered because it hinders the full usability of the spectral variation. The loss function used in both models is the categorical cross entropy function. It optimizes the classification accuracy based on probabilities of each class during classification.The learning rate and the number of epochs are set to 0.01 and 200 respectively.

Figure 5 . 4

 54 Figure 5.4 Different convolution strategies: (a) Data cuboid with M by N pixels and P bands, (b) Convolution with 1 × 1 × 𝑃 convolutional filter where 1 × 1 represents to the spatial extent and P represents total number of bands in the data. Each filter convolves with all the bands as all times. This strategy is employed in 1D-CNNs during per pixel classification of satellite images. (c) Convolution with 1 × 1 × 3 convolutional filters where 1 × 1 represents the spatial extent and 3 represents the number of adjacent bands to be convolved simultaneously. One filter can convolve with three adjacent bands only. This strategy is also employed in 1D-CNNs. (d) Convolution with 1 × 1 × 3 convolutional filters where 1 × 1 represents to the spatial extent and 3 represents number of arbitrarily selected bands to be involved. This strategy is employed in the proposed Perm-LS-CNN framework.

  Spectral CNN (Perm-LS-CNN) and Permuted Local Spectral-Spatial CNN (Perm-LSS-CNN)In 1D-CNN-v2, a local convolution is performed over the spectral dimension. The convolution filter size in the spectral dimension i.e. 4 is less than the total number of input bands i.e.[START_REF] Avtar | Characterization of forests and deforestation in Cambodia using ALOS/PALSAR observation[END_REF]. This local spectral convolution improves the classification performance as discussed in[START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF]. A pictorial representation of local spectral convolution is depicted in Figure5.4(c). For example, a first filter (in yellow) convolves with bands 1, 2, and 3 whereas a second filter (in blue) convolves with bands 4, 5, and 6. However, an issue arises in the process of local spectral convolution with high dimensional satellite image data such as hyperspectral data or multisensor data e.g. PolSAR-MS data. While performing local spectral convolution, various combinations of spectral bands are not convoluted together and therefore, under-utilizing the full potential of the spectral information present in the data. For example, it is evident from Figure5.4(c), that the second spectral band and the tenth spectral band cannot not get convoluted together using 1D-CNN-v2 and hence the spectral information is under-utilized. The issue needs more attention for multisensor data processing with CNNs. For PolSAR-MS data, some polarimetric channels might not convolute together with some multispectral bands. For example, out of the[START_REF] Avtar | Characterization of forests and deforestation in Cambodia using ALOS/PALSAR observation[END_REF] bands of the PALSAR-2-Landsat-8 dataset considered in the present study, only 4 consecutive bands get convoluted together with a convolutional kernel window of 4. Moreover, stacking PolSAR and MS data in a common feature set induces an arbitrary ordering and irrelevant band neighbourhood relationship which might impact the performance of the local spectral convolution strategy employed in 1D-CNN-v2. Different band ordering might result in different classification results. MS neighbour bands may be related by some wavelength ordering relation, but the boundary between PolSAR and MS bands is irrelevant. Therefore, it is important to perform convolutions in a manner which compensates the ordering of PolSAR-MS bands. To provide a solution which compensates for the under-utilization of the spectral information due to PolSAR-MS band ordering, the present study proposes an ensemble approach which attempts to improves the utilization of spectral information. To present a proof of concept, this study introduces a permuted local spectral CNN model namely Perm-LS-CNN which is based on the 1D-CNN-v2 discussed in section 5.4.1. Perm-LS-CNN performs local spectral convolutions on permuted sets of the PolSAR-MS data. The permuted sets of the PolSAR-MS data provide increased combinations of PolSAR-MS bands that are missing from using only the original set. The idea of local spectral convolution with permuted PolSAR-MS bands can be realized in Figure 5.4(d). For example, first filter (in yellow) convolves with bands 1, 3, and 7 whereas second filter (in blue) convolves with bands 2, 5, and 6. The proposed Perm-LS-CNN model configurations are provided in Table 5.3. A permutation layer before the first convolution layer is added to provide permuted sets of the original PolSAR-MS data. Although the optimal number of permuted sets required for full utilization of the spectral information is not derived in this study, 20 permuted sets are used as a proof of concept. Other configuration and hyperparameters are similar to 1D-CNN-v2 and are listed in Table 5.3. Further, based on the Perm-LS-CNN, another novel classification framework for land cover classification using PolSAR-MS data is introduced here. This framework extends the idea of Perm-LS-CNN to spatial context. The proposed permuted local spectral-spatial CNN or the Perm-LSS-CNN utilizes the spatial feature extraction capability of CNNs, along with the idea of permuted local spectral convolution. The added spatial context helps CNNs to extract more complex and abstract features, which improves classification results. The idea of permuted spectral-spatial convolution is presented in Figure 5.5(d). For example, a first filter (in yellow) convolves with bands 1, 2, and 4 whereas a second filter (in blue) convolves with bands 5, 7, and 10. The proposed Perm-LSS-CNN model configurations are provided in Table 5.3. Similar to Perm-LS-CNN, 20 permuted sets are created in the Perm-LSS-CNN model. All hyperparameters are set as for 3D-CNN-v1 and are listed in Table 5.3. The advantage and significance of the permuted local spectral-spatial convolutions used in Perm-LSS-CNN over the 2D-CNN-v1 and 3D-CNN-v1 is evaluated based on a class separability measure [175]. The separability measure or separability index (SI) is computed using equation (3.4). The class separabilities are measured on the ground truth class samples of Roorkee study area (refer Table 5.1) and at the first convolutional layer i.e. C1, of the 2D-CNN-v1, 3D-CNN-v1 and the proposed Perm-LSS-CNN models. For each model and class pair, the highest SI value achieved using equation (3.4) is given in Table 5.4. For example, in the BSwater class pair case, the highest SI value of; 1.82 for 2D-CNN-v1, 1.99 for 3D-CNN-v1, and 2.87 for the Perm-LSS-CNN are obtained. In other example, in the tall vegetation (denoted as TV), short vegetation (denoted as SV) class pair case, the highest SI value of; 1.46 for 2D-CNN-v1, 1.56 for 3D-CNN-v1, and 2.77 for the Perm-LSS-CNN are obtained. Overall, it is observed from Table 5.4 that the Perm-LSS-CNN model provides a feature space where the classes are more significantly separable than what is achieved by the 2D-CNN-v1 and the 3D-CNN-v1 models. The experiment establishes the advantage of permuted local convolution in the spectral dimension along with the spatial context.

Figure 5 .

 5 Figure 5.7 and Table 5.6 for the Haridwar data set.Figure 5.6(a) and Figure 5.7(a) show Google Earth images for the Roorkee and the Haridwar data sets respectively for reference purpose.

Figure 5 . 8 ,

 58 Figure 5.8(b, and c), water flow in the river is not consistent which is undesirable, whereas, in Figure 5.8(d, e and f), water flow is consistent.Figure 5.8(g) shows an agriculture field with a

Figure 5 .

 5 Figure 5.8(d, e and f), water flow is consistent.Figure 5.8(g) shows an agriculture field with a building located in the upper right corner. In Figure 5.8(h, and i), the building is classified as SV by RF and RBF-SVM, whereas it is correctly classified as built-up by 2D-CNN-v1, 3D-CNN-v1, and the Perm-LSS-CNN in Figure 5.8(j, k and l).Figure 5.8(m) shows a dried up river, a

Figure 5 .

 5 8(m) shows a dried up river, a canal, and a bridge over the canal. The river is partially classified as SV in Figure5.8(n, and o) whereas, it is correctly classified as BS in Figure5.8(p, q and r). The bridge is well off the ground and oriented perpendicular to the orbit of the SAR sensor, hence exhibits double bounce scattering (only from one side). It is consistently classified as a built-up structure in Figure5.8(p and r). The same is wrongly classified in Figure5.8(n, o and q). Observing the above two examples suggest that RF and RBF-SVM methods are unable to capture subtle changes in land covers. Also, 3D-CNN-v1 classified the bridge as Bare Soil, missing the double bounce nature inherited in the PolSAR features.

Figure 5 .

 5 8(s) show a peculiar TV pattern within a golf ground.The structural and spectral identity are better preserved by Perm-LSS-CNN (refer Figure5.8(x))

Figure 5 .

 5 8(y) shows a tree line. Again, the structural and spectral identity is most significantly preserved by Perm-LSS-CNN as shown in in Figure5.8(d1)).Overall observation of Figure5.8 suggest that the proposed Perm-LSS-CNN method yields better classification performance both in terms of classification accuracy and generalization ability by capturing the structural and spectral identity of land covers. In fact, classification accuracies alone cannot be used as reliable indicator for comparing classification methods because of overfitting. For example, RF and RBF-SVM provides good classification accuracies (refer Table5.5 and Table5.6) but lacks generalization performance (refer Figure5.8). The generalization accuracy on well-known ground truth is the real test. The reason for the inferior generalization ability of the 2D-CNN-v1 is that the 2D convolutional filter is convolving with all the spectral features, which reduces the possibility for generation of local spectrally significant features (refer Figure5.5(b)). The proposed Perm-LSS-CNN generates such combinations of spectral and spatial properties and hence shows improved generalization performance.

  8 indicate the improved classification performance of the proposed Perm-LSS-CNN over other classifiers considered for comparison. The discussion also indicates an increase in the generalization ability of the Perm-LSS-CNN because of the ensemble learning scenario.

Figure 5 . 6

 56 Figure 5.6 Classification image: (a) Google Earth image of the Roorkee study area, India. Classified images are shown for different classifiers: (b) 1D-CNN-v1, (c) 1D-CNN-v2, (d) Perm-LS-CNN, (e) 2D-CNN-v1, (f) 3D-CNN-v1, (g) Perm-LSS-CNN, (h) RBF-SVM, and (i) RF. Colour scheme; red = built-up, light green = short vegetation, green = tall vegetation, blue = Water, and yellow = bare soil.

Figure 5 . 7

 57 Figure 5.7 Classification image: (a) Google Earth image of the Haridwar study area, India. Classified images are shown for different classifiers: (b) 1D-CNN-v1, (c) 1D-CNN-v2, (d) Perm-LS-CNN, (e) 2D-CNN-v1, (f) 3D-CNN-v1, (g) Perm-LSS-CNN, (h) RBF-SVM, and (i) RF. Colour scheme; red = built-up, light green = short vegetation, green = tall vegetation, blue = water, and yellow = bare soil.

Figure 5 . 8

 58 Figure 5.8 Results from different classification techniques applied on the Roorkee data set. The areas include data points collected as ground truth.

  analysed and evaluated for the task of MSTS images based crop classification. 1D, 2D, and 3D CNNs, RNNs, and combinations of these are considered for analysis. Then, based on the critical analysis of the working and performance of these conventional models over MSTS images based crop classification, few novel models are proposed and developed in sub task 2.The novel models involve permuted spectral and/or spatial convolutions and/or bidirectional recurrent layers. The Bidirectional RNNs capture the temporal profile and the permuted spectral and/or spatial CNNs capture the spectral and/or spatial profile of a particular crop. Bidirectional strategy in RNNs enables the influence of the past, present, and future inputs on the current output. Bidirectional RNNs are beneficial in time series images based crop classification because crop classification can utilize the bidirectional process i.e. classification result can be observed from any time node. Also, the permuted spectral band stacking strategy developed in chapter 5 is also used in this study and presumed to improve the significance of the spectral information present in multisensor satellite time series images. In particular, a synergistic utilization of permuted spectral-spatial CNNs and Bidirectional RNNs helps in better exploitation of the multisensor satellite time series images.

  y b are coefficients that are shared temporally, 1g is an activation function (usually a hyperbolic tangent), and 2 g is a softmax activation function[START_REF] Elman | Finding Structure in Time[END_REF].

Figure 6 . 1

 61 Figure 6.1 Typical architecture of a recurrent neural network [250].

Figure 6 . 2

 62 Figure 6.2 Signal flow graph in a simple recurrent cell [250].

f  with 1 u

 1  and does not utilizes the output gate 0 [START_REF] Cho | Learning phrase representations using RNN encoder-decoder for statistical machine translation[END_REF].

Figure 6 . 3 A

 63 Figure 6.3 A LSTM recurrent cell [250].

Figure 6 . 4 A

 64 Figure 6.4 A GRU recurrent cell [250].

Figure 6 . 5

 65 Figure 6.5 RNN architectures used popularly in satellite time series image based crop classification: (a) Many to one, and (b) Many to many RNN architectures [250].

Figure 6 .

 6 [START_REF] Townsend | Modeling floodplain inundation using an integrated GIS with radar and optical remote sensing[END_REF] shows the Google Earth image of the study area. The region is a land parcel which is managed by local farmers. Seasonal crops and sugarcane are grown in this region. The study area is approximately a 6km by 7km land parcel, covering 42km 2 of area. The region also has forests, grasslands, deciduous tree plantations along with various crops in the study area. This makes the region more realistic environment and mixed class scenario for algorithm testing. The presence of multiple land covers will evaluate the generalization ability of the classifiers. Areas marked in the image(1, 2, 3, and 4) are subsets of study area used for qualitative and visual evaluation of the classification performance of classifiers developed in the study.

Figure 6 . 6

 66 Figure 6.6 Google Earth image of the study area (GE Image Copyright 2018). Subset areas selected for ground truth via visual inspection. 1: GTI-1, 2: GTI-2, 3: GTI-3, and 4: GTI-4(highlighted in yellow boxes).

Figure 6 . 7

 67 Figure 6.7 False colour composite from the Sentinel-2 time series dataset. Colour scheme; red: Band 4 (November 21, 2017), green: Band 4 (February 9, 2018), blue: Band 4 (March 31, 2018). Image subset areas selected for ground truth via visual inspection (highlighted in black boxes). 1 = GTI-1, 2 = GTI-2, 3 = GTI-3, and 4 = GTI-4.

  used in standalone mode to extract temporal features directly from satellite time series images at pixel level. The CNNs, RNNs, and the FC modules can be used in numerous other fashions to achieve classification. The next few sections present few such possibilities and configurations for multisensor satellite time series image based crop classification.Although the current study used multisensor satellite time series (MSTS) images, however, the architectures discussed above are independent of thee notion of multisensor. This means that all or any other variation of the aforementioned DNN architectures will work with multisensor satellite time series images in the similar fashion.

Figure 6 . 9 A 6 . 5 . 6 . 5 . 1 .

 6965651 Figure 6.9 A generic framework for DNN models for satellite time series images based crop classification.

  10. ReLU, hyperbolic tangent (Tanh) and softmax activation functions are used in the convolutional & FC, LSTM recurrent cells, and output layer respectively. A dropout strategy is considered for regularization and dropout fraction values of 0.2, 0.1 and 0.1 for (C1, C2, C3), R1 and FC are set respectively. The loss function used is the categorical cross entropy function. It optimizes the classification accuracy based on probabilities of each class during classification. The learning rate and the number of epochs are set to 0.01 and 200 respectively.

C1, C2, and C3 layers are set to 3 3 7  , 1 1 21  , and 1 1 63 

 72163 respectively. The kernel sizes are set based on experiments. Then, a conventional RNN model is considered identical to the RNN model used in Perm-1D-CRNN-v1 and Perm-2D-CRNN-v1. The architecture of the Perm-3D-CRNN-v1 is listed in Table 6.10. The hyper-parameter settings are similar to the Perm-3D-CRNN-v1 model and are listed in Table 6.10. the second component is GTI from February, 2018. It can be observed from GTI of November, 2017 that no major crop is growing in the area and the green patches and tree lines are popular plantations (highlighted in yellow box). Whereas, in the GTI of February, 2018, wheat crop is grown in the area (sample highlighted in green box). Visual interpretation of the corresponding patches from the classified images of the compared models indicate that all the models have correctly classified the crop as wheat. However, it is critical to observe that though Perm-1D-CNN-v1 or Perm-3D-CNN-v1 does not perform temporal convolution, they able to correctly classify the image patch. This indicates to the ability of CNNs to focus on any particular time stamp based on training labels and provide classification. Even though, Perm-1D-CNN-v1 show highest classification accuracy, it inability to not consider temporal pattern makes it less generalized model in MSTS image based crop classification.

Figure 6 .

 6 Figure 6.12 shows a ground truth image (GTI-2) which is patch 2 from the original study area. The GTI-2 also has two components; the first component is a GTI from November 2017 and the second component is GTI from February, 2018. It can be observed from GTI of November, 2017 that major crop growing in the area is sugarcane (highlighted in yellow box) and some fallow lands are present. Whereas, in the GTI of February, 2018 that wheat crop is grown in these fallow lands (highlighted in green box) and most of the sugarcane is harvested which in turn leaves new fallow lands. Visual interpretation of the corresponding patches from the classified images of the compared models reveal peculiar information.

Figure 6 .

 6 Figure 6.11 Visual interpretation and evaluation of classification performance of the compared classifiers for GTI-1.

  misclassified few sugarcane fields (highlighted in red box) as built-up due to dominance of HH backscatter response in the spectral dimension. Whereas, the Perm-RNN-v1 and Perm-3D-CRNN-v1 models have correctly classified the same sugarcane fields. This indicates that models such as Perm-RNN-v1 or Perm-3D-CRNN-v1 are able to capture the spectral attributes more efficiently because of the training of these models with ensemble of permuted spectral bands (averaging out the dominance of any particular spectral band). This novel approach of permuted spectral stacking improved the generalization ability of these models and complemented the temporal attribute of the crops.

Figure 6 .

 6 Figure 6.12 Visual interpretation and evaluation of classification performance of the compared classifiers for GTI-2.

Figure 6 .

 6 Figure 6.13 Visual interpretation and evaluation of classification performance of the compared classifiers for GTI-3.

Figure 6 .

 6 Figure 6.14 Visual interpretation and evaluation of classification performance of the compared classifiers for GTI-4.

Chapter 4 ,

 4 CNN and RNN models in MSTS images based crop classification. The analysis indicated to satisfactory performance of conventional CNN and RNN models in MSTS based crop classification. Also, preferences given to temporal and spatial characteristics of MSTS images by the conventional DNN models is observed. This led to underutilization of spectral characteristics of MSTS images and therefore resulted in less generalized crop classification models. To address this issue, novel CNN and RNN models are proposed and developed on the concept of local spectral convolutions on ensemble stack of permuted spectral bands of MSTS images. The local spectral convolutions in an ensembled spectral dimension improves the generalization ability of DNN models and also complements the spatial and temporal features. Overall, 12 CNN and RNN models are developed during the study namely 1D-CNN-v1, 3D-tall veg class (73.8%, and 89.1% respectively) indicate that the proposed approach, i.e. the optimal class boundary estimator based DT classifier, achieved better classification results in mixed LC class scenarios. In the third sub task i.e. sub task 3, development of a two dimensional CNN (2D-CNN) based novel model for LC classification is presented. The 2D-CNNs can directly exploit PSs to perform LC classification, and therefore, the approach reduces the efforts to compute PSCFs. Also, since 2D-CNNs are powerful image extractors, they are able to extract the subtle contextual patterns present in PSs of different LCs. In summary, a novel PSs-CNN LC classification model is realized. Classification results from this PSs-CNN model are compared with NED and NSCM based minimum distance classifiers which are popularly used with PSs for land cover classification. Comparison indicated a superior performance of the novel PSs-CNN model for land cover classification.Comparison between the two novel approaches indicates that the optimal class boundary estimator based DT classifier is showing an approx. 3% improved classification accuracy than PSs-CNN classifier. However, the PSs-CNN model is easier to implement, does not require an extra step (computing PSCFs), and is scalable.Overall, the study concludes that; i) PSs have shown significant potential as PolSAR features for land cover applications, ii) segregation of built-up class and tall vegetation class in PolSAR data can be improved by using optimal class boundary estimators with decision trees for classification, and iii) 2D-CNNs have shown potential as classifiers with PSs for LC classification in terms of ease of implementation, and scalability. presents an efficient approach to map flood affected villages using PolSAR image. The approach included three stages i.e. -first, built-up area extraction, -second, village detection, and -third, flood affected village mapping.In the first stage, the approach utilized PolSAR polarization signatures (PSs) and CNNs to extract built-up areas from a PolSAR image. A novel PSs-CNN classifier is developed. The PSs-CNN model provided improved built-up area extraction performance in contrast to the NED and the NSCM classifiers. In the second stage, the approach introduced and implemented a novel village detection methodology. The methodology uses a detection filter to detect whether a particular object in the built-up area image (obtained in stage 1) is a village or not. The detection filter uses apriori knowledge and image morphology to detect villages in the built-up area image.The detection methodology also significantly removes false alarms and misclassifications. In the third and final stage, flood affected villages are identified and non-flooded villages are masked out using another image morphology based methodology. This methodology computed overlap ratios between objects (successfully detected as villages) and flooded area and utilized it for identification of flood affected villages.Quantitative and qualitative assessment of the performance of the novel approach over two different study areas is carried out. Different artificial flood scenarios are created to evaluate the performance of developed approach. The developed approach is evaluated under these flood scenarios and flood affected villages are identified and mapped. Number of villages successfully detected and false alarms are computed with the help of ground truth information (actual number of villages or towns in the study regions are known). Results indicate to high detection accuracy (above 90%), low false alarms and true negatives. The approach is scalable and potentially satisfactory for real time usage. Chapter 5 presents a critical analysis of the potential and performance of the one, two, and three dimensional CNNs, where dimensionality refers to the convolved features: spectral, spatial or both. These CNNs are applied to land cover classification with multisensor (PolSAR-MS) satellite image data. Two novel models are introduced, where local filters in either spectral domain (Perm-LS-CNN) or spatial-spectral domains (Perm-LSS-CNN) are applied after a permutation layer. The permutation layer generates an ensemble of band combinations, which are drawn upon by the CNN in order to generate powerful features for land cover classification. Classification results demonstrate the improved generalization ability of these models compared to both alternative CNN architectures and other machine learning models (SVM and random forests). Perm-LSS-CNN also fully uses both PolSAR and multispectral information, improving results compared to other popular methods for PolSAR-MS data based land cover classification. A separability measure is used for the model significance evaluation. Separability analysis states that the Perm-LSS-CNN transforms the input data space into a feature space where separability among classes increases. This increase in separability is achieved due to the inclusion of permuted local spectral convolution along with the local spatial convolution which is not available in the 1D-CNN-v1, 2D-CNN-v1, or 3D-CNN-v1 CNN models. A fine analysis on ground truth data provides an extra qualitative assessment which complements the quantitative classification results. These demonstrate that the proposed Perm-LSS-CNN model is able to generalize much better while keeping high classification accuracies. Chapter 6 presents a comprehensive study and analysis of the performance of conventional CNN and RNN models in MSTS images based crop classification. The analysis indicated to satisfactory performance of conventional CNN and RNN models in MSTS based crop classification. Also, preferences given to temporal and spatial characteristics of MSTS images by the conventional DNN models is observed. This led to underutilization of spectral characteristics of MSTS images and therefore resulted in less generalized crop classification models. To address this issue, novel CNN and RNN models are proposed and developed on the concept of local spectral convolutions on ensemble stack of permuted spectral bands of MSTS images. The local spectral convolutions in an ensembled spectral dimension improves the generalization ability of DNN models and also complements the spatial and temporal features. Overall, 12 CNN and RNN models are developed during the study namely 1D-CNN-v1, 3D-CNN-v1, RNN-v1, 1D-CRNN-v1, 2D-CRNN-v1, 3D-CRNN-v1, Perm-1D-CNN-v1, Perm-3D-CNN-v1, Perm-RNN-v1, Perm-1D-CRNN-v1, Perm-2D-CRNN-v1, and Perm-3D-CRNN-v1. These models are trained on MSTS satellite images of areas having crop fields in challenging landscape. Wheat and sugarcane are the major crops practices in the area considered here for study. The area, apart from croplands, also contain, forests, built-up, grasslands, and deciduous plantations. The study area is a good representation of real world crop classification scenarios particularly in India. The performance of these models in this study area presents be good indicator of their generalization ability. Quantitative analysis of the classification results from these models indicate to superior performance of the Perm-1D-CNN-v1 model with classification accuracy of 99.7%. However, the Perm-3D-CRNN-v1, which provides 96.00% classification accuracy, indicate to have higher generalization ability during qualitative and visual assessments. The spectral and spatial attributes of land covers (e.g. built-up or popular plantations) is well captured by the Perm-3D-CRNN-v1 whereas the Perm-1D-CNN-v1 appears to be overfitting. Therefore, the Perm-3D-CRNN-v1 is the best model among the classifiers considered.

  5.3.1. Study area ............................................................................................................................................ 5.3.2. Dataset ................................................................................................................................................ 5.3.3. Ground truth ........................................................................................................................................ 5.4. 1D, 2D, AND 3D CNNS FOR LAND COVER CLASSIFICATION WITH POLSAR-MS DATA .

  1. 1D-CNN model for MSTS images based crop classification .............................................................. 6.5.2. 2D-CNN model for MSTS images based crop classification .............................................................. 6.5.3. 3D-CNN model for MSTS images based crop classification .............................................................. 6.5.4. RNN model for MSTS images based crop classification ..................................................................... 6.5.5. CNN-RNN models for MSTS images based crop classification .......................................................... 6.6. THE CONCEPT OF PERMUTED LOCAL SPECTRAL CONVOLUTIONS AND ITS SYNERGISTIC USE WITH RNNS FOR MSTS IMAGES BASED CROP CLASSIFICATION ..................................... 6.7. SUB TASK 2: NOVEL CNN AND RNN MODELS WITH PERMUTED LOCAL SPECTRAL CONVOLUTIONS FOR MSTS IMAGE BASED CROP CLASSIFICATION ........................................ 6.7.1. 1D-CNN model with permuted local spectral convolution for MSTS Images based crop classification ...................................................................................................................................................................... 6.7.2. 2D-CNN model with permuted input for MSTS images based crop classification ............................. 6.7.3. 3D-CNN model with permuted local spectral convolution for MSTS images based crop classification ...................................................................................................................................................................... 6.7.4. RNN model with permuted input for MSTS images based crop classification ....................................

6.7.5. CNN-RNN models with permuted local spectral convolution for MSTS images based crop classification .................................................................................................................................................

Table 1 .

 1 1 Broad categorization of the electromagnetic spectrum in terms of spectral bands and their spectral bandwidths [21]............................................................................................... 6

Table 1 .

 1 2 Landsat-8 (multispectral) OLI and TIRS band specifications [22]. ........................... 10

Table 1 .

 1 3 Sentinel-2 band specifications [23]. ........................................................................... 10

Table 1 .

 1 4 MODIS sensor band specifications and primary usage [25]. ..................................... 11

Table 1 .

 1 5 Hyperion sensor specifications [26]. .......................................................................... 12

Table 1 .

 1 6 Polarimetric parameters. ............................................................................................. 15

Table 1 .

 1 7 Designation of microwave bands [30]. ....................................................................... 18

Table 1 .

 1

8 Brief list of few past, current and upcoming spaceborne SAR sensors with their properties [30]. .................................................................................................................... 18

Table 3.1 Summary of ground truth sample points collected for study. ..................................... 56 Table 3.2 Separability index (SI) values for various class pairs for all features. ...................... 63 Table 3.3 Separability index (SI) values for various class pairs for popular features. .............. 63 Table 3.4 Classification performance of PS based and decomposition based PolSAR features.

Classes: built-up (BU), water (W), bare soil (BS), short vegetation (SV), and tall vegetation (TV). ................................................................................................................................... 64 Table 3.5 List of selected feature corresponding to land cover classes. .................................... 65

Table 3.6 Updated list of selected feature corresponding to land cover classes. ........................ 66 Table 3.7 Polarization signature correlation feature statistics. .................................................. 66 Table 3.8 Empirical relations between individual overall accuracies (IOA) and i n values. ...... 68 Table 3.9 Optimal i n values obtained using the genetic algorithm. ........................................... 69

Table 3 .

 3

Table 6 .

 6

7 Architecture and hyperparameter settings of CNN-RNN based models for MSTS images based crop classification. ...................................................................................... 141 xxxv

Table 6 .

 6 8 Architecture and hyperparameter settings of CNN models with permuted spectral convolution for MSTS images based crop classification. ................................................. 145

Table 6 .

 6 9 Architecture and hyperparameter settings of RNN model with permuted input for MSTS images based crop classification. .......................................................................... 146

Table 6 .

 6 10 Architecture and hyperparameter settings of CNN-RNN with permuted spectral convolution based models for MSTS images based crop classification. .......................... 149

Table 6 .

 6 11 Summary of the classification performance of the compared classifiers. .............. 150

	xxxvi

Table 1 .

 1 2 Landsat-8 (multispectral) OLI and TIRS band specifications[START_REF] Usgs | What are the band designations for the Landsat satellites?[END_REF].

	infrared and short-wave infrared sensors to form images of the earth's surface by detecting the
	solar radiation reflected or scattered from objects on the earth's surface. Different earth surface
	objects reflect and absorb differently at different wavelengths. Thus, these objects can be
	identified, differentiated, or detected based on their spectral reflectance signatures recorded by
	satellite optical sensors. Optical imaging systems are majorly classified into four types namely
	panchromatic, multispectral, superspectral, and hyperspectral depending on the number of
	spectral bands used in the imaging process.

Table 1 .

 1 4 MODIS sensor band specifications and primary usage[START_REF] Laads Daac | Moderate Resolution Imaging Spectroradiometer (MODIS) -LAADS DAAC[END_REF].

	Band	Bandwidth 1	Radiance 2 Spectral	SNR 3 Required	Potential Applications
		620-670	21.8	128	Land/Cloud/Aerosols Boundaries
		841-876	24.7	201
		459-479	35.3	243	Land/Cloud/Aerosols Properties
		545-565	29.0	228
		1230-1250	5.4	74
		1628-1652	7.3	275
		2105-2155	1.0	110
		405-420	44.9	880	Ocean Colour/ Phytoplankton/
		438-448	41.9	838	Biogeochemistry
		483-493	32.1	802
		526-536	27.9	754
		546-556	21.0	750
		662-672	9.5	910
		673-683	8.7	1087
		743-753	10.2	586
		862-877	6.2	516
		890-920	10.0	167	Atmospheric Water Vapour
		931-941	3.6	57
		915-965	15.0	250
		3.660-3.840	0.45 (300 K)	0.05	Surface/ Cloud Temperature
		3.929-3.989	2.83 (335 K)	2.00
		3.929-3.989	0.67 (300 K)	0.07
		4.020-4.080	0.79 (300 K)	0.07
		4.433-4.498	0.17 (250 K)	0.25	Atmospheric Temperature
		4.482-4.549	0.59 (275 K)	0.25
		1.360-1.390	6.00	150	Cirrus Cloud Water vapour
		6.535-6.895	1.16 (240 K)	0.25
		7.175-7.475	2.18 (250 K)	0.25
		8.400-8.700	9.58 (300 K)	0.05	Cloud Properties
		9.580-9.880	3.69 (250 K)	0.25	Ozone
		10.780-11.280	9.55 (300 K)	0.05	Surface/ Cloud Temperature
		11.770-12.270	8.94 (300 K)	0.05
		13.185-13.485	4.52 (260 K)	0.25	Cloud Top Altitude
		13.485-13.785	3.76 (250 K)	0.25
		13.785-14.085	3.11 (240 K)	0.25
		14.085-14.385	2.08 (220 K)	0.35
	Bands 1 to 19 are in nm; Bands 20 to 36 are in μm
	Spectral Radiance values are (W/m2 -μm-sr)
	SNR = Signal-to -Noise Ratio
	NE (Delta)T = Noise-equivalent temperature difference
	Note: Performance goal is 30-40 % better than required

Table 1 .

 1 [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF] Hyperion sensor specifications[START_REF] Liew | Interpreting Optical Remote Sensing Images[END_REF].

	Specification

Table 1 .

 1 [START_REF] Townsend | Modeling floodplain inundation using an integrated GIS with radar and optical remote sensing[END_REF] Polarimetric parameters. energy in one of the microwave frequency bands shown in Table1.7. Roughly speaking, radar systems use frequencies from 1 to 90 GHz. This frequency range is subdivided into multiple frequency bands, as shown in the first column of Table1.7. These bands were defined initially according to the different equipment(s) needed to generate and detect these particular frequencies. However, now they are understood as the equivalent of colours in the visual range. Unfortunately, the latter-based naming scheme (Ka-band to P-band) was used during World war II and never deemed necessary to be modified since then. This nomenclature sometimes leads to confusion among new users of SAR. Not all of the microwave bands shown in Table1.7 used are for SAR imaging. While some experimental airborne Ka-and Ku-band SAR systems exist, civilian spaceborne sensors have been using the frequency bands ranging from P-to X-band.

	Parameter	Application (Discrimination)
	HH Polarization	Double bounce scattering
	HV Polarization	Volume scattering
	VH Polarization	Volume scattering
	VV Polarization	Surface scattering

The operating wavelength of a SAR sensor is linked intrinsically to its transmitted signal's penetration capabilities. It means that longer wavelength signals (e.g., signals at L-and P-band) penetrate deeper into land covers and earth surface compared to shorter wavelength signals i.e.

Table 1 .

 1 7 Designation of microwave bands[START_REF] Meyer | Spaceborne Synthetic Aperture Radar: Principles, data access, and basic processing techniques[END_REF].

	Band	Frequency (GHz)	Wavelength (cm)

Little but increasing use for SAR-based earth observation; agriculture monitoring (NISAR will carry an S-band channel; expands C-band applications to higher vegetation density) L 1-2 30-15 Medium resolution SAR (Geophysical monitoring; biomass and vegetation mapping; high penetration; InSAR) P 0.3-1 100-30 Biomass. First P-band spaceborne SAR will be launched ~2020; vegetation mapping and assessment. Experimental SAR.

Table

1

.8 Brief list of few past, current and upcoming spaceborne SAR sensors with their properties

[START_REF] Meyer | Spaceborne Synthetic Aperture Radar: Principles, data access, and basic processing techniques[END_REF]

.

. Satellite Images: Summary and Context to the Thesis It

  is evident from section 1.1 that satellite images are critical to earth monitoring and are the most techno-economical means of doing so. Following are key takeaways from satellite images based earth monitoring.Satellite images provide wide coverage, timely response, reliable and consistent information, and high quality[START_REF] Jain | Real-time quality analysis of digital remote sensing satellite signals[END_REF] earth observation data. Satellite image are available at different characteristics i.e. spatial, spectral, and temporal resolutions. Also, satellite images are available in various modalities such as surface reflectance (multispectral, or hyperspectral), backscatter

(active or passive microwave), brightness-temperature (microwave, thermal), polarization (linear, or circular), and others. These different satellite image characteristics and modalities are useful in standalone or synergistic fashion for various applications. This thesis focuses on efficiently utilizing these characteristics and modalities, in standalone or synergistic fashion, for different applications.

On top of this, satellite images are available in large amounts. Specifically, in the last decade, there has been a surge in the availability of high quality (spatial, spectral, and temporal), open-source or economical earth observation satellite images. Today, more than 600 satellites provide earth observation data, mostly in the form of images. Although there are various satellite image modalities available today, but PolSAR (backscatter) and multispectral (surface reflectance) are the most techno-economical, widely used satellite image modalities. This thesis focuses on studies based on these two image modalities for different applications.

Popular satellite image applications include land cover classification and mapping, agriculture crop monitoring, disaster monitoring, environment health and earth surface dynamics monitoring, and mineral exploration. These applications vary from being performance-critical such as land cover classification to being time-critical such as disaster management. Therefore, different applications require focus on one or more characteristics (spatial, spectral or temporal)

Research Gaps in Deep Learning based Satellite Image Processing A

  These studies reported good classification results and promoted further studies. A detailed review on the recent trends and applications of CNNs and RNNs based satellite image processing is provided in chapter 2 here. The review involves major satellite image applications developed on uni-and multi-modal, single-and multi-sensor, single time and further exploration of the potential of these models, especially CNNs is motivating. Studies such as, improving the generalization performance of CNN based land cover classification models by focusing on the spectral attributes of satellite images or, exploring synergistic use of bi-directional RNNs with spectrally focussed CNNs for agriculture crop monitoring applications may be beneficial. brief literature review is carried out in the field of deep learning based satellite image processing. Chapter 2 provides brief review, description and discussions on various novel deep learning based satellite image processing techniques developed for various applications.

stamp and time series satellite images with the help of deep learning and DNNs. The review indicates to the significant success and advantage of deep learning methods, especially CNNs and RNNs in the field of satellite image processing. However, the CNN and RNNs deep neural network models are still young, promising, and growing techniques in the satellite image processing community 1.5. Although, these techniques have proved their potential and have gained popularity over the years, there are areas that require special attention. There are areas of research in deep learning and DNNs based satellite image processing where improvements are worth mentioning. Few of these are:

 Although numerous DNN models have been developed so far for satellite image analysis and processing, we believe there is more to explore.  Improving the performance of current DNN models or development of novel DNN models in mixed land cover scenarios is still a challenge.  Spectral

information in satellite images is under exploited with current DNN techniques

: Earth observation satellite images contain relatively higher spectral information in comparison to normal camera images. This means unlike the normal camera images which have three spectral bands i.e. red, green, and blue, spectral bands in these images may vary from tens to hundreds. These characteristics of earth observation images makes them challenging to handle and require special attention. Current DNN techniques, especially CNNs and RNNs mostly focus on spatial and temporal characteristics of satellite images during satellite image processing for different applications. Focussing spectral information along with spatial/temporal information might be beneficial.  Need

more attention on development of more generalized and robust DNN based satellite image processing techniques:

  It is intuitive to say that similar to conventional neural networks, DNNs also suffer from overfitting and low generalizability. Special focus is required to improve the generalization ability of DNNs in mixed land cover scenarios. Ensemble training is key to achieve good generalization in DNN models. However, literature reveals lack in ensemble training based DNN models for satellite image processing. Also, visual inspection based evaluation of generalization ability of DNN models needs to be promoted.  Multisensor satellite images, especially optical and PolSAR complement each other in various land cover applications. There is a lack of CNN and RNNs based models which can exploit multisensor, especially PolSAR and optical images directly at various temporal, spectral and spatial resolutions for land cover applications. Special focus is required on the complementary spectral information present in high dimensional multisensor satellite images.  Although, processing time series satellite imagery with CNNs and RNNs for crucial applications such as agriculture crop classification is gaining popularity, the weightage on the exploitation of temporal, spatial, and spectral information via CNNs and RNNs is not well investigated and needs attention. Also, the element of multisensor makes it more interesting and motivating.  Since PolSAR and multispectral satellite images are the most abundant, economical, and popular modalities, special focus is required to develop CNN and RNN methodologies which can utilize these modalities for different land cover applications.

  The experiment setup includes materials such as acquiring datasets, selecting study areas, and collecting ground truth information and set of hypothetical methods which are suitable candidates for providing a solution are investigated. Since the present research is focussed on development of DNN based models for satellite image (especially PolSAR and multispectral) based

applications, a set of satellite images (PALSAR-2, Landsat-8, Sentinel-1, and Sentinel-2) are considered for study. Study areas which are easily accessible and potential for subject of interest are identified. The study areas considered are mostly in the Haridwar district of Uttarakhand, India. These study areas considered have mixed land cover scenario and are interesting to evaluate any novel techniques. Extensive ground truth is collected in the study areas over the period of research. Hypothetical techniques considered in the present research are CNNs and RNNs.

  as well for task 4. Further for task 4, synergistic exploitation of both the CNNs and RNNs is investigated and novel CNN-RNN models are proposed. The developed models are trained with ground truth information and under different scenarios.

general approach in this component of the research framework involves development of novel approach, algorithm, or model based on hypothetical methods and nature of data. In the present research, various CNN and RNN models are either investigated and newly developed for various applications. As per CNNs, one, two, and three dimensional convolution based novel models are proposed in task 1, 2, 3, and 4. Also, LSTM and GRU recurrent cell based RNN models are developed

RECENT TRENDS AND CHALLENGES IN DEEP LEARNING AND DNNS BASED SATELLITE IMAGE PROCESSING

  Chapter 6 presents a study in which an attempt is made; to analyse the conventional CNN and RNN models, and to develop novel CNN and RNN (in standalone as well as synergistic modes) models for agriculture crop classification with multisensor/multimodal satellite time series imagery. For this purpose, two sub tasks are set in this study. In the first sub task i.e. sub task 1, conventional CNN and RNN models are analysed and evaluated for the task of multisensor satellite time series (MSTS) images based crop classification. 1D, 2D, and 3D CNNs, RNNs, and combinations of these are considered for analysis. Then, based on the critical analysis of the working and performance of these conventional models over (MSTS) images based crop classification, 6 novel models are proposed and developed in sub task 2.Finally, chapter 7 concludes the different studies reported in the thesis. The conclusions drawn from each study, contributions made by the studies, and future scopes of the studies are also listed in this chapter.

	Chapter 2
	BRIEF LITERATURE REVIEW: Deep learning and deep neural networks have shown substantial potential as successful satellite
	image processing techniques. It has been established that DNNs and deep learning (DL) can be
	extended and generalized to satellite images from natural images [4]. In the recent years, many
	satellite image based applications such as land cover classification, crop monitoring, building

extraction, etc. have been proposed and developed with the help of DNNs and their popularity is still growing

[START_REF] Ma | Deep learning in remote sensing applications: A meta-analysis and review[END_REF]

,

[START_REF] Kamilaris | Deep learning in agriculture: A survey[END_REF]

.

The deep neural network models are high performance, scalable, automatable, and can learn directly from raw data

[START_REF] Lecun | Deep learning[END_REF]

. The deep learning models improve their performance with increase in data whereas other machine learning techniques, which depend on hand-crafted features for learning, are unable to learn beyond a certain amount of data. This means, more the training data, better the deep learning model performs. Another advantage of deep learning is its ability to learn directly from data and independency from hand-crafted features. Traditional machine learning techniques require hand-crafted features for learning. These attribute of deep learning models placed them in prominent position to be used for satellite image processing. This chapter presents a brief review of algorithms developed, challenges faced, and applications proposed in the field of deep learning and DNNs based satellite image processing.

The review involves the contribution and significance of deep learning and DNNs in unior multi-modal satellite image processing, single sensor or multi sensor image processing, different satellite image applications, single time stamp or multi-temporal satellite images, and any combination of the above. The review is divided into three categories; -first: brief review of recent developments in DNNs based unimodal satellite image processing analysis, and applications, -second: brief review of recent developments in DNNs based multisensor satellite image processing, analysis, and applications, and -third: brief review of recent developments in DNNs based models for satellite time series image processing, analysis, and applications. Since the studies on DNN based satellite image processing cover loads of fields, the current review is however focussed but not restricted to classification (e.g. land cover classification, crop classification).

2.1.

Brief Review of Recent Developments in DNNs based Unimodal Satellite Image Processing

  This section presents a brief review of the developments in DNNs based unimodal satellite image processing. Unimodal here refers to a single modality of satellite images i.e. either optical (surface reflectance), PolSAR (backscatter), thermal (brightness temperature) etc. This means that the studies reviewed in this section are developed or evaluated on a single modality of satellite images, may it be optical or PolSAR or any other. Studies on multimodal or multisensor satellite image processing with DNNs are not reviewed in this section. Studies on multimodal or multisensor satellite image processing with DNNs are reviewed in section 2.2 Also, this section does not include studies involving time series satellite images. Such studies are discussed later in section 2.3.

  Geng et al. in[START_REF] Geng | High-Resolution SAR Image Classification via Deep Convolutional Autoencoders[END_REF] proposed a deep convolutional autoencoder (DCAE) for automatically extracting features and performing classification. The first layer of DCAE are a grey level cooccurrence matrices and Gabor filters. The second layer of the DCAE integrates correlated neighbour pixels to reduce speckle. Following these two hand-crafted layers, a trained SAE, Satellite hyperspectral sensors capture images in hundreds of contiguous narrow spectral bands, producing three-dimensional hyperspectral imagery (HSI). The HIS images provide spectral and spatial information. With the launch of EnMAP, scheduled in 2020, and DESIS, which was planned originally for 2017, high-quality satellite hyperspectral data will be available.

	which is similar to [81], is attached for learning more abstract features. The method achieved Such wealthy spectral information data (with appropriate spatial context) can potentially reveal
	remarkable classification accuracy when tested on high-resolution single-polarization TerraSAR-interesting unknown content within the hyperspectral images. In particular, hyperspectral images
	X images. Similarly, Qin et al. in [83] applied adaptive boosting of RBMs to PolSAR image have been used widely in a range of practical applications, such as land cover classification,
	classification, stacked sparse AE to PolSAR image classification by taking into account local change detection and agriculture monitoring. Indeed, applications of hyperspectral image data
	spatial information is developed in [84], Zhao et al. in [85] proposed a discriminant deep belief have been one of the most active research interests.
	networks (DBNs) for land cover classification. Deep learning methods have recently been introduced into hyperspectral image
	Moreover, taking into account that most current deep learning methods aim at exploiting classification, achieving impressive results [96]-[99]. In hyperspectral images, spectral
	Compared to conventional methods, deep learning is advantageous in land cover classification The techniques mentioned in the last paragraph are focussed to develop DNN models for
	due to its capability of automatically learning discriminative features. Moreover, deep learning either preprocessing SAR images or obtaining desirable features that can be learned with DNNs.
	approaches such as CNNs can effectively extract not only polarimetric characteristics but also Direct utilization of raw SAR or PolSAR data with DNNs could lead to simpler models which
	spatial patterns of PolSAR images [75]. Deep learning is applied to land cover classification can extract abstract features directly from raw PolSAR data based on input-output relationship.

PolSAR images is rapidly advancing with the help of deep learning. For land cover classification from SAR and Polarimetric SAR (PolSAR) images, effective feature extraction is essential. Deep learning feature extraction has proved to overcome this challenge, to some degree

[START_REF] Parikh | Classification of SAR and PolSAR images using deep learning: a review[END_REF]

. Regarding feature extraction, most conventional methods rely on exploring physical scattering properties and texture information in PolSAR images.

using PolSAR images at first in

[START_REF] Xie | Multilayer Feature Learning for Polarimetric Synthetic Radar Data Classification[END_REF]

. They employed a stacked auto encoder (SAE) to automatically learn deep features from PolSAR data and then fed them to a softmax classifier.

Remarkable improvements in both classification accuracy and visual effect proved that this method can effectively learn a comprehensive feature representation for classification purposes. features either from polarization information or spatial information of PolSAR images, Gao et al. in

[START_REF] Gao | Dual-branch deep convolution neural network for polarimetric SAR image classification[END_REF]

proposed a dual-branch CNN to learn features from both perspectives for land cover classification.

Different variations of CNNs have been used for land cover classification with satellite

PolSAR images. A 6-channel covariance matrix is first extracted in

[START_REF] Zhou | Polarimetric SAR Image Classification Using Deep Convolutional Neural Networks[END_REF]

and then, the 6-channel covariance matrix is fed into a trainable CNN model to achieve land cover classification. In

[START_REF] Dong | PolSAR image classification with lightweight 3D convolutional networks[END_REF]

, the authors focused on the computational efficiency of deep learning methods, proposing the use of lightweight three dimensional CNNs. They showed that classification accuracy comparable to other CNN methods was achievable while significantly reducing the number of learned parameters and therefore gaining computational efficiency. Wang et al. in

[START_REF] Wang | A hierarchical fully convolutional network integrated with sparse and low-rank subspace representations for PolSAR imagery classification[END_REF]

proposed a fully convolutional network integrated with sparse and low-rank subspace representations for classification of satellite PolSAR images. The knowledge of target scattering mechanism interpretation and polarimetric feature extraction is used to improve land cover classification performance in

[START_REF] Chen | PolSAR image classification using polarimetric-feature-driven deep convolutional neural network[END_REF]

. He et al. in

[START_REF] He | Nonlinear manifold learning integrated with fully convolutional networks for PolSAR image classification[END_REF]

proposed the combination of features learned from nonlinear manifold embedding and applying a fully convolutional network on input PolSAR images; the final classification was carried out in an ensemble approach by SVM.

In this direction, Zhang et al. in

[START_REF] Zhang | Complex-valued convolutional neural network and its application in polarimetric SAR image classification[END_REF]

proposed a novel complex-valued CNN specifically designed to process complex values in PolSAR data with CNNs. Other notable complex-valued deep learning approaches for classification using PolSAR images can be found in

[START_REF] Mullissa | Polsarnet: A deep fully convolutional network for polarimetric sar image classification[END_REF]

-

[START_REF] Xie | PolSAR image classification via a novel semi-supervised recurrent complex-valued convolution neural network[END_REF]

.

resolution is relatively more important as compared to spatial information. In deep learning (DL) based spectral feature classification methods, DNN models take raw spectral vectors as inputs and output deep spectral features, which are used then for classification. Hu et al. in

[START_REF] Hu | Deep convolutional neural networks for hyperspectral image classification[END_REF]

Table 3 .

 3 1 Summary of ground truth sample points collected for study.

	Land Cover	Roorkee Training and validation	Testing
	BS	200	40
	TV	200	40
	SV	200	40
	Built-up	200	40
	Water	200	40
	Total	1000	200

Table 3 .

 3 2 Separability index (SI) values for various class pairs for all features.

	Class pair

Table 3 .

 3

	Class pair

3 Separability index (SI) values for various class pairs for popular features.

Table 3 .

 3 4 Classification performance of PS based and decomposition based PolSAR features.

	Method	Overall Accuracy (%)	Kappa Class	User Accuracy (%)	Producer Accuracy (%)
				BS	63.2	82.9
	H-A-Alpha-SVM	53	0.40	TV SV BU	44.6 58.8 14.3	88.9 53.3 1.3
				Water	47.7	12.8
				BS	66.1	82.3
				TV	70.9	88.0
	M-Y4D-SVM	72.4	0.65	SV	76.9	77.5
				BU	61.5	50.3
				Water	88.9	54.8
				BS	63.4	79.9
				TV	67.5	76.5
	GFD-SVM	71.9	0.63	SV	76.5	74.5
				BU	65.0	72.3
				Water	87.4	60.7
				BS	54.1	77.3
				TV	68.2	66.2
	Cloude-SVM	63.7	0.53	SV	67.9	59.5
				BU	52.5	58.5
				Water	85.9	57.0
				BS	69.5	79.9
				TV	69.8	89.1
	PSCF-SVM	74	0.69	SV	79	65.0
				BU	68.4	58.5
				Water	83.6	70.1

Classes: built-up (BU), water (W), bare soil (BS), short vegetation (SV), and tall vegetation (TV).

Table 3 .

 3 [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF] List of selected feature corresponding to land cover classes.

	Class	Feature selected for segregation
	Built-up	Corr_co_Di
	Water	Corr_co_HD, corr_co_VD
	Canal	VV polarimetric band
	Bare soil	Corr_co_FP

Table 3 .

 3 6 Updated list of selected feature corresponding to land cover classes.

	Class	Feature to be used for segregation
	Built-up	Corr_co_Di
	Water	Corr_co_HD, Corr_co_VD
	Water Canal	VV polarimetric band
	Bare soil	Corr_co_FP
	Tall vegetation	Corr_co_FP
	Short vegetation	Corr_co_FP

Table 3 .

 3 [START_REF] Singh | Spatial and temporal variability of soil moisture over India using IRS P4 MSMR data[END_REF] Polarization signature correlation feature statistics.

	Feature	Mean (M) Std. dev. (S)
	Corr_co_FP	0.5269	0.1266
	Corr_co_Di	0.357	0.0716
	Corr_co_HD	0.0580	0.1850
	Corr_co_VD	-0.0580	0.1850
	VV	-11.243	1.564

  𝑀 corr_co_Di + 𝑛 1 * 𝑆 cor_co_Di is the class lower boundary value and 𝑀 corr_co_Di + 𝑛 2 * 𝑆 corr_co_Di is class upper boundary value. This expression means, if corr_co_Di pixel value lies in between these boundary values then the pixel belongs to

	 For Tall vegetation
	𝑀 corr_co_FP + 𝑛 9 * 𝑆 corr_co_FP ≤ corr_co_FP ≤ 𝑀 corr_co_FP + 𝑛 10 * 𝑆 corr_co_FP	(3.9)
	Experiments suggest that class boundary thresholds for canal are not affected by VV band
	statistics and hence are not included for further study.
	The goal of representing the class boundaries as in equation (3.5) through (3.9) is to obtain
	class boundaries that achieve best classification results under a given situation. Overall accuracy
	(OA) is an indicator of classification performance. Higher the OA, better the classification result.
	It is straightforward to understand that more accurate class boundaries result in higher OA. An
	empirical relationship is intuitive and can be developed between OA and class boundaries. The
	empirical relationship can then be treated as an optimization problem which can be addressed by
	an optimization technique. Since, M and S are constant for a given PSCF, therefore class
	segregated by "corr_co_Di" feature, orr_co_Di c M corr_co_Di. Therefore, built-up class spread can be expressed as; and orr_co_Di c S are mean and standard deviation of 𝑀 𝑐orr_co_Di + 𝑛 1 * 𝑆 corr_co_Di ≤ 𝑐orr_co_Di ≤ 𝑀 𝑐orr_co_Di + 𝑛 2 * 𝑆 corr_co_Di (3.5) Where, 1 n and 2 n are real and scalar. built-up class. Similar expressions are developed for other classes (refer equation (3.6) through (3.9)).  For Bare Soil 𝑀 corr_co_FP + 𝑛 3 * 𝑆 corr_co_FP ≤ corr_co_FP ≤ 𝑀 corr_co_FP + 𝑛 4 * 𝑆 corr_co_FP (3.6)  For Water 𝑀 corr_co_HD + 𝑛 5 * 𝑆 corr_co_HD ≤ corr_co_H ≤ 𝑀 corr_co_HD + 𝑛 6 * 𝑆 corr_co_HD (3.7)  For Short vegetation 𝑀 corr_co_FP + 𝑛 7 * 𝑆 corr_co_FP ≤ corr_co_FP ≤ 𝑀 corr_co_FP + 𝑛 8 * 𝑆 corr_co_FP (3.8) boundary varies with only with i n (i = 1 to 10). An empirical relationship between OA and i = 1 to 10) is relevant. Section 3.5.3 discusses the development of an empirical relationship between OA and n (i i

.6

). The relationship is formulated and explained with an example. Let's take the case of "built-up" class. Built-up is strongly

Table 3

 3 .6 for segregating corresponding classes. Hence, for each class, individual overall accuracy (IOA) is computed and OA is rewritten as equation(3.11). .8. These relations are developed using surface fitting methods. For example,

	OA =	IOA	built-up	+IOA	baresoil	+IOA 5	water	SV +IOA +IOA	TV	(3.11)
	Where, for example	IOA	tall_veg	is individual overall accuracy (IOA) in TV vs other class
	classification scenario using corr_co_FP as feature. Further, these IOAs can be empirically
	related to

i n (i = 1 to 10). Empirical relations developed between IOAs and i n (i = 1 to 10) are 68 provided in

Table 3 .

 3 8 Empirical relations between individual overall accuracies (IOA) and i n values.

	Performance	Derived empirical relation	𝑹 𝟐 (coefficient of
	parameters	determination)
	IOA		built-up	sin (33.5 * 𝑛 1 + 6.4) 80.41 * sin (5.3 * 𝑛 1 -2.9) + 5.1 *	0.99
	IOA	bare_soil	sin (69.2 * 𝑛 3 -21.6) 79.44 * sin(5.5 * 𝑛 3 -2.9) + 1.1 *	0.96
	IOA		water	sin(4.7 * 𝑛 5 -1.3) + 1.23 * sin(51.2 * 𝑛 5 + 2000 * sin(4.7 * 𝑛 5 + 1.9) + 1993 *	0.92
					2.2)
	IOA	SV	93.5 * 𝑛 7 2 + 1250 * 𝑛 7 * 𝑛 8 -1984 * 𝑛 8 2 -513.4 -902.6 * 𝑛 7 + 2210 * 𝑛 8 +	0.91
	IOA	TV	-573.7 -998.9 * 𝑛 9 + 1742 * 𝑛 10 -1491 * 𝑛 9 2 -27.5 * 𝑛 9 * 𝑛 10 -1568 * 𝑛 10

Table 3 .

 3 [START_REF] Venkataraman | Fully polarimetric ALOS PALSAR data applications for snow and ice studies[END_REF] Optimal i n values obtained using the genetic algorithm.

	n1	n2	n3	n4	n5	n6	n7	n8	n9	n10
	0.315	2.0	0.446	1.0	-2.0	-1.667	-1.4	-0.2	-0.022	0.998

Table 3 .

 3 [START_REF] Singh | Categorization of the glaciated terrain of Indian Himalaya using CP and FP mode SAR[END_REF]. From Table3.10, it is evident that the proposed method achieves best classification accuracy (75.3%) with kappa coefficient (0.69). Also, the user accuracy for built-up and TV have improved to 73.8% and 72.4% respectively (Marker A, and B respectively). Markers A and B in

Table 3 .

 3 10: Classification performance indicators for the proposed method and RBF-SVM classifier with PSCFs as features.

	Method	OA (%) Kappa Class	User Accuracy (%)	Producer Accuracy (%)
				BS	69.5	79.9
	Proposed optimal class			TV	73.8	89.1
	boundary estimator based	75.3	0.69	SV	78.9	65.0
	DT classifier			BU	72.4	71.0
				Water	83.6	70.0
				BS	69.5	79.9
				TV	69.8	89.1
	RBF-SVM	74	0.69	SV	79	65.0
				BU	68.4	58.5
				Water	83.6	70.1

  Further, two fully connected or FC layers, first with 50 nodes and second with 10 nodes are considered. Both FC layers used the ReLU activation function and a dropout fraction of 10% . Finally, a softmax classification layer with 5 labels (based on 5 output land cover classes) is used. A batch normalization process is carried out before the convolutions to speed up the training process and to reduce the network sensitivity. A seven layered PSs-CNN classification model is realized as shown in Figure 3.13.

	The proposed PSs-CNN model is trained on labelled samples provided in Table 3.1. The
	loss function considered during training is the categorical cross entropy function. The learning

CNN model is constructed and configured with the help of convolutional and fully connected layers to perform land cover classification with PSs. The proposed CNN model is termed as PSs-CNN model. The proposed PSs-CNN model configuration is provided in Table 3.11. The input to the PSs-CNN model is a pixel PSs. Pixel (observed) PSs used here are obtained as discussed in section 3.4.1. Each PS corresponds to a 19 37  image. The 19 37  input image has two channels i.e. copol and crosspol. Therefore, overall the size of input to the PSs-CNN is19 37 2  . At first, the PSs-CNN used four convolutional layers. Number of convolution filters and filter sizes used in each layer are listed

Table 3.11. ReLU activation function and dropout fraction of 10% is used in each convolutional layer. The dropout regularization strategy is employed to achieve less overfitting and better generalization. rate is set to 0.01 and the number of training epochs is set to 200. Adam optimizer is used for network weight optimization.

Table 3 .

 3 11 Summary of PSs-CNN model configuration, hyperparameters and training

	Layer	Input	Convolutional layers;	Fully connected	Output layer
			C1, C2, C3, C4,C5	layers, FC1, FC2
	PSs-CNN	Input image size =
	Architecture

Table 3 .

 3 12 Classification performance indicators for the PSs-CNN, NSCM, and NED classifiers with PSs as features.. BS = bare soil, TV = Tall vegetation, SV = Short vegetation, BU = Built-up.

	Method	OA (%) Kappa Class User Accuracy (%) Producer Accuracy (%)
				BS	88.5	79.9
				TV	73.8	89.1
	PSs-CNN	72.8	0.69	SV	62.9	65.0
				BU	70	58.5
				Water	79.6	70.1
				BS	100.0	61.5
				TV	100.0	44.5
	NSCM	64.1	0.54	SV	65.6	82.2
				BU	38.5	57.6
				Water	56.2	73.2
				BS	88..4	61.1
				TV	100.0	44.0
	NED	62.3	0.51	SV	65.5	82.2
				BU	33.8	47.0
				Water	56.2	73.2

Table 3 .

 3 13 Classification performance indicators for the optimal class boundary estimator based DT, PSs-CNN, and RBF-SVM classifier. Classification performance of the proposed approach especially user and producer accuracies for built-up class (72.4% and 71.0% respectively) and tall veg class (73.8%, and 89.1% respectively) indicate that the proposed approach, i.e. the optimal class boundary estimator based DT classifier, achieved better classification results in mixed LC class scenarios. In the third sub task i.e. sub task 3, development of a two dimensional CNN (2D-CNN) based novel model for LC classification is presented. The 2D-CNNs can directly exploit PSs to perform LC classification, and therefore, the approach reduces the efforts to compute PSCFs. Also, since 2D-CNNs are powerful image extractors, they are able to extract the subtle contextual patterns present in PSs of different LCs. In summary, a novel PSs-CNN LC classification model is realized. Classification results from this PSs-CNN model are compared with NED and NSCM based minimum distance classifiers which are popularly used with PSs for land cover classification. Comparison indicated a superior performance of the novel PSs-CNN model for land cover classification.

	.13. The comparison

Overall a study where an attempt is made to extract information from one of the most popular satellite image modality i.e. PolSAR images is presented in this chapter. The study is composed of three sub-tasks. Under the first sub task i.e. sub task 1, the investigation of polarization signatures (PSs) as potential PolSAR features for land cover (LC) classification in mixed class scenarios is done. Novel PSs based features i.e. PSCFs are computed based on the degree of similarity between observed and standard PSs. Further, PSCFs are analysed, and are compared with decomposition based features for the purpose of LC classification. Comparisons indicate potential of PSCFs as significant features for LC classification in mixed land cover class scenarios. boundary values. Optimization of this empirical relationship with genetic algorithm resulted in optimal class boundaries which are utilized in a DT to perform LC classification. Classification results using this approach are compared with classification results from RBF-SVM, while keeping the features identical i.e. PSCFs. Comparison between the two novel approaches proposed in this study i.e. the optimal class boundary estimator based DT classifier and the PSs-CNN model based classifier is made. The comparison indicated that the optimal class boundary estimator based DT classifier is showing an approx. 3% improved classification accuracy than PSs-CNN classifier. However, the PSs-CNN model is easier to implement, does not require an extra step (computing PSCFs), and is scalable.

  . PolSAR data is usually captured in four linear polarization states namely horizontal transmithorizontal receive (HH), horizontal transmitvertical receive (HV), vertical transmithorizontal receive (VH), and vertical transmitvertical receive (VV). In fact, the HH, HV, VH, and VV polarization states indicate to the double bounce, multiple bounce, and single bounce scattering behaviours of a target respectively. The scattering responses in different polarization states help the identification and classification of different land covers which is helpful during flooded area assessment. For example, built-up structures usually show double bounce scattering response. This means that these built-up structures appear bright (high scattering response) in HH PolSAR image. On the other hand, tall vegetation areas (e.g. forests) show multiple bounce scattering (volume scattering) response, or smooth surfaces such as water show single bounce scattering response

  , design and development of a novel PSs based CNN model for extraction of builtup areas in PolSAR image. The design and development of the novel PSs based CNN model is discussed in sections 4.3.1 and 4.3.2. Model performance evaluation and comparisons are discussed in section 4.3.2.Second, development of a novel methodology for design of village detection filters using apriori knowledge and image morphology. These filters help in detection of villages in a builtup area image and also help suppress false alarms or misclassifications. Section 4.3.3.1 covers the development of this methodology. Qualitative and quantitative evaluation of the detection performance of this methodology is also covered in this section.

Third, development of methodology for identification and mapping of villages that are affected by flood and masking out villages that are not affected by floods. A novel methodology, which exploits the phenomena of enclosing of a village by water if it's flooded, is developed. Section 4.4.3.3.2 discusses the development of this methodology. Evaluation of this novel methodology is discussed in section 4.4.3.3.2. The three methodologies proposed in the three stages sum up to provide a novel flood affected village detection and mapping approach.

  this stage, CNNs are utilized to extract features from PSs for built-up area extraction. A CNN based PSs classification technique is first introduced in chapter 3. The technique was able to capture the influence of the neighbourhood of a PS pixel and thereby utilizing the PSs more efficiently. A novel CNN model similar to the model discussed in section 3.6 is developed here with the help of convolutional and fully connected, layers. The proposed CNN model is termed as PSs-CNN model from here onwards in this chapter. The proposed PSs-CNN model configuration is provided in Table 4.2. Unlike the CNN model in chapter 3 (refer section 3.6)

Table 4 .

 4 3 Summary of the classification performance of the considered classifiers considered for Roorkee and Haridwar datasets. OA = overall accuracy. Rest = BS + Water + SV.

	Dataset	Method	OA	Quadratic	User accuracy (%)	Producer accuracy (%)
			(%)	Kappa	Rest TV	Built-	Rest	TV	Built-up
						up
	Roorkee PSs-CNN 87.0	0.77 96.2 71.4	81.8	85.8 87.5	90.0
		NSCM	74.7	0.56 86.7 57.6	60.6	83	75	50
		NED	75.0	0.55 86.2 56.8	63.6	83.3 72.5	52.5
	Haridwar PSs-CNN 89.0	0.80 96.5 75.8	82.5	93.3 72.5	92.5
		NSCM	76.0	0.58 91.8 52.7	58.4	85 47.5	77.5
		NED	73.0	0.51 83.7 51.2	62.1	83.3 47.5	67.5

Table 4 .

 4 4 summary of quantitative assessment of performance of PSS-CNN, NED, and NSCM

	Dataset	Measures	NED	NSCM	PSs-CNN
	Roorkee Total number of villages in the study area	93	93	93
		TP (Villages successfully mapped)	26	22	75
		TN (Villages missed)	67	71	18
		FP ( or False Alarms)	31	41	1
	Haridwar Total number of villages in the study area	60	60	60
		TP (Villages successfully mapped)	20	27	47
		TN (Villages missed)	40	33	13
		FP (or False Alarms)	33	39	5

Table 5 .

 5 [START_REF] Penatti | Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?[END_REF] Separability index (SI) values for all the possible class pairs for the 2D CNN-v1, 3D CNN-v1, and 'Perm-LSS-CNN filters.

	Class pairs	Highest Separability Index (SI) value
		2D CNN-v1	3D-CNN-v1	Perm-LSS-CNN
	BS vs Water	1.82	1.99	2.87
	BS vs Built-up	2.09	2.35	2.90
	BS vs TV	2.98	3.82	3.75
	BS vs SV	1.29	1.69	1.87
	Water vs Built-up	3.05	3.04	3.57
	Water vs TV	2.59	3.75	4.38
	Water vs SV	1.43	1.29	1.64
	Built-up vs TV	2.81	3.64	5.47
	Built-up vs SV	1.70	2.10	2.27
	TV vs SV	1.47	1.56	1.77

Table 5

 5 reports a detailed comparison of deep neural networks with SVMs for land cover classification with satellite image data. They deplore the lack of case studies where a The number of features to consider when looking for the best split is set to 4, and (b) The number of trees in the forest is set to 41. From Table5.5 and Table5.6, it is .5 and Table5.6 for the Roorkee and the Haridwar data sets. The difference between precision and recall averaged over all land cover classes (termed as AD in Table5.5 and Table5.6 is computed to evaluate the generalization ability and consistency of the classifiers.AD is minimum with Perm-LSS-CNN for Roorkee data and second minimum for the Haridwar data. It is also observed that the AD is consistent for both data sets, i.e. around 3 whereas the AD for other classifiers fluctuates with data sets. This indicates the good generalization ability of the proposed classifier. Further, Table5.5 and Table5.6 also show that even if the overall accuracy of the proposed Perm-LSS-CNN is best, the other classifiers have shown good overall accuracies as well.

	quantitative and qualitative comparison of deep neural networks with SVMs with multi or high
	spectral satellite image data. Therefore, two non-CNN classifiers namely radial basis function-
	support vector machine (RBF-SVM), and random forest (RF) [33] are also considered for
	comparisons. The RBF-SVM classifier parameters settings are; (a). Regularization parameter is
	set to 12.7, and (b). Kernel influence parameter is set to 0.056568. The random forest classifier
	parameters settings are; (a) evident that the proposed Perm-LSS-CNN provides the best performance for both the datasets
	with a 10-fold cross validation overall accuracy of 97.8 and quadratic kappa of 0.972 for the
	Roorkee data set, overall accuracy of 97.9 and quadratic kappa of 0.974 for the Haridwar data
	set. Individual class precision (a.k.a. user) and recall (a.k.a. producer) accuracies are also
	provided in

Table 5 .

 5 [START_REF] Zhang | Deep learning for Remote Sensing Data: A Technical Tutorial in the State of the Art[END_REF] Classification performance of various classifiers considered for PolSAR-MS Roorkee dataset. *AD is the difference between precision and recall, averaged over all the land cover classes considered. BS = bare soil, TV = tall vegetation, SV = short vegetation, BU = built-up, W = water.

	Method	OA (%) Kappa	User accuracy (%)		Producer accuracy (%) AD*
				BS TV SV BU	W	BS TV SV BU	W
	1D-CNN-v1	95	0.938 94.5 100 85.2 100 97.5 93.4 92.4	97 97.5 96.3	4.9
	1D-CNN-v2	95.2	0.939 96.4 100 84.5 100 97.1	93 91.8 96.6 98.1 98.4	5.3
	Perm-LS-CNN	95.7	0.946	97 100 86.5 100 97.1 94.4 92.1 94.2 98.1 98.4	4.3
	2D-CNN-v1	95.3	0.941 96.4 100 83.9 100 98.4 94.1 92.7	98 96.9 96.3	5.8
	3D-CNN-v1	96.3	0.933 98.8 100 84.5 100 99.6 95.6	91 100 98.7 98.4	6.1
	Perm-LSS-CNN	97.8	0.972 99.7 99.6 91.1 100 99.3 94.3 98.3 99.6 98.7	99	3.3
	RBF-SVM	94.6	0.93 94.6 99.3 90.2 96.2	94 92.6	94 93.5	98 96.7	3.2
	RF	91.7	0.932 86.8 96.8 88.2 96.8 93.1 92.1 97.1	65 88.6 90.6	7.9

Table 5 .

 5 [START_REF] Townsend | Modeling floodplain inundation using an integrated GIS with radar and optical remote sensing[END_REF] Classification performance of various classifiers considered for PolSAR-MS Haridwar dataset. *AD is the difference between precision and recall, averaged over all the land cover classes considered. BS = bare soil, TV = tall vegetation, SV = short vegetation, BU = built-up, W = water.

	Method	OA (%) Kappa	User accuracy (%) BS TV SV BU W	Producer accuracy (%) AD* BS TV SV BU W
	1D-CNN-v1	94.1	0.925 100 100 70.2 96.6 99.3 97.1 87.1 98.9	95 99.3	9.2
	1D-CNN-v2	95	0.937 100 100 73.2 99.1 99.3 99.2 87.8 100 95.9 99.3	8.6
	Perm-LS-CNN	96.1	0.95 100 100 77.8 100 100 100 89.2 100 97.5 100	7.1
	2D-CNN-v1	94.7	0.933 100 100 70.9 99.6 100 97.8 89.2 100 92.9 99.3	9.9
	3D-CNN-v1	96.5	0.955 100 100 80.1 99.1 100 98.5 90.3	99 100 100	6.2
	Perm-LSS-CNN	97.9	0.974 100 100 89.3 99.1 100 99.2	96 99.1 96.7 100	3.4
	RBF-SVM	96.1	0.95 98.5 100 86.2 95.8 97.2 97.8 92.7 95.7 96.6 100	4.3
	RF	94.4	0.928	97 98.6 86.2 91.6 95.1 97.7 95.9 84.9 91.6 100	1.9

RNN, AND CNN-RNN MODELS FOR MULTISENSOR SATELLITE TIME SERIES IMAGE BASED CROP CLASSIFICATION

  These CNNs are applied to land cover classification with multisensor (PolSAR-MS) satellite image data. Two novel models are introduced, where local filters in either spectral domain (Perm-LS-CNN) or spatial-spectral domains (Perm-LSS-CNN) are applied after a permutation layer. The permutation layer generates an ensemble of band combinations, which are drawn upon by the CNN in order to generate powerful features for land cover classification. Classification results demonstrate the improved generalization ability of these models compared to both alternative CNN architectures and other machine learning models (SVM and random forests). Perm-LSS-CNN also fully uses both PolSAR and multispectral information, improving results compared to other popular methods for PolSAR-MS data based land cover classification.Crop classification is an important task in many crop monitoring applications. Satellite image based monitoring provides an easy, techno-economical, reliable, stable and fast approaches to crop monitoring tasks. Numerous techniques have been reported for satellite time series image based crop classification. However, with the availability of more and more high quality satellite images from new and powerful sensors, which are increasing by the day, it has become a challenge for techniques to keep up in terms of performance, high dimensional operability, Therefore, it is still a challenge for existing DNN based techniques to synergistically exploit the spatial, spectral, and temporal information of a crop present in the multimodal and/or multisensor satellite time series images and provide accurate and reliable crop type information in mixed land cover scenarios.

	CHAPTER 6
	NOVEL CNN,
	123

A separability measure is used for the model significance evaluation. Separability analysis states that the Perm-LSS-CNN transforms the input data space into a feature space where separability among classes increases. This increase in separability is achieved due to the inclusion of permuted local spectral convolution along with the local spatial convolution which is not available in the 1D-CNN-v1, 2D-CNN-v1, or 3D-CNN-v1 CNN models. A fine analysis on ground truth data provides an extra qualitative assessment which complements the quantitative classification results. These qualitative assessments demonstrate that the proposed Perm-LSS-CNN model is able to generalize much better while keeping high classification accuracies. scalability, and generalizability. The new technologies must should have a unified framework to be able to process plethora of multi-resolution, multisensor, and high dimensional satellite time series images. In the context, CNNs and recurrent neural networks (RNNs) are the most recent and trending deep neural network frameworks for satellite time series image based agriculture crop monitoring applications. Both, CNN and RNN models have been developed in, standalone and synergistic modes for agriculture crop classification by many researchers. However, the CNN and RNN models are still young, promising, and growing techniques in the satellite image proceeding community especially in multisensor satellite time series image processing and therefore further exploration of the potential of these models is motivating. Also, certain limitations such as presence of mixed land cover scenario still exist. Mixed land cover scenario in crop classification refers to the presence of other land covers (built-up, forests, or grasslands) in-between and around the croplands in the study region (e.g. in India).

  , It is observed that random forest, maximum likelihood classifier, and neural networks are the most popular methods utilized in satellite time series image based crop classification applications. However, with the availability of more and more high quality satellite images from new and powerful sensors, which are increasing by the day, it has become a challenge for these techniques to keep up in terms of performance, high dimensional operability, scalability, and generalizability. Therefore, it is important to level up the technologies. The technologies must be able to process plethora of multi-resolution, multisensor, and high dimensional satellite time series images.In this regard, recently, deep learning with deep neural networks is one of the most promising techniques. Numerous successful approaches have been reported in the recent years which are based on deep neural networks for satellite time series image applications. In particular, CNNs and recurrent neural networks (RNNs) are the most popular deep neural network types utilized in satellite time series image based agriculture crop monitoring applications. Further, novel CNN and RNN models have also been developed and utilized in both, standalone and synergistic, modes for agriculture crop classification by many researchers. A detailed review on popular CNN and RNN models for agriculture crop monitoring and classification applications is presented in chapter 2 (refer section 2.3). The review indicated to the success and significance of deep neural network models in satellite time series image based crop classification applications. However, the technology is still young and growing, and there is scope for much more. In particular, the DNN model based studies for time series satellite image processing discussed in section 2.3 are focused more on either the spectral, spatial, or temporal characteristic or at most more focussed on two characteristics at a time ignoring the third characteristic. For example, some CNN and of other land covers (built-up, forests, or grasslands) in-between and around the croplands in the study region (e.g. in India). Therefore, it is still a challenge for existing DNN based techniques to synergistically exploit the spatial, spectral, and temporal information of a crop present in the multimodal and/or multisensor satellite time series images and provide accurate and reliable crop classification.The above arguments motivate towards new studies focused on using multisensor satellite time series images, especially optical and PolSAR, for crop classification and development of novel CNN and RNN models which focus on the judicious exploitation of the spectral, temporal and spatial characteristics of crops to achieve improves classification results.In the study presented in this chapter, an attempt is made; to analyse the conventional CNN and RNN models, and to develop novel CNN and RNN (in standalone as well as synergistic modes) models for agriculture crop classification with multisensor/multimodal satellite time series (MSTS) imagery. For this purpose, two sub tasks are set in this study, -first, critical analysis of conventional CNN and RNN models for MSTS images based crop classification, and

	RNN studies focused on spectral features more than temporal or spatial features during
	classification which lead to under-utilization of temporal and spatial features during crop
	classification. Similarly, some other studies focused more on temporal and spatial characteristics
	and less on spectral characteristic which again lead to under-utilization of spectral features of the

crops. This imbalance, in priority, requires investigation on the subject and demands novel CNN and RNN model based approaches that can address this issue. To this day, only few studies focussed on utilizing all three characteristics have been reported but a trend is underway. Other major limitation during crop classification is the presence of mixed land cover scenario i.e. presence -second, development of novel CNN and RNN models for MSTS images based crop classification. In the first sub task i.e. sub task 1, conventional CNN and RNN models are

Table 6 .

 6 1 Geographical extent of study area.

	Coordinates (upper left) Coordinates (lower right)
	Latitude	Longitude	Latitude	Longitude
	29.84183	77.02115	29.78843	78.09080

Table 6 .

 6 3 lists the specifications of bands considered from Sentinel-1 and Sentinel-2. Overall, 10 multispectral bands and 2 PolSAR channels are considered for the study. Also, 7 time steps from October 2017 to April 2018 are considered for the study. Figure6.7 shows a false colour composite of the dataset for the study area and highlights the varied crop types.

1

PolSAR and Sentinel 2 multispectral satellite sensor images are used. Sentinel-1 C band dual pol SAR images are available at 10 m spatial resolution and at a revisit time of 6 days (12 if pass direction is considered). Similarly, Sentinel-2 multispectral images are available at 10m, 20m, and 60m spatial resolutions and at a revisit time period of 5 days Both sensor datasets are freely available for public use. Based on the growing season of wheat, Sentinel-1 ground range detected (GRD) and Sentinel-2 L1C satellite time series images are collected every month throughout the wheat growing season for year 2017-18. Acquisition dates of each month for both sensors are listed in Table

6

.2. Also,

Table 6 .

 6 2 Sentinel-1 and Sentinel-2 data acquisition dates.

	Month	Acquisition dates
		Sentinel-1 Sentinel-2
	October	3/10/2017 7/10/2017
	November 20/11/2017 21/11/2017
	December	2/12/2017 1/12/2017
	February	12/2/2018	9/2/2018
	March	8/3/2018	6/3/2018
	March	1/4/2018 31/3/2018
	April	13/4/2018 15/4/2018

Table 6 .

 6 3 Band specifications of Sentinel-1 and Sentinel-2.

	Sensor	Band selection Spatial resolution (meters)
	Sentinel-1	VH	10
		VV
	Sentinel-2	Band 2: Blue	10
		Band 3: Green
		Band 4: Red
		Band 8: NIR
		Band 5: Vegetation Red Edge	20
		Band 6: Vegetation Red Edge
		Band 7: Vegetation Red Edge
		Band 8A: Vegetation Red Edge
		Band 11: SWIR-1
		Band 11: SWIR-2

0.99

Chapter 4

NOVEL APPROACH FOR DETECTION OF HUMAN SETTLEMENTS AFFECTED DURING FLOODS USING POLARIZATION SIGNATURES AND CNNS

During floods, an updated and accurate information on flood affected human settlements helps mitigate the disaster efficiently. This chapter presents an efficient approach to map human settlements, especially villages in the rural parts of India that are affected during flood using PolSAR imagery, convolutional neural networks (CNNs), and image morphology. This chapter is an extension of chapter 3 in a sense to utilize the PSs-CNN model (proposed in chapter 3) for a crucial, and practical application.

The approach involves three stages: -first, built-up area extraction from a PolSAR image, -second, detection of villages in a built-up area image, and -third, identification and mapping villages that are affected by flood. In the first stage, a CNN based novel classification model is proposed to extract built-up areas from PolSAR image. This model utilizes PolSAR polarization signatures (PSs) as features. The second stage proposes a novel village detection methodology.

This methodology detects whether a particular object in the built-up area image (obtained in the first stage) is a village or not. In the third stage, flood affected villages are identified and mapped and non-flooded villages are masked out using another image morphology based methodology.

This chapter is divided into following sections. Section 4.2 provides detailed information on the experiment setup, study areas and datasets considered and, ancillary information collected for the study. Section 4.3 discusses the methodologies developed in the three stages of the study.

Qualitative and quantitative evaluation of the results from each methodology is also done and discussed in this section. At last, section 4.4 concludes the study.

Experiment Setup

Study area

Two areas are considered for the analysis, development and testing of the proposed methodology.

The first study area is common from chapter 3. This study area is termed as 'Roorkee' study area in this chapter. Details of this study area are already provided in section 3. This second study area is termed as 'Haridwar' study area in this chapter. Both study areas have mixed class type landscape, which is beneficial for the evaluation of classifiers performance in mixed class scenarios.

Satellite data

PolSAR data used in chapter 3 (refer section 3.3.2) is considered here as well for experiment purpose. Preprocessing is required with raw PolSAR data as it suffers from artefacts such as speckle noise. Preprocessing of the PALSAR-2 PolSAR data is similar to as explained in chapter 3 (refer section 3.3.4). Four features namely HH, HV, VH, and VV are obtained. Since, HV and VH are similar due to the reciprocity theorem, only HH, HV, and VV are used further.

2D-CNNs for PolSAR-MS data

The use of spectral information without spatial information by 1D-CNNs undermines the full potential of satellite image data. Adding spatial information provides contextual aid to the classifiers, especially CNNs in the classification process. Two dimensional or 2D-CNNs have also been successfully used with unimodal satellite image data in the recent years. In [START_REF] Sharma | A patch-based convolutional neural network for remote sensing image classification[END_REF],

authors used patch based 2D-CNN model for land cover classification with multispectral data.

They opted a 5 × 5 × 8 Landsat-8 image patch and a 3 × 3 2D convolutional kernel. In [START_REF] Geng | High-Resolution SAR Image Classification via Deep Convolutional Autoencoders[END_REF],

authors used 2D-CNNs for land cover classification with PolSAR data. Many studies used 2D-CNNs for classification with hyperspectral satellite image data (refer section 2.1). The need for dimension reduction in these studies is already discussed in section 5.4.1. These studies support the utilization of 2D-CNNs with PolSAR-MS data for land cover classification. In [START_REF] Kussul | Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[END_REF], authors used 2D-CNNs for crop type classification with PolSAR and multispectral data. They used Landsat-8 multispectral and Sentinel-1 C-Band PolSAR data. Although in [START_REF] Kussul | Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[END_REF] authors have studied the PolSAR-MS data potential for crop type classification with 2D-CNNs, the current study differs on:first, using PolSAR-MS on a single time stamp with a focus on spectral and spatial features. In [START_REF] Kussul | Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[END_REF], authors used time series PolSAR-MS data with a focus on temporal and spectral features; andsecond, Quad PolSAR data is considered in the present study in contrast to dual PolSAR used in [START_REF] Kussul | Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[END_REF]. The increased channels add complementary information which in turn add an indirect weighting of the spatial features. Analysis of fully polarimetric PolSAR with multispectral data for classification is therefore beneficial. 5.2.

3D-CNNs for PolSAR-MS data

The 2D-CNN-v1 model discussed in section 5.4.2 is successful because of its spatial contextual awareness. The localized spatial convolutions provide local connectivity, which controls the impact of neighbouring pixels on the pixel to classify. Parameter sharing between local filters also makes the learning faster and less cumbersome, in contrast to fully connected neural networks. In satellite image data applications, spectral information is as important, if not more, as spatial information. Local connectivity in spectral dimension also provides the same benefits.

Advantages of local spectral convolutions and local spatial convolutions together can be exploited using 3D-CNNs. The 3D-CNNs have been successfully used with hyperspectral (unimodal) satellite image data in the recent years. Authors in [START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF], [START_REF] Zhang | Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network[END_REF], and [START_REF] Feng | CNN-based multilayer spatial-spectral feature fusion and sample augmentation with local and nonlocal constraints for hyperspectral image classification[END_REF] 6.4. Few images from the ground truth survey are also shown in Figure 6.8.

Four subset areas from the study area are selected for visual and qualitative evaluation.

3D-CNN model for MSTS images based crop classification

The 3D-CNN model proposed here operates (window based convolution) along the time and spatial dimensions. The spectral dimension corresponds to the channels in the input data during the convolution.

In the 3D-CNN model namely 3D-CNN-v1, the size of the three dimensional kernel of the filter in the convolutional layer i.e. C1 layer is set equal to the temporal depth of MSTS image i.e. 7 in the temporal dimension and is set equal to 33  in the spatial dimension. Three dimensional temporal-spatial convolution is similar to spectral-spatial convolution shown in Figure 5.5(c) with the difference that spectral dimension is replaced with temporal dimension.

Localized convolution is not considered in the temporal dimension as the temporal depth is already low. The present study also focuses on the analysis of the performance of 3D-CNNs for MSTS image based crop classification. The 3D-CNN-v1 architecture and hyper-parameter settings are listed in Table 6.5.

RNN model for MSTS images based crop classification

A conventional RNN model is considered here for analysis. The input to this RNN model namely RNN-v1 is a 7 12  matrix where 7 indicates the temporal depth (time stamps) and 12 indicates to the spectral depth in the multisensor image data in each time stamp. Since, RNNs do not accommodate spatial contextual information, per-pixel approach is considered here. A simple LSTM unit is selected as recurrent cell. Since the temporal depth is 7, seven LSTM units constitute the R1 recurrent layer. The architecture is used in a many-to-many fashion. It means the input to the recurrent layer is more than one dimensional (in this case, 7) and output from the recurrent layer is more than one dimensional (in this case, 7). Also, the flow of activation from each RNN cell in the recurrent layer is considered in a bidirectional manner. It means that the activation from particular recurrent cell can move to next recurrent cell (next time stamp, usual) and previous recurrent cell (previous time stamp). This helps in the crop classification and its significance is discussed in the results and discussion section later in this chapter. Further a FC layer with 20 nodes and output layer with 6 outputs is set.

The RNN-v1 architecture and hyper-parameter settings are listed in

CNN-RNN models for MSTS images based crop classification

In this section, CNN and RNN architectures are used synergistically to form CNN-RNN models for crop classification with MSTS images. The previous standalone CNN or RNN models are not able to completely exploit the spectral, spatial, and temporal information simultaneously. At best, the standalone 3D-CNN model i.e. 3D-CNN-v1 is able to exploit spectral-spatial information or temporal-spatial simultaneously but not all three simultaneously. Alternatively, the standalone RNN model i.e. RNN-v1 could benefit from the temporal information but is unable to extract spatial or spectral features. Therefore, a synergistic utilization of CNN and RNN models is straightforward beneficial. In the novel models developed in the next few sections, the CNNs are used to extract spectral and/or spatial features and RNNs are used to extract temporal features.

The CNN architectures are used prior to RNN architectures in a time-distributed fashion to preserve the temporal information of the MSTS images.

1D-CNN-RNN model for MSTS images based crop classification

At first, a 1D-CNN architecture is used here to extract spectral features as the temporal information is exploited using a RNN architecture. The kernel size of the filter in the convolutional layer i.e. C1 layer is equal to the spectral depth of MSTS image i.e. 12. One dimensional spectral convolution is shown in Figure 5.4(b).

Then, a conventional RNN model is considered. Since, 1D-CNNs or RNNs cannot accommodate spatial contextual information, per-pixel approach is considered here. A simple LSTM unit is selected as recurrent cell similar to the RNN-v1 model. Since the temporal depth is 7, seven LSTM units constitute the R1 recurrent layer. Here again, the RNN architecture is used in a many-to-many fashion. It means the input to the RNN layer is more than one dimensional (in this case, 7) and output from the RNN layer is more than one dimensional (in this case, 7). Also, the flow of activation from each recurrent cell in the recurrent layer is considered in a bidirectional manner. It means that the activation from particular recurrent cell can move to next recurrent cell (next time stamp, usual) and previous RNN cell (previous time stamp). This helps in the crop classification and the significance is discussed in the results and discussion section later in this chapter. Further a FC layer with 20 nodes and output layer with 6 outputs is set.

The consolidated CNN-RNN model is termed as 1D-CRNN-v1 hereafter in this chapter.

The architecture and hyper-parameter settings are listed in Table 6.7. ReLU, hyperbolic tangent (Tanh) and softmax activation functions are used in the convolutional & FC layers, LSTM recurrent cells, and output layer respectively. A dropout strategy is considered for regularization and dropout fraction values of 0.2, 0.1 and 0.1 for C1, R1 and FC are set respectively.

.2. 2D-CNN-RNN model for MSTS images based crop classification

The 2D-CNN architecture is used here to extract spatial features as the temporal features are capture by the RNN architecture. Since, 2D-CNNs can accommodate spatial contextual information, patch based approach is considered here. The kernel size of the filter in the convolutional layer i.e. C1 layer is a 33  window. The 2D-CNN architecture is employed in a time-distributed fashion in order to preserve the temporal information. Then, a conventional RNN model is considered which is identical to the RNN model used in 1D-CRNN-v1. This consolidate CNN-RNN model is termed as 2D-CRNN-v1 hereafter in this chapter. The architecture and hyper-parameter settings are listed in Table 6.7.

3D-CNN-RNN model for MSTS images based crop classification

The 3D-CNN architecture is used here to extract spectral-spatial features as the temporal features are capture by the RNN architecture. Since, 3D-CNNs can also accommodate spatial contextual information, patch based approach is considered here. The kernel size of the filter in the convolutional layer i.e. C1 layer is a 3 3 12  window. The 3D-CNN architecture is employed in a time-distributed fashion in order to preserve the temporal information.

After the convolutional layer, a conventional RNN model is considered similar to the RNN models used in 1D-CRNN-v1 and 2D-CRNN-v1. This consolidate CNN-RNN model is termed as 3D-CRNN-v1 hereafter in this chapter. The architecture and hyper-parameter settings are listed in Table 6.7.

The Concept of Permuted Local Spectral Convolutions and its Synergistic Use with RNNs for MSTS images based Crop Classification

The concept of permuted local spectral convolution is introduced and discussed in chapter 5

(refer section 5.5). Chapter 5 also establishes the significance and advantage of permuted local spectral convolutions over the conventional spectral convolutions or local spectral convolutions. In this chapter, the concept is analysed in presence of two rather one complementary information i.e. spatial and temporal. Usually, the DNN models developed for high dimensional time series image data, focus on temporal and/or spatial information more than the spectral information present. However, in context to satellite based earth observation images, the spectral information is as important as the spatial and temporal or even more.

Therefore, the idea of utilizing the spectral information more efficiently along with the spatial and temporal information motivates to development of novel DNN models.

The Perm-1D-CNN-v1 architecture and hyper-parameter settings are listed in Table 6. The learning rate and the number of epochs are set to 0.01 and 200 respectively.

2D-CNN model with permuted input for MSTS images based crop classification

Since the input MSTS image data has 1 spectral, 2 spatial and 1 temporal dimension, it is not possible for 2D-CNNs to process this 4-dimensional data simultaneously and directly.

Therefore, the permuted local spectral convolution version of the 2D-CNN model is not explored further.

3D-CNN model with permuted local spectral convolution for MSTS images based crop classification

The 3D-CNN model proposed here namely Perm-3D-CNN-v1 operates window based convolution along the spectral and spatial dimensions in a time distributed fashion.

The first layer is a permutation layer similar to the Perm-1D-CNN-v1 with the difference in the spatial dimension. The Perm-3D-CNN-v1 uses a 33  window in the spatial dimensions input in contrast to Perm-1D-CNN-v1 which uses per-pixel approach. After the ensembled input is generated, a series of three dimensional convolutions are employed in a time distributed fashion. The three dimensional convolutions operate over the spectral and spatial dimensions.

RNN model with permuted input for MSTS images based crop classification

Permuted RNN model is similar to the conventional RNN model (RNN-v1) discussed in section 6.5.4 apart from the addition of a permutation layer. The permutation layer prepares an ensemble of permuted spectral bands similar to as in Perm-1D-CNN-v1 or Perm-3D-CNN-v1., Therefore, the input to this RNN model namely Perm-RNN-v1 is a 7 120  matrix where 7 indicates the length of time dimension (time stamps) and 120 indicates to the permuted and ensembled spectral depth in data in each time stamp. Since, RNNs do not accommodate spatial contextual information, per-pixel approach is considered here. A simple LSTM unit is selected as recurrent cell. Similar to RNN-v1, seven LSTM units constitute the R1 recurrent layer and the architecture is used in a many-to-many fashion. It means the input to the recurrent layer is more than one dimensional (in this case, 7) and output from the recurrent layer is more than one dimensional (in this case, 7). Also, the flow of activation from each recurrent cell in the recurrent layer is considered in a bidirectional manner. It means that the activation from particular recurrent cell can move to next recurrent cell (next time stamp, usual) and previous recurrent cell (previous time stamp). This helps in the crop classification and the significance is discussed in the results and discussion section later in this chapter. Further a FC layer with 20 nodes and an output layer with 6 outputs is set. The Perm-RNN-v1 architecture are listed in Table 6.9. The hyper-parameter settings are similar to RNN-v1 are also listed in Table 6.9.

Perm-RNN-v1

Input data size = Input data size =

Results and Discussion

A. Quantitative analysis

The results from the models presented in section 6.5, and 6.7 are discussed in this section. Both, qualitative and quantitative assessment are carried out. Overall, results from 12 different models are compared and discussed here. These are 1D-CNN-v1, 3D-CNN-v1, RNN-v1, 1D-CRNN-v1, 2D-CRNN-v1, 3D-CRNN-v1, Perm-1D-CNN-v1, Perm-3D-CNN-v1, Perm-RNN-v1, Perm-1D-CRNN-v1, Perm-2D-CRNN-v1, and Perm-3D-CRNN-v1. Figure 6.10 show classification results obtained using these 12 models. Also, Table 6.11 shows quantitative assessment of the classification performance of these 12 models. It is evident from

CONCLUSIONS AND FUTURE SCOPE

The main objective of this thesis is the study and critical analysis of the performance of conventional CNN and RNN models and further development of the novel CNN and RNN models to exploit the spectral and/or spatial and/or temporal information present in unimodal/single senor or multimodal/multisensor satellite images. Special focus during the development of novel DNN models is given to develop models showing better generalization performance in mixed class scenarios.

Contributions of the thesis

In the attempt, four tasks are set to forward the stated objective. The tasks are based on; type(s) of satellite image(s) used, application for which the task is realized, multi-modality images involvement, and presence of time series information. The tasks are satisfactorily completed and are presented in four chapters i.e. chapter 3, 4, 5, and 6.

In