Romuald Jolivot
email: romuald.jolivot@u-bourgogne.fr

Pierre Vabres
email: pierre.vabres@u-bourgogne.frfranck.marzani@u-bourgogne.fr

Franck S Marzani

Reconstruction of hyperspectral

Keywords: Imaging system, Multispectral images, Hyperspectral cube reconstruction, Neural networks, Spectral reflectance, Skin cancer

come

Introduction:

Early melanoma detection has gained major attention for the development of non-invasive imaging techniques [START_REF] Esmaeili | Imaging Techniques for the In Vivo Diagnosis of Melanoma[END_REF] [2] and computer-based solution [START_REF] Marchesini | Optical Imaging and Automated Melanoma Detection: Questions and Answers[END_REF] [START_REF] Voigt | Computer Vision and Digital Imaging Technology in Melanoma Detection[END_REF]. Those efforts try to bring objective measurement of skin lesions. The use of skin colour in dermatological diagnosis relies mainly on direct visual analysis and/or RGB (red, green, blue) colour photography. This approach has limits due to the poor colour discrimination of the human eye and brain, and their inability for comparisons at different times. In order to overcome such limitations, measurement of optical reflectance from skin lesions can provide valuable spectral information, potentially useful for diagnosis. This can be achieved by combining advantages of spectrophotometers (spectral resolution) and digital cameras (spatial resolution), in MultiSpectral (MSI) and HyperSpectral (HSI) Imaging systems. MSI and HSI systems have the capacity to acquire images at different short spectral bands, including wavelengths which the human eye is unable to capture. Therefore, it is possible to extract extra information about light-matter interaction that is usually concealed within RGB images. Some skin diseases appear to exhibit specific reflectance properties. For instance, according to Tomatis et al. [START_REF] Tomatis | Automated melanoma detection with a novel multispectral imaging system: Results of a prospective study[END_REF], cutaneous melanoma may have different variegation above specific wavelength values as compared to nonmelanoma lesions. Currently, the use of MSI devices in dermatology is limited to few devices [START_REF] Tomatis | Automated melanoma detection with a novel multispectral imaging system: Results of a prospective study[END_REF] [6] [START_REF] Gutkowicz-Krusin | Precision of automatic measurements of pigmented skin lesions parameters with a melafind multispectral digital dermoscope[END_REF]. A new category, consisting of HSI systems is currently under investigation in several research groups, in microscopy [START_REF] Ornberg | Analysis of stained objects in histological sections by spectral imaging and differential absorption[END_REF] [START_REF] Rothmann | Spectral imaging for quantitative histology and cytogenetics[END_REF] and for in-vivo optical diagnosis [START_REF] Vo-Dinh | A hyperspectral imaging system for in vivo optical diagnosis[END_REF] but limitations of HSI are generally their complexity and cost [START_REF] Balas | A novel hyper-spectral imaging system: Application on in vivo detection and grading of cervical precancers and of pigmented skin lesions[END_REF]. MSI usually involved 4 to 16 spectral bands when HSI is able to record a much higher number of very narrow bands . HSI provides a Hyperspectral Cube (HC) which contains contiguous spectra. In contrast, multispectral images should be considered as discrete spectra.

In this contribution, the development of an integrated MSI system, composed of a CCD camera, a rotating wheel and a set of interference filters, is described. The MSI is coupled with software that Special Issue: Skin Cancer Imaging reconstructs hyperspectral cubes from acquired multispectral images. Our system is innovative since it evolves from a MSI into a HSI system without the need for an increased number of filters.

In this article, the following section presents the MSI systems. Then, the hyperspectral cube reconstruction method is detailed. Finally, preceding the discussion, results of cube reconstruction are presented.

Materials and methods:

Acquisition system of multispectral images:

The multispectral imaging system is called ASCLEPIOS standing for Analysis of Skin Characteristics by Light Emission and Processing of Images Of Spectrum. For ergonomic purpose, the acquisition system (Figure 1) is divided in two parts, a light source compartment and a hand-held device. Considering this configuration, the camera and the illuminating optics fibres, both oriented toward the skin, are enclosed in a hand-held device protecting the acquisition of external light. Thus, the skin is only illuminated with light of specific wavebands. The extremity of the hand-held device includes a nozzle which sets a constant distance of 10 cm between the skin area and the camera. For each spectral band illumination, the system acquires images 1280 x 960 pixel images coded in 16 bits using a single monochrome CCD-based camera. The monochrome camera is based on the IEEE-1394 data transferring protocol, with acquisition rate capacity of 30 frames.s -1 at full resolution. A magnifying lens is mounted in front of the camera which provides an useful area of 2.85 cm x 2.2 cm, yielding a spatial resolution of 45 pixels mm -1 . The spectral sensitivity range of the CCD image sensor matches with the set of interference filters and the light source spectral range.

After positioning the hand-held device nozzle on the skin area under study, the user presses a wireless remote control located in the handle of the hand-held device. The pressure triggers the acquisition which is performed in less than one second. The system yields seven monoband images which compose one multispectral image.

Hypersectral cube reconstruction:

The aim of hyperspectral cube reconstruction is to retrieve, from the camera signal, the reflectance spectrum in each pixel (linked to the physical property of the skin element) but also to increase the number of bands within the same spectral range. The reconstructed spectral volume of cutaneous data, also called hyperspectral cube (HC), provides a 3-dimensional volume (x,y,z) where x and y are for spatial dimensions and z for spectral dimension.

The HC is reconstructed using a neural network-based algorithm proposed by Mansouri et al [START_REF] Mansouri | Neural networks in two cascade algorithms for spectral reflectance reconstruction[END_REF]. The artificial neural networks (ANN) are composed of two steps, learning and reconstruction [START_REF] Kröse | An Introduction to Neural Networks[END_REF]. In order Special Issue: Skin Cancer Imaging to reconstruct spectra linked to the physical properties of the skin element, the proposed method takes into account a model of light propagation, used to extract the reflectance information.

Figure 2 presents the model we used with the spectral response of all the elements involved in the acquisition process. Such a model aims at separating each element of the acquisition process to retain the reflectance information r(λ) only. By using this model and by considering a linear optoelectronic transfer function, I(λ), Ф k (λ), O(λ) and α(λ) can be substituted by the spectral sensitivity S k (λ) of the k th filter. Therefore, the reflectance spectra r needs to be estimated from the known camera response and the spectral sensitivity leading to an inverse problem.

Artificial Neural Networks are modelled on the human brain and aim to mimic the function of biological neural networks. ANN is composed of neurons link to each other by synapses. These synapses have coefficients which correspond to the weight of the connection. ANN have to be configured, meaning that the weights of the synapses have to be set to produce desired output from specific input during the learning process. A self-supervised learning method is chosen to determine the correct weight of the different connections. Perceptron is a basic model of neural networks. It is a binary classifier where the output, which is function of the neuron activation, is either 0 or 1 for deterministic perceptron. This limits the generalisation capacity. Knowing that perceptrons give same response to same stimulus after training, the function that converts the activation into a response is Special Issue: Skin Cancer Imaging modified. It is changed to give probabilistic response using the Boltzmann distribution. Rather than outputting 0 or 1, this distribution delivers proportional response (restricted into [1 0]). Such perceptrons lead to the creation of associative memories (in our case heteroassociative).

Heteroassociative memories is used due to its modularity in regards to the different size of the input and output vectors. Such neural networks have a capacity of generalization, meaning that a trained network can output expected data from input it has never seen before. Such capacity improves the stability of inverse problem.

The self-supervised learning method trains the network with provided input and known output. The learning process stops when the minimum of the validation error is reached. The rule of training applies is the Delta rule known as Widrow-Hoff rule which consists of continuous modification of the input connections weight in order to reduce the difference (mean squared error delta) between an expected theoretical response and the observed one of the neuron. The learning process is performed using the GretagMacBeth ColorChecker made of 24 patches which represent spectra encountered on a daily basis and whose spectral properties are known. The learning procedure involves acquisition, by a spectrophotometer, of the p patches (with p=1,...,24), providing N (N depends on the sampling rate) values of spectral reflectance curve for each patch which composed a learning matrix R (size [NxP]). Also, the acquisition by the MSI of a multispectral image, containing K (with K=7) gray-level values for each patches is embedded in a matrix D (size [KxP]). Both set of data provides a set of corresponding pair which are used by the neural network to perform a supervised learning, using matrix D as input and matrix R as expected output. The learning stops when it reaches the minimum of the validation error. Once learned, ANN outputs a coefficient matrix Q (called synaptic coefficient matrix) of size [NxK].

The neural network employed in the hyperspectral cube reconstruction uses heteroassociative memories due to its modularity with regard to the different size of the input and output vectors. From The reconstruction of a hyperspectral cube is fast and simple as the operation is a product between the coefficient matrix and the camera response. The implemented reconstruction function is flexible and allows different sampling rate from 10 nm to 1 nm yielding reconstructed spectra of N values (ranging from 40 to 400 respectively).

Results:

As hyperspectral cubes contain spectra for each pixel; comparison should be done between the theoretical values from the spectrophotometer and the reconstructed spectrum. In our experiment, a 10 nm reconstruction sampling rate is selected in order to compare with a commercial spectrophotometer Minolta CM-2600d.

To evaluate the performance of hyperspectral cube reconstruction, two different metric scales are used, the Goodness of Fit Coefficient (GFC) and the Reconstruction Percentage (PRec).

The GFC is based on the Schwartz inequality and is calculated using the following formula: [START_REF] Ruocco | Non-invasive Imaging of Skin Tumors[END_REF] where Rm(λ j ) is the spectral value measured with the spectrophotometer at wavelength λ j and Rr(λ j ) is the reconstructed spectra value at wavelength λ j . j refers to the wavelength from 1 to N, where N represents the total number of sampled spectral reflectance values. In our system, this criterion has the advantage of being bounded between 0 and 1, providing an easy interpretation. According to

Figure 1 .

 1 Figure 1. ASCLEPIOS system descriptionThe light source compartment houses the illumination device, the rotating wheel and the set of

Figure 2 .

 2 Figure 2. Synopsis of the spectral model of the acquisition process in a multispectral system.

Figure 2 ,

 2 Figure 2, the reflectance spectrum r of each pixel is reconstructed using the camera response d and the

Acknowledgements:

The authors would like to thank the financial support provided by Conseil Régional de Bourgogne, France and Laboratoire Bioderma, Lyon France.

Special Issue: Skin Cancer Imaging Hernandez et al. [START_REF] Hernandez-Andrés | Colorimetric and spectroradiometric characteristics of narrow-field-ofview clear skylight in Granada, Spain[END_REF], the reconstruction is good if the GFC is above 0.99 and perfect if higher than 0.9999.

The reconstruction percentage is a criterion which evaluates the distance between two spectra. It requires a higher precision for the spectrum part having low amplitudes. A good reconstruction will be close to 100%. It is defined by:

(3)

The norm operator ⋅ defines a distance between two vectors. Two sets of data were used to validate the reconstruction. The first set included the 24 reconstructed patches of the GretagMacBeth chart and was compared with the value of the 24 patches acquired by the spectrophotometer. The average GFC is 0.99717086 with a standard deviation of 0.00450589 and the average PRec is 0.97975842 with a standard deviation of 0.04107166. These results reveal a good reconstruction of the learned patches.

To validate reconstruction, the set of 14 skin-tone colour patches reconstructed by ASCLEPIOS and the one recorded by the spectrophotometer were compared. The average GFC was 0.99237746 (standard deviation = 0.00647645) and the average PRec is 0.94137762 (standard deviation = 0.02454477). PREC has a lower value for skin tone patches; this might be the effect of the strong reflection from the digital MacBeth colorchecker chart in comparison to the GretagMacBeth chart used for the 24 patches. These results confirm the good reconstruction already obtained with the learned patches. They also provide evidence that the ANN using heteroassociative memories have a capacity of generalization.

Discussion:

Our ASCLEPIOS system has shown that the cube data improve spectral resolution of the system but also provide skin reflectance information in each pixel of the entire hyperspectral data cube. In Special Issue: Skin Cancer Imaging comparison with other MSI systems, the range of dermatological applications for ASCLEPIOS is wide because it provides two types of processing, spatial processing and spectral processing. Spatial processing allows image processing on the data cube as a whole or each plane at a time. The spectral approach allows signal processing on all the spectra of the data cube. The strength of the system is the capacity offered by the possible combination of spatial and spectral processing over the hyperspectral data cube. Whether this device can effectively improve clinical diagnosis and follow-up in dermatology remains to be studied.