A fully automatized processing chain for high-resolution multispectral image acquisition of crop parcels by UAV

Keywords: Near infrared image, mosaicking, ground control point

In the context of the RHEA European project addressing site-specific weeding on arable crops, a complete aerial image acquisition chain has been developed, providing a georeferenced multispectral image of a total parcel with 1 cm spatial resolution. Robust automatic ground control point detection and near infrared -visible sensor image registration modules have been implemented and associated with the MicMac photogrammetric software suite to provide a fully automatized mosaicking procedure. This procedure has been successfully applied on various maize and wheat crops for weed patch detection, but could be extended to other precision agriculture applications, especially those requiring a very high spatial resolution.

Introduction

Unmanned aerial vehicles (UAV) have emerged as an alternative to aircraft and satellite remote sensing for agricultural monitoring, especially in the context of precision agriculture [START_REF] Zhang | The Application of Small Unmanned Aerial Systems for Precision Agriculture: A Review[END_REF]. Indeed, UAV provide significant advantages: short revisit time due to their great flexibility in flight scheduling, image usability in cloudy conditions, and very high spatial resolution thanks to low altitude flight capabilities. As a counterpart, UAV operation has important limitations in terms of flight duration, positioning accuracy, available payload and power source for imaging devices. As a consequence, most embedded cameras are limited to RGB image acquisition [START_REF] Shahbazi | Recent applications of unmanned aerial imagery in natural resource management[END_REF]. Although some commercial multispectral devices are available (e.g. Tetracam, Chatsworth, CA, USA), they usually do not allow a spatial resolution better than 10 cm [START_REF] Xiang | Development of a low-cost agricultural remote sensing system based on an autonomous unmanned aerial vehicle (UAV)[END_REF][START_REF] García-Ruiz | Comparison of two aerial imaging platforms for identification of Huanglongbing-infected citrus trees[END_REF]. In this context, the European FP7 project RHEA (2010)(2011)(2012)(2013)(2014), devoted to site specific weeding management in arable crops, had to face a cutting-edge challenge. One of the tasks to achieve was to perform an aerial inspection of the targeted crop parcels in order to locate weed patches for further weeding operation. Due to the poor feasibility of spectral approaches for weed detection [START_REF] Slaughter | Autonomous robotic weed control systems: A review[END_REF], an approach based on crop row detection has been chosen [START_REF] Gée | Crop/Weed discrimination in perspective agronomic images[END_REF]. However, in the case of winter wheat where the inter-row is about 15 cm, crop row detection requires a centimetric spatial resolution [START_REF] Torres-Sánchez | Multitemporal weed mapping using UAV imagery for early site-specific control: the case of wheat as a narrow row crop[END_REF]. Moreover, a multispectral image including a near infrared band (NIR) is required for robust discrimination between vegetation and soil in any lighting conditions [START_REF] Peña | Comparing visible and color-infrared UAV imagery for early-season weed mapping: the case of maize as a wide row crop[END_REF]. A UAV vision system made up of two compact still cameras has been designed to meet both these requirements, i.e. high spatial resolution and multispectral acquisition. It provides color and NIR images with a spatial resolution of 1 cm at 60 m altitude. As a counterpart, numerous image acquisitions are necessary to cover the whole parcel. The software processing chain which consists in merging images is known as mosaicking. Many commercial or open source mosaicking software solutions are now available. However, they usually lead to a loss of spatial accuracy in the resulting image [START_REF] Zhang | The Application of Small Unmanned Aerial Systems for Precision Agriculture: A Review[END_REF]). In the present case, in order to preserve as much as possible the initial image accuracy, open source software MicMac (MicMac, IGN, France) has been chosen. This software offers a high degree of flexibility in terms of computation algorithms, distortion model definition, etc. In counterpart, it needs the management of many input parameters and requires high technical knowledge in photogrammetry. Another important challenge in the RHEA project was thus to automatize the image mosaicking process as much as possible, the objective being to launch and execute the whole processing chain from a single order issued from the RHEA supervising system. In the following, the main steps of the whole processing chain are presented.

General description of the mosaicking process

Mosaicking principle

Mosaicking consists of merging several overlapping images into one unique larger image covering the whole scene under study. For this purpose, source images have to be ortho-rectified, i.e. spatially transformed so that they appear as images taken from a zenithal camera with an infinite focal length. If the scene was perfectly flat (assuming no camera lens distortion), ortho-rectification would be a simple geometrical transformation. However, in usual conditions, ortho-rectification requires knowledge of the scene relief, and thus a kind of stereo reconstruction. Therefore, mosaicking of aerial images is made up of several steps:

-First, sets of matching points between various source images are searched for using a feature point extraction algorithm such as the Scale Invariant Feature Transform or SIFT [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF]. The 3D positions (x,y,z) of matching points are then determined as well as the 3D position and orientation of the camera for each image acquisition, using a global optimization technique. This results in a "sparse cloud" model, as illustrated in Figure 1a. -Second, the 3D position of any point in the scene has to be recovered. Because the camera positions are known, the point matching can be made by looking for maximal local correlation between images along epipolar lines. This second step results in a digital surface model (DSM) of the scene (Figure 1b). It requires that every point in the scene is viewed at least in two source images, and thus an overlapping of at least 50% in one direction during image acquisition. -Once the DSM is computed, the grey level of each 3D point in the scene can be retrieved in the relevant source images, and the ortho-rectified images can be built and merged. These various steps make up the bulk of the Micmac software suite.

Geo-referencing

The sparse cloud and DSM described above are first expressed in relative coordinates, the length unit being the pixel camera size. In order to re-scale them according to the real world coordinates, a set of ground control points (GCP) must be available (or precise IMU and GPS data, which are unavailable on small UAV hardware). It consists in some particular points in the scene, for which the 3D coordinates are known as well as the pixel coordinates in relevant images. Usually, GCP are made of artificial targets or natural points (cross-roads, trees, etc.) whose position has been previously determined, and that are visually retrieved in the source images. In the RHEA project, artificial color targets have been settled around the field and automatically retrieved by image processing, as explained in the next section.

Multispectral management

Basically, the mosaicking process must be applied on grey level source images. In the usual case, source images are color images captured from a single camera, and thus a luminance image is computed as a combination of the RGB channels for mosaicking computations (feature point and correlation matching). The color information is recovered at the last step, i.e. during source image ortho-rectification and merging. In the RHEA case, images are captured from two cameras, for which acquisition positions are slightly different. Therefore, two management possibilities can be considered:

Author-produced version of the article published in Precision Agriculture, 2015, p.135-141 The original publication is available at http://www.wageningenacademic.com Doi: 10.3920/978-90-8686-814-8_16 -a) Two different color and NIR mosaic images are built separately, and then superimposed based on their geo-referencing. This solution just requires detecting automatically the GCP in both kinds of images. -b) Original images from the two cameras are registered at each acquisition. A single mosaicking process is then launched using the color images, and NIR information is recovered at the last step of mosaic image building. The second solution appears much more complicated, mainly because it requires an accurate registration between different kinds of images (color and NIR), which is not straight forward (see next section). Nevertheless, this solution has been preferred, because it preserves the spectral content of the mosaic image to be affected by mosaicking accuracy (i.e. by positioning differences between color and NIR bands).

Mosaicking process automatization

Image acquisition devices The image acquisition system has been built around two compact still cameras DP2 Merrill (Sigma, Kawasaki, Japan), embedded on a UAV AR 200 (AirRobot® GmbH, Arnsberg, Germany). These are high-end commercial cameras, with a 3 layer Foveon® CCD sensor of 4704x3136 pixels and 30 mm focal length. One of the cameras was modified for NIR acquisition (internal NIR blocking filter removed and replaced by an external NIR band pass filter). Both cameras have also been modified to allow image shooting through an external signal provided by the UAV control unit (Figure 2). With a flight altitude of 60 m, images cover a ground area of about 40x30 m, with 1 cm per pixel. Before each aerial mission, a flight program specifying UAV position for every image acquisition is established accordingly, so that the crop parcel is entirely covered with an image overlapping of 60% in both directions. After the flight, a software module compares this log file with the time data included in Exif metadata of each image (Exchangeable image file format), to recover its position and orientation. This allows roughly prepositioning each image in a local geographic system for further use (Figure 3).

GCP automatic detection and labelling Specific targets were designed for automatic detection as GCP. They consist of 30 cm wood disks with two colors: blue for easy recognition in crop field environment and grey for lighting correction purpose (not used yet). These targets were roughly distributed around the field before the aerial mission, and their 3D position collected using a Real Time Kinematic (RTK) GPS receiver (e.g. Leica RTK GPS, Leica Geosystems®, Heerbrugg, Switzerland). Typically 8 to 10 targets are required, and a specific number label is associated to each one during RTK measurement. At the beginning of the mosaicking process, GCP are systematically searched in every color image and their pixel coordinates stored. The detection process starts with an image binarization using a color look up table to select blue pixels. Resulting binary objects are then checked according to minimal and maximal area values. Finally, an elliptical model is fitted to their external contour [START_REF] Fitzgibbon | Direct Least Square Fitting of Ellipses[END_REF]. Objects with a ratio between major and minor ellipse radius lower than a given threshold (typically 1.2) are retained, and the ellipse center is stored in floating point pixel coordinates. Once all targets in all images have been detected, their pixel position is reported in the image prepositioning representation (Figure 3), as well as RTK positioning data. This allows recovering their number label: the label of the nearest RTK position data is chosen, provided the second nearest is at least twice the distance (otherwise the detected target is declared ambiguous and not retained).

Registration of color and NIR images

Due to the very different nature of NIR and color images in a vegetation context, a registration approach based on the Fourier-Mellin transform has been developed. First, images are partitioned in small sub-images on which translation, rotation and scaling between color and NIR bands are identified using Fourier spectrum analysis. Finally, a global homographic transformation between both images is determined. Radial distortion of both cameras is taken into account as well. More details can be found in [START_REF] Rabatel | Registration of visible and near infrared aerial images based on Fourier-Mellin Transform[END_REF]. Fourier computations were implemented on the computer Graphics Processing Unit (GPU). An example of image registration is given in Figure 4.

Global processing chain summary Figure 5 summarizes the flow chart of the global processing chain. Inputs consist of two text files and both camera SD card contents. Internal software modules have been implemented as independent command line executables managed by a supervisor module, as well as MicMac command line executables. Some complementary modules (such as Sigma raw image conversion in TIFF format) are not represented.

  reconstruction during mosaicking process. a) sparse cloud. b) DSM. Green points in a) correspond to camera positions. White spots in b) correspond to trees.

Figure 2 .

 2 Figure 2. Two Sigma DP2 Merril cameras mounted on UAV

  Figure 3 tail of a colo an

  Author-produced version of the article published in Precision Agriculture, 2015, p.135-141 The original publication is available at http://www.wageningenacademic.com Doi: 10.3920/978-90-8686-814-8_16

	Discu ussion and	conclusion n
	This	processing		chain has b been succes ssfully appli ied to vario us aerial m missions on w wheat
	and m maize crop	s in the con ntext of the e RHEA pr roject, in D ecember 20 013 (Montp pellier
	area, , France, 4 43.6469°N, , 3.8827°E E), January y and Apri il 2014 (A Arganda, S Spain,
	40.31 160°N , 3.4 4865°W). T Target detec ction and co olor-NIR re egistration h had proven	to be
	partic cularly robu ust. Though h variable t translations s between 1 100 and to	300 pixels	were
	obser rved betwee en color an d NIR imag ges, mainly	due to rand dom shootin ng latency o of the
	came eras, residua al registratio on error rem mained belo ow 0.5 pixel ls in every c case.
	As an n example,			for a set of f 39 images s covering a approximate ely 1 ha (12 20mx85 m), , total
	proce essing time e was about t 3h on a P	C compute r equipped	with a 4-co ore i7-480 00MQ
	proce essor and a a NVIDIA	Quadro L6 610M GPU U (2.5 hour rs for MicM Mac mosai cking
	proce ess, 30 min			for image T TIFF conve ersion and c olor-NIR re egistration). .
	As	a conclusi ion, this a automatized d processin ng chain c could brin ng a signif ficant
	contr ribution to			the promot tion of UAV V aerial im magery in v arious prec cision agricu ulture
	appli ications, by y facilitatin ng the acq quisition pr rocess. Th e only rem maining m manual
	opera ations conc cern the GC CP distributi ion and pos sition measu urement in	the field (w which
	could d be avoide ed by using	fixed targe ets in some	cases), and d the SD ca ard transfer	from
	the c cameras to			the compu ter. Next im mprovemen nts concern	the manag gement of i image
	acqu uisition with h automatic c exposure	(which re equires tuni ing image	conversion	gain
	accor rding to Ex xif data) an d reflectanc ce correctio on using GC CP as refer rence to imp prove
	the ra adiometric	quality of th he resulting g mosaic im mage.
					Fig gure 5: Glob bal processi ing flow ch art.
	Ackn nowledgem ments
	The	research lea ading to the ese results	has receive ed funding f from the Eu uropean Un nion's
	Seve enth Framew work Progra am [FP7/200 07-2013] un nder grant a agreement n n°245986.

Author-produced version of the article published in Precision Agriculture, 2015, p.135-141 The original publication is available at http://www.wageningenacademic.com Doi: 10.3920/978-90-8686-814-8_16