Laura Garcia
email: laugarg2@teleco.upv.es

Lorena Parra

Daniel A Basterrechea

Jose M Jimenez

Javier Rocher

Mar Parra

José Luis García-Navas

Sandra Sendra
email: ssendra@ugr.es

Jaime Lloret
email: jlloret@dcom.upv.es

Pascal Lorenz
email: lorenz@ieee.org

Jesus Tomás
email: jtomas@upv.es

Abdelhafid Abouaissa
email: abdelhafid.abouaissa@uha.fr

Miguel Rodilla
email: mrodilla@hma.upv.es

Silvia Falco
email: sfalcog@hma.upv.es

María Teresa Sebastiá
email: mtsebastia@hma.upv.es

Jesus Mengual

Juan Andrés González

Bernat Roig
email: bernat@upv.es

Quantifying the Production of Fruit-Bearing Trees Using Image Processing Techniques

Keywords: Persimmon, fruit, image processing, RGB bands, histogram. I

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

Agriculture is the main way of obtaining food nowadays. More or less, the 95% of the consumed food is produced directly or indirectly in the soil. This consume is so large that the Food and Agriculture Organization (FAO) had made a project of world crops demand between the years 2015-2030. FAO concluded that is expected by 2030, the demand for life consumption will be large and it needs more agriculture production, expecting that last year an increase of 67 million people. In recent years, the growth rate of agriculture production and crop yield have declined. This has raised fears that the world will not be able to sufficiently increase the production of food and other products to ensure adequate food for the future population [START_REF] Bruinsma | World agriculture: towards 2015/2030[END_REF].

The decline in the production can be explained by a lot of causes like intensive agriculture, which causes degradation in the soil, losing most of the nutrients in this process [START_REF] Tsiafouli | Intensive agriculture reduces soil biodiversity across Europe[END_REF]. Among these nutrients are: Magnesium (Mg), Potassium (K), Phosphorus (P), Nitrogen (N), Sulfur (S), Calcium (Ca) and Iron (Fe), where nitrogen plays an important role, maintaining an optimal ratio with C:N which allows bacteria to provide nutrients to crops [START_REF] Satoh | Irrigated Agriculture in Egypt: Past, present and future[END_REF]. Other causes could be the abrupt changes produced by time due to climate. These sudden changes can cause crop losses in the harvest, due to the rise in temperature, hailstorm, scarcity in water, etc. [START_REF] Simon | Nitrogen nutrition of beech forests in a changing climate: importance of plant-soil-microbe water, carbon, and nitrogen interactions[END_REF]. This problem intensifies in countries like Egypt where the weather is very dry and the rains are scarce [START_REF] Ali | Climate Change and Its Impact on the Yield of Major Food Crops: Evidence from Pakistan[END_REF]. Because of this, water is a precious resource and implementations are made to improve crop irrigation.

One of the agricultural crops in this area is persimmon, which needs low humidity to obtain an optimal crop. This makes Egypt a place with suitable conditions for the cultivation of this fruit. Due to this factor, the monitoring of the amount of fruit in the tree has been necessary to be able to control the evolution of the crop and regulate irrigation [START_REF] Llácer | Persimmon Production and Market[END_REF]. By monitoring the quantity and quality of the yields of this fruit, it can be regulated the amount of water needed for each tree in each moment, with the advantage of doing the best use of the country's water resources. On the other hand, this agricultural product has caused a lot of controversy among farmers and buyers of this fruit. The technique applied in this paper will help to reduce this problem by providing data of great relevance and improving the capacity of the area to increase the production.

In this paper, we propose a system will consist of the treatment techniques of images, with which to be able to quantify the amount of fruit (in this case of persimmons) relating the pixels to the weight, of each tree and in this way to be able to obtain reliable data of the harvest, being able to take control over these. In this article, a first essay has been carried out, in which the images have been taken with a mobile device. In the future, this method will be carried out by using a drone, allowing measurements of a larger area.

The rest of the paper is structured as follows. Section II presents related works about the techniques of processing images and methods for monitoring irrigation water by other authors. A proposal of the technic is going to be proposed in Section III. Moreover, Section IV shows the results of the used method. Finally, Section V presents the conclusion and future works II.

RELATED WORK

In this section, we are going to compare other systems that used image processing techniques in farming. The image processing is an important issue for agriculture. Therefore, many scientists have been work on their identification based on pictures.

Other techniques are also analyzed, which is of great importance for agricultural development.

Parra et al. [START_REF] Parra | The Use of Image Processing Techniques for Detection of Weed in Lawns[END_REF] showed the use of image processing techniques to detect the presence of undesired grass species. They utilize a drone with an Arduino module to take pictures. The obtained images are used to determine the best option to detect the presence of weeds. Their results pointed out that the combination of different layers of a single image can be used for detecting the presence of undesired plants.

Ulzii-OrshikhDorj et al. [START_REF] Ulzii-Orshikh | An yield estimation in citrus orchards via fruit detection and counting using image processing[END_REF] proved the utility of a computer vision algorithm to detect and count citrus on the tree using processing techniques. Their system was able to estimate the yield and comparate the yield estimation results obtained through several methods. They collected 84 images from 21 trees.

Marín et al. [START_REF] Mello | System and method for monitoring crops[END_REF] showed how the Information and Communication Technology (ICT) offers the possibility of monitoring the grass state to adjust the irrigation regime. They used a drone to take pictures with different vegetation coverage and can obtain results processing this information.

Mello [START_REF] Marín | Urban Lawn Monitoring in Smart City Environments[END_REF] created a system, which includes an aerial platform including a spectral imaging device, a position sensor, an orientation sensor and includes a ground-based sensor platform including at least one soil sensor.

Hutchinson et al. [START_REF] Hutchinson | Monitoring vegetation change and dynamics on U.S Army training lands using satellite image time series analysis[END_REF] presented a technique for monitoring vegetation changes and dynamics using satellite image time series analysis. The BFAST time series decomposition method was applied, to a ten-year MODIS NDVI time series dataset for the Fort Riley military installation and Konza Prairie Biological Station (KPBS) in northeastern Kansas. Their results had shown that the generated indicators are useful for different uses in land management.

In other cases, different sensors are used for monitoring the requirements of the plants, including irrigation needs. In [START_REF] Achouak | Sensor Network Proposal for Greenhouse Automation placed at the South of Algeria[END_REF], Achouak et al. showed the use of a wireless network composed of sensors and actuators for controlling the requirements of crops in a greenhouse. Finally, Rocher et al.in [START_REF] Rocher | A New Conductivity Sensor for Monitoring the Fertigation in Smart Irrigation Systems[END_REF] presented a sensor for monitoring the fertigation in smart irrigation systems for fruit-bearing trees which can be used jointly with our system.

Our technique has advantages over the others, (i) it is easier to apply only being necessary a front photo of the tree, and (ii) we use a combination of the RGB bands (Red, Green, and Blue). Thus, we obtain an image in which the persimmons are observed correctly and it is possible to apply different equations to predict the harvest quantity.

III. DESCRIPTION OF THE PROPOSAL

In this section, we are going to present the description of the system that we have used, through which we have achieved results that support our initial proposal, as shown in Section IV.

As this is a preliminary test, to minimize the use of resources, the pictures will be taken in the horizontal plane using a camera. In the future, the pictures will be taken in the vertical plane with a drone. To carry out our proposal it has been necessary to use a "curtain" of an opaque material. It allows us to hide the trees behind the tree to be studied. Furthermore, a mobile device has been used from which the images are obtained, which will then be treated.

As we take pictures in different fields and the distances between trees might are not always the same, the distance between the camera and the trees is not constant in all the cases. To solve this problem, we propose to use a reference object. Using this reference object we will be able to correct the distance effect, which may appear when acquiring the images. Figure 1 shows how a tree image is captured. Our experiment is based on the fact that these fruit trees, when the fruit is collected, has lost its leaves. Due to this phenomenon, it is possible to recognize and differentiate the fruits from the rest of the objects that may appear in the images. Therefore the possible effects of overlapping structures are minimized. The branches are the sole overlapping structure and, since they are thinner than the fruits, their possible influence in the prediction model can be despised. Moreover, as the trees are similar in shape and height, the possible underestimation is a constant value. Therefore, it will not affect our prediction model.

As shown in Section IV, once the images were obtained, they have been processed. Decomposing them in the basic colors R, G and B layers (or rasters). The RGB images are to be combined using mathematical operations to obtain new rasters. Then, the histograms of these new rasters are obtained. In this way, it will be possible to detect the region of the histogram that represents the fruit and does not represent the soil.

Curtain to cover the trees behind

Persimmon fruit

Persimmon tree

Reference object

Mobile device

Subsequently, the histogram is normalized by the surface of the reference object. Finally, the histograms of the difference images are compared with the real kilograms of fruit that were harvested from the tree. A regression model is used to relate the histogram and fruit production.

The data obtained can be used by the farmer as a tool, because he can estimate the harvest he is going to obtain. In addition, as it is already stated in Section 1, this system allows the irrigation to be adjusted according to the observations made.

IV. RESULTS

In this section, we show the obtained pictures and the processing techniques used to quantify the fruit production. First, the combination of bands to find the best composition for fruit detection is presented. Following the correction of the effect of distance is described. Finally, we show the correlation between the number of pixels and production.

A. Evaluation of best band combination

The first step is to evaluate which band or bands of the image can be used for fruit detection. The coloration of Persimmon in the moment of yield is between orange and red, therefore we expect to find high pixel-values in the band of red, intermediate values in the green band and low values in the blue band. In Figure 2, the RGB composition and the representation of single bands are presented. The representation of a single band is shown as a color ramp from black (when the pixel value is low) to white (when the pixel value is high). As expected, the fruits have been clearly identified in the red band. Nonetheless, the color of clouds and soil is similar to the color of fruits. The spectral signature is a well-known term, used in remote sensing, which is used to identify different surfaces by comparing the data of two or more bands. It has been used to identify fruits [START_REF] Herold | 3.2 Monitoring and Mapping of Fresh Fruits and Vegetables Using Vis Spectroscopy[END_REF].

Therefore, it is necessary to combine the red band with a second band (green or blue). In order to evaluate which is the best band composition, it is necessary to represent and compare the different options. In this paper, and with the purpose of maintaining the system operation as simple as possible, we are going to evaluate the combination of only two bands and using the mathematical operators of addition, subtraction, multiplication, and division.

Figure 3 summarizes the different combinations of R and G or B band in greyscale. The fruits can be distinguished in five out of ten combinations. From those combinations where fruits can be distinguished, in two of them the differences between fruits and soil are higher (R -B and R -G). The highest difference between value of fruit pixels and soil pixels is observed in R -G combination. Moreover, in Figure 3, we can observe the effect of the curtain, and how the curtain is more effective in some band combinations (R+G or R-G) than in others (R/G or G/R).

B. Correction of the distance effect

Following, it is necessary to consider the effect of distance from the camera to the tree. Since our units of surface are the pixel and the distance from the camera to the object affects the relation of surface in the picture (pixel 2 ) to surface in reality (cm 2 ), the effect of different distances must be included in our method. During the picture collection, a reference was included in all the pictures. The reference was attached to the trunk of the tree.

Then, the surface of the reference object in pixels is calculated. The size of the reference in each picture is shown in Table 1. The value in pixels of the reference object will be used to normalize the values of the histogram. Other relevant data for future subsections, such as code of crop field and fruit weight are included. The field number is used as picture ID and the weight is used for the correlation of histograms and fruit production.

RGB composition

Red Band Green Band Blue Band

C. Analysis of histogram data

The next step is to obtain the histogram of the picture, it can be done using different options. In our case, we use the code described in [START_REF] Marín | Urban Lawn Monitoring in Smart City Environments[END_REF] to obtain the data of the histogram using Matlab. Then the obtained histogram is represented, see Figure 4. In this graphic, the comparison of different histograms can be observed. The histogram represents the number of pixels in the image that have a certain value. In the original raster (RGB rasters) the value of pixels is between 0 and 255. Besides, in the newly generated raster, the pixels can have values from -255 to 255. In Figure 4, the different pictures are numbered according to the fruit weight in kg. Before analyzing the differences in the histograms of different pictures, it is necessary to normalize the data. Therefore, we are going to consider the surface of the reference object as 1UA (1 unit of area). Thus, the results of the histograms can be comparable even if the pictures have different sizes and are taken at different distances. The corrected histograms are presented in Figure 5. It is possible to see that the histogram shape has changed. In Figures 4 and5, the different pixel values represent different surfaces. In our case, the persimmon, according to Figure 3, will be represented by the highest values. Therefore, the last part of the histograms is where the data about the fruit is contained. As different parts of the fruit might have different colorations, which correspond to different pixel values, several pixel-values will be used in the correlation. First of all, we check the values of pixels in the generated rasters. The observed values of pixels in persimmon fruits go from 72to 171. The representation of histograms in this range is displayed in Figure 6. In this graphic, the images of trees with more fruits have higher values in the Count (UA). The distributions of values along the range (72 to 171) is related to the fruit coloration. In images 4 (47 kg), 2, (35 kg) and 3 (20 kg) the peak of pixels associated with fruit is found around 73 to 75 pixel-value. Meanwhile, in other images, this peak is found at 102 pixel-value. This is caused by the differences in fruit maturity or fruit quality. The fruits of the images with a peak located at the highest pixel-value have a better coloration (uniform red color).

R + G R -G R x G R / G G / R R + B R -B R x B R / B B / R
Finally, the correlation between the summation of pixels with pixel values between 72 and 171 and fruit production is presented in Figure 7. The mathematical model can be expressed as shows the Eq. ( 1). The correlation coefficient of the mathematical model is 0.97. The correlation coefficient offers information about the goodness of the correlation. It can take a value between 0 and 1, the closer to 1 the better the model. V.

CONCLUSION

We propose a system to quantify the amount of fruit, based on image processing techniques.

We use pictures of the trees and data of their harvested fruit in kg. We decompose the original RGB image into single bands colors. The fruit of Persimmon usually is orange or red. The fruits have been clearly identified in the red band. But, it is necessary to combine the red band with a second band (green or blue), to differentiate the fruits to the color of the soil. The highest difference is observed in the R -G combination.

For our system, it is necessary to consider the effect of distance from the camera to the tree. We include a reference object in the original picture collection to normalize the data of the histograms. Finally, we used the normalized histograms for the correlation with fruit production.

For future work, we will perform a new study using a drone to capture the pictures. We expect that taking the images from the drone will be easier to carry out the process of identifying the fruit. Therefore we can better adjust the regression model for estimating productivity. Moreover, we will evaluate the option of using a camera that allows having a thermal image. The use of non-visible bands, such as infrared or ultraviolet, can increase the performance of our model.

Figure 1 .

 1 Figure 1. Scenario description

Figure 2 .

 2 Figure 2. System description.

Figure 3 .

 3 Figure 3. Combination fo R band with G or B band to maximize the fruit detection

Figure 4 .Figure 5 .

 45 Figure 4. Histogram of studied images

Figure 6 .

 6 Figure 6. Detail of the corrected histograms

Figure 7 .

 7 Figure 7. Correlation between the summation of pixels and fruit weight

TABLE I .

 I REFERENCE OBJECT IN THE USED PICTURES

	Image nº	1	2	3	4	5
	Reference surface (pixels 2 ) 17192 6299 8771 6299 8273
	Field number	2446 2696 3230 1535 1541
	Fruit weight (kg)	37	35	20	47	43

Copyright (c) IARIA, 2019. ISBN: 978-1-61208-758-0 INNOV 2019 : The Eighth International Conference on Communications, Computation, Networks and Technologies

Copyright (c) IARIA, 2019. ISBN: 978-1-61208-758-0

ACKNOWLEDGMENT

This work has been partially supported by European Union through the ERANETMED (Euromediterranean Cooperation through ERANET joint activities and beyond) project ERANETMED3-227 SMARTWATIR by the Conselleria de Educación, Cultura y Deporte with the Subvenciones para la contratación de personal investigador en fase postdoctoral, grant number APOSTD/2019/04, and by the Cooperativa Agrícola "Sant Bernat" Coop.V.