Valentina Baldazzi

Pierre Valsesia

Michel Génard

Nadia Bertin

Organ-wide and ploidy

Keywords: cell size, division, development, expansion, endoreduplication, growth, model, ploidy, tomato

INTRODUCTION

Understanding the mechanisms underpinning fruit development from its early stages is of primary importance for biology and agronomy. Indeed, early stages are highly sensitive to biotic and abiotic stresses, with important 1 1 2 3 4 5 6 7 8 consequences on fruit set and yield. The development of a new organ is the result of coordinated events of cell division and expansion. Fruit growth starts immediately after pollination with intensive cell division. As development proceeds, the proliferative activity of cells progressively slows down giving way to a phase of pure cell enlargement during fruit growth and ripening. In many species, including tomato, the transition from cell division to expansion phases is accompanied by repeated DNA duplications without mitosis, a process called endoreduplication. The exact role of endoreduplication is still unclear. A strong correlation between cell ploidy (i.e number of DNA copies) and final cell size has been observed in different species [START_REF] Bertin | Analysis of the tomato fruit growth response to temperature and plant fruit load in relation to cell division, cell expansion and DNA endoreduplication[END_REF][START_REF] Lee | Temperature effects on systemic endoreduplication in orchid during floral development[END_REF][START_REF] Melaragno | Relationship between endopolyploidy and cell size in epidermal tissue of Arabidopsis[END_REF][START_REF] Rewers | Endoreduplication in cucumber ( Cucumis sativus ) seeds during development, after processing and storage, and during germination[END_REF], suggesting a role of endoreduplication into the control of organ growth [START_REF] Breuer | Developmental control of endocycles and cell growth in plants[END_REF][START_REF] Chevalier | Elucidating the functional role of endoreduplication in tomato fruit development[END_REF].

Understanding the way cell division, endoreduplication and expansion processes interact is crucial to predict the emergence of important morphological traits (fruit size, mass, shape and texture) and their dependence on environmental and genetic factors. Historically, a big debate has opposed two contrasting views, the cellular vs the organismal theory that set the control of organ growth at the level of the individual cell or of the whole tissue, respectively (reviewed in [START_REF] Beemster | Cell cycle: the key to plant growth control?[END_REF][START_REF] Fleming | The integration of cell proliferation and growth in leaf morphogenesis[END_REF][START_REF] John | Cell division and endoreduplication: doubtful engines of vegetative growth[END_REF]. In the recent years, a consensus view, the neo-cellular theory, has eventually emerged.

Accordingly, although individual cells are the units of plant morphology, their behavior (division, expansion) is not autonomous, but coordinated at the organ level by cell-to-cell communication mechanisms, thus creating an effective interaction between cellular and whole-organ behavior [START_REF] Beemster | Cell cycle: the key to plant growth control?[END_REF][START_REF] Sablowski | Interplay between cell growth and cell cycle in plants[END_REF][START_REF] Tsukaya | Organ shape and size: a lesson from studies of leaf morphogenesis[END_REF]. The existence of non-cell autonomous control of organ development has been demonstrated in Arabidopsis leaf [START_REF] Kawade | Non-cell-autonomously coordinated organ size regulation in leaf development[END_REF] but the underlying modes of action remain unclear and often system-specific [START_REF] Ferjani | Analysis of leaf development in fugu mutants of Arabidopsis reveals three compensation modes that modulate cell expansion in determinate organs[END_REF][START_REF] Han | Transcription factormediated cell-to-cell signalling in plants[END_REF][START_REF] Horiguchi | Organ Size Regulation in Plants: Insights from Compensation[END_REF][START_REF] Norman | Intercellular communication during plant development[END_REF][START_REF] Okello | What drives fruit growth?[END_REF].

Computational models offer a unique tool to express and test biological hypotheses, in a well-defined and controlled manner. Not surprisingly, indeed, computational modeling has been largely used to investigate the relationships between organ development and the underlying cellular processes. Many works have addressed the question of organogenesis, relating local morphogenetic rules and cell mechanical properties with the emerging patterns near the meristem [START_REF] Boudon | A Computational Framework for 3D Mechanical Modeling of Plant Morphogenesis with Cellular Resolution[END_REF][START_REF] Dupuy | Coordination of plant cell division and expansion in a simple morphogenetic system[END_REF][START_REF] Kuchen | Generation of leaf shape through early patterns of growth and tissue polarity[END_REF][START_REF] Löfke | Auxin regulates SNAREdependent vacuolar morphology restricting cell size[END_REF][START_REF] Lucas | Lateral root morphogenesis is dependent on the mechanical properties of the overlaying tissues[END_REF][START_REF] Robinson | Generation of spatial patterns through cell polarity switching[END_REF][START_REF] Von Wangenheim | Rules and Self-Organizing Properties of Postembryonic Plant Organ Cell Division Patterns[END_REF]. At the tissue scale, a few models have addressed the issue of cell's size variance based on observed kinematic patterns of cell division or growth rates, with a particular attention to the intrinsic stochasticity of cell-cycle related processes [START_REF] Asl | Model-based analysis of Arabidopsis leaf epidermal cells reveals distinct division and expansion patterns for pavement and guard cells[END_REF][START_REF] Kawade | Probing the stochastic property of endoreduplication in cell size determination of Arabidopsis thaliana leaf epidermal tissue[END_REF][START_REF] Roeder | Variability in the control of cell division underlies sepal epidermal patterning in Arabidopsis thaliana[END_REF]. In most of these models, cell expansion is simply described via an average growth rate, possibly modulated by the ploidy level of the cell, without any reference to the underlying molecular mechanisms or to the environmental conditions.

To our knowledge, very few attempts have been made to explicitly model the interaction among cell division, expansion and endoreduplication from first principles and at the scale of organ development. In [START_REF] Fanwoua | A dynamic model of tomato fruit growth integrating cell division, cell growth and endoreduplication[END_REF] model of tomato fruit development have been developed that integrates cell division, expansion and endoreduplication processes based on a set of biologically-inspired rules. The fruit is described by a set of q classes of cells with the same age, ploidy and mass. Within each class, cell division and endoreduplication are described as discrete events that take place within a welldefined window of time, whenever a specific mass-to-ploidy threshold is reached.

Cell growth in dry mass is modeled following a source-sink approach as a function of the thermal time, the cell ploidy-level and the external resources. The model is able to qualitatively capture the effect of environmental conditions (temperature, fruit load) on the final fruit dry mass, but hypotheses and parameters are hard to validate as comparaison to experimental data is lacking.

Moreover, the water content of the cell is not considered preventing the analysis of cell volumes.

Baldazzi and coworkers developed an integrated model of tomato fruit development which explicitly accounts for the dynamics of cell proliferation as well as for the mechanisms of cell expansion, in both dry and fresh masses, based on biophysical and thermodynamical principles [START_REF] Baldazzi | A model of fruit growth integrating cell division and expansion processes[END_REF][START_REF] Baldazzi | In-silico analysis of water and carbon relations under stress conditions. A multi-scale perspective centered on fruit[END_REF]. Here, a new version of this model, which includes cell endoreduplication, is proposed. The model was used to investigate different hypotheses concerning the regulation and the interaction among cellular processes, with special attention to 1) the importance of an organ-wide regulation on cell growth and 2) the mechanisms of interaction between endoreduplication and cell expansion processes.

We focus on a natural, wild type organ development and we analyze the effect of organ-wide or cell ploidy-dependent regulation onto the dynamics of cell expansion. To this aim, different control schemes (either cell-autonomous or organ-controlled, with or without ploidy effect on cell expansion) were tested in silico by means of specific model variants. Simulation results were analyzed and compared to cell size distributions observed on two contrasted genotypes, a cherry and a large-fruited tomato variety.

The model shows that a pure cell-based control cannot reproduce the observed cell size distribution, and an organ-wide control is required in order to get realistic cell and fruit sizes. The model also supports the role of endoreduplication as an important modulator of the cell expansion potential, although the strength of this interaction might be genotype-specific. In particular, results suggest a likely interaction through carbon allocation and metabolism.

MATERIALS AND METHODS

Experimental data

Two datasets were collected from two glasshouse experiments performed at and cherry (cv. Cervil) tomato genotypes of Solanum lycospersicum L.

The 2004 experiment fruits were collected from April to May (planting in February) whereas in the 2007 experiment fruits were sampled from October to December (planting in August). Plants were grown according to standard cultural practices.

Trusses were pruned in order to homogenise truss size along the stem within each genotype. Maximum number of flowers left on each inflorescence was 12

for Cervil and 6 for Levovil. Flowers were pollinated by bumblebees. Air temperature and humidity were recorded hourly in each experiment and input in the model as external signals.

Flower buds and fruits were sampled at different time points relative to the time of flower anthesis (full-flower opening). Fruit fresh and dry mass and pericarp fresh mass were systematically measured at all times points, before further measurements. In 2004, half of the fruit pericarps were then analyzed by flow cytometry and the other half were used for the determination of cell number. The number of pericarp cells was measured after tissue dissociation according to a method adapted from that of Bünger-Kibler and Bangerth, 1982 and detailed in [START_REF] Bertin | A model for an early stage of tomato fruit development: cell multiplication and cessation of the cell proliferative activity[END_REF]. Cells were counted in aliquots of the cell suspension under an optical microscope, using Fuchs-Rosenthal chambers or Bürker chambers for the large and small fruits, respectively. Six to 8 aliquots per fruits were observed and the whole pericarp cell number was calculated according to dilution and observation volumes. The ploidy was measured in the pericarp tissue, as described in [START_REF] Bertin | A model describing cell polyploidization in tissues of growing fruit as related to cessation of cell proliferation[END_REF]. The average value of three measurements per fruit (when allowed for by the fruit's size), was included in the analysis.

In the 2007 experiment, the dynamics of cell number (but not endoreduplication)

was measured following the same method as in the 2004 experiment. In addition, cell size distribution (smallest and largest radii and 2D-surface) distributions were measured with ImageJ software (imagej.nih.gov/ij/) in the cell suspension aliquots. About 20 to 25 cells per fruit were measured randomly on different fruits. Cell size distribution were derived for ripe fruits at about 43 days after anthesis (DAA) for Cervil and 60 DAA for Levovil in the considered growing conditions.

Model description

The model is composed of two interacting modules, both issued from previously published models [START_REF] Bertin | A model describing cell polyploidization in tissues of growing fruit as related to cessation of cell proliferation[END_REF][START_REF] Fishman | A biophysical model of fruit growth: simulation of seasonal and diurnal dynamics of mass[END_REF][START_REF] Liu | Model-assisted analysis of tomato fruit growth in relation to carbon and water fluxes[END_REF]. The fruit is described as a collection of cell populations, each one having a specific age, ploidy and volume, which evolve and grow over time during fruit development. The number, age (initiation date) and physiological state (proliferating or endoreduplicating-expanding cells) of each population is predicted by the cell division-endoreduplication module [START_REF] Bertin | A model describing cell polyploidization in tissues of growing fruit as related to cessation of cell proliferation[END_REF], based on genotype-specific parameters. It is assumed that the onset of endoreduplication coincides with the beginning of the expansion phase, i.e.

expanding cells are endoreduplicating.

At any time, mass (both fresh and dry component) of expanding cells is computed by a biophysical expansion module according to cell's characteristics (age, ploidy) and depending on available resources and environmental conditions [START_REF] Fishman | A biophysical model of fruit growth: simulation of seasonal and diurnal dynamics of mass[END_REF][START_REF] Liu | Model-assisted analysis of tomato fruit growth in relation to carbon and water fluxes[END_REF]. Briefly, cell expansion is described by iteratively solving the Lockhart equation relating the rate of volume increase to the cell's internal pressure and cell's mechanical properties (Lockhart, 1965).

Flows of water and solutes across the membrane are described by thermodynamic equations and depend on environmental conditions. The relative importance of each transport process may vary along fruit developmental stages, depending on specific developmental control. A full description of the model and its equations can be found in the section S2 of the Supplemental Material.

In its standard version, the model assumes that all cells have equal access to external resources, independently from the number of cells (no competition). All the parameters of the division-endoreduplication module are considered to be independent from environmental conditions for the time being.

6

Model initialisation and input

The model starts at the end of the division-only phase, when the proliferative activity of the cells declines and the expansion phase begins [START_REF] Baldazzi | In-silico analysis of water and carbon relations under stress conditions. A multi-scale perspective centered on fruit[END_REF]. For Cervil genotype this corresponds to approximatively 8 days before anthesis and to 3 days before anthesis for Levovil genotype [START_REF] Bertin | A model describing cell polyploidization in tissues of growing fruit as related to cessation of cell proliferation[END_REF]. The initial number of cells, n0, was estimated to 1e4 for the cherry tomato (Cervil) and 1.8e5 for the large-fruited (Levovil) genotype based on a few measurements.

At the beginning of the simulation, all cells are supposed to be proliferating with a ploidy level of 2C (transient ploidy of 4C during cell cycle is not considered). Proliferating cells are supposed to have a constant cell mass, m0, as often observed in meristematic cells (homogeneous/uniformity in cell size) [START_REF] Sablowski | Interplay between cell growth and cell cycle in plants[END_REF][START_REF] Serrano-Mislata | Active Control of Cell Size Generates Spatial Detail during Plant Organogenesis[END_REF]. The initial mass of the fruit is therefore Mf(0)=n0*m0=n0*(w0 +s0), where w0 and s0 are initial cell water and dry mass, respectively. At any time, cells leaving the proliferative phase start to grow, from an initial mass 2*m0 and a ploidy level of 4C, according to the expansion model and current environmental conditions.

Cell expansion depends on environmental conditions and resources provided by the mother plant. The phloem sugar concentration is assumed to vary daily between 0.15 and 0.35 M whereas stem water potential oscillates between -0.05 and -0.6 MPa i.e. typical pre-dawn and minimal stem water potential measured for the studied genotypes. Temperature and humidity are provided directly by real-time recording of greenhouse climatic conditions.

Choice of the model variants: control of cell expansion capabilities

In the integrated model, a number of time-dependent functions account for developmental regulation of cell's metabolism and physical properties during the expansion phase [START_REF] Baldazzi | In-silico analysis of water and carbon relations under stress conditions. A multi-scale perspective centered on fruit[END_REF][START_REF] Liu | Model-assisted analysis of tomato fruit growth in relation to carbon and water fluxes[END_REF]. Two characteristic timescales are recognizable in the model: the cell age, i.e. the time spent since an individual cell has left the proliferative phase, and organ age i.e. the time spent since the beginning of the simulation (Figure 1). Depending on the settings of the corresponding time-dependent functions, different cellular processes may be put under cell-autonomous or non-cell autonomous control (hereafter indicated as organ-wide control), allowing for an in silico exploration of alternative control hypotheses in the perspective of the cellular and organismal theories. Moreover, a direct effect of cell DNA content onto cell expansion capabilities may be tested according to biological evidences [START_REF] Chevalier | Elucidating the functional role of endoreduplication in tomato fruit development[END_REF][START_REF] Edgar | Endocycles: a recurrent evolutionary innovation for post-mitotic cell growth[END_REF][START_REF] Sugimoto-Shirasu | Big it up": endoreduplication and cell-size control in plants[END_REF].

As a default all cellular processes are supposed to be dependent on cell age (cell-autonomous control) with the only exception of cell transpiration which is computed at the organ scale, on the basis of fruit external surface and skin conductance, and then distributed back to individual cells, proportionally to their relative water content (see section S2).

Based on literature information and on preliminary tests ( [START_REF] Baldazzi | In-silico analysis of water and carbon relations under stress conditions. A multi-scale perspective centered on fruit[END_REF][START_REF] Baldazzi | Cell division, endoreduplication and expansion processes: setting the cell and organ control into an integrated model of tomato fruit development[END_REF], the switch between symplastic and apoplastic transport, σ p has been selected as the candidate process for an organ-wide control. Indeed, intercellular movement of macromolecules across plasmodesmata has been shown to be restricted by organ age in tobacco leaves [START_REF] Crawford | Non-targeted and targeted protein movement through plasmodesmata in leaves in different developmental and physiological states[END_REF][START_REF] Zambryski | Cell-to-cell transport of proteins and fluorescent tracers via plasmodesmata during plant development[END_REF] and it is known to be important for cell-to-cell communication [START_REF] Han | Transcription factormediated cell-to-cell signalling in plants[END_REF].

The exact mechanisms by which cell DNA content may affect cell expansion remains currently unknown. Based on literature information and common sense, three distinct mechanisms of interaction between endoreduplication and cell expansion were hypothesized.

1) Endoreduplication has been often associated to an elevated protein synthesis and transcriptional activity [START_REF] Chevalier | Endoreduplication and fruit growth in tomato: evidence in favour of the karyoplasmic ratio theory[END_REF] suggesting a general activation of the nuclear and metabolic machinery of the cell to sustain cell growth [START_REF] Sugimoto-Shirasu | Big it up": endoreduplication and cell-size control in plants[END_REF]. Following these insights, a first hypothesis assumes an effect of endoreduplication on cell expansion as a ploidydependent maximal import rate for carbon uptake. For sake of simplicity, the relation was supposed to be linear in the number of endocycles. The corresponding equation, as a function of the cell DNA content (DNAc, being 2 for dividing cells, 4 to 512 for endoreduplicating cells), was

ν m =⟨ ν 0 ⟩ * log 2 ( DNAc ) ,
where ⟨ ν 0 ⟩ is the average C uptake activity per unit mass.

2) In addition to a high transcriptional activity, endoreduplicating cells are characterized by a reduced surface-to-volume ratio with respect to 2C cells. As a consequence, it is tempting to speculate that one possible advantage of a high ploidy level may reside in a reduction of carbon demand for cell wall and structural units [START_REF] Barow | Endopolyploidy in seed plants[END_REF][START_REF] Pirrello | Transcriptome profiling of sorted endoreduplicated nuclei from tomato fruits: how the global shift in expression ascribed to DNA ploidy influences RNA-Seq data normalization and interpretation[END_REF]. Such an economy may impact cell expansion capabilities in two ways. First, the metabolic machinery could be redirected towards the synthesis of soluble components, thus contributing to the increase of cell's internal pressure and consequent volume expansion.

In the model, the ssrat fraction of soluble compound within the cell is developmentally regulated by the age t of the cell [START_REF] Baldazzi | In-silico analysis of water and carbon relations under stress conditions. A multi-scale perspective centered on fruit[END_REF] as

ssrat =b ssrat (1-e -a ssrat * t )+ssrat 0
In the presence of a ploidy effect, the final bssrat value was further increased as

b ssrat =b ssrat 0 * log 2 (DNAc )
3) Alternatively, "exceeding" carbon may be used to increase the rate of cell wall synthesis or related proteins, resulting in an increase of cell wall plasticity [START_REF] Proseus | Identifying cytoplasmic input to the cell wall of growing Chara corallina[END_REF].

In the original expansion model on tomato [START_REF] Liu | Model-assisted analysis of tomato fruit growth in relation to carbon and water fluxes[END_REF] cell wall extensibility Phi declines during cell maturation [START_REF] Proseus | Separating Growth from Elastic Deformation during Cell Enlargement[END_REF] as

Phi=Phi min + ( Phi max -Phi min ) 1+e k (t -t 0)
In the presence of a ploidy effect, the maximal cell wall extensibility was increased as

Phi max =Phi max 0 * log 2 ( DNAc )
The individual and combined effects of organ-wide and ploidy-dependent control (one process at time) on cell expansion were investigated and compared to a full cell-autonomous model. A total of 8 model variants have been tested for each genotype, following the complete experimental design shown in Table 1.

Model calibration

Calibration has been performed using genetic algorithm under R software (library 'genalg'). A two-steps procedure has been used for each tomato genotype.

First, the division-expansion module (7 parameters) was calibrated on data from the 2004 experiment by comparing measured and simulated values of the total cells number and the proportion of cells in different ploidy classes, all along fruit development. The best fitting parameters were selected and kept fixed for the second phase of the calibration, assuming they are independent from environmental conditions.

The expansion module was calibrated on the evolution of pericarp fresh and dry mass from the 2007 experiment, for which cell size distribution were measured.

Six parameters have been selected for calibration based on a previous sensitivity analysis [START_REF] Constantinescu | Model-Assisted Estimation of the Genetic Variability in Physiological Parameters Related to Tomato Fruit Growth under Contrasted Water Conditions[END_REF], whereas the others have been fixed to the original models' values [START_REF] Baldazzi | In-silico analysis of water and carbon relations under stress conditions. A multi-scale perspective centered on fruit[END_REF][START_REF] Fishman | A biophysical model of fruit growth: simulation of seasonal and diurnal dynamics of mass[END_REF][START_REF] Liu | Model-assisted analysis of tomato fruit growth in relation to carbon and water fluxes[END_REF]). An additional parameter was estimated for model variants M3 to M7 in order to correctly evaluate the strength of the ploidy-dependent control (see section S3 for more information).

Due to their different structures, the expansion module was calibrated independently for each model variant. The quality of model adjustment was evaluated using a Normalized Root Mean Square Error (NRMSE):

NRMSE ( x )=100 √ 1 n ∑ (O i -S i (x ) ) 2 1 n ∑ O i
where Oi and Si are respectively, the observed and simulated values of pericarp fresh and dry masses, and n is the number of observations. x= { x 1, x 2 ... x p } is parameter set of the evaluated solution. The smaller the NRMSE the better the goodness-of-fit is. A NRMSE < 20% is generally considered good, fair if 20% < NRMSE <30% and poor otherwise.

To avoid high mean fitting errors in each condition, the choice of the best calibration solution was based on a min-max decision criterion [START_REF] Constantinescu | Model-Assisted Estimation of the Genetic Variability in Physiological Parameters Related to Tomato Fruit Growth under Contrasted Water Conditions[END_REF]:

min x∈ X { max (NRMSE FM ( x ) , NRMSE DM ( x ) )}
where NRMSEFM and NRMSEDM are the normalized root mean square errors for fresh and dry masses respectively. Selected parameters are reported in tables S3

and S4.

Model comparison and selection

On the basis of the best calibration solution, model selection has been performed by comparing measured and simulated cell size distribution, for each model variant. A semi-quantitive comparison approach has been used due to the limited experimental information available: the general distribution characteristics (shape, positioning and dispersion) have been characterized rather than a perfect fit. To this aim, 8 descriptive statistical indicators have been computed for each model variant and compared to those derived from real-data distribution, namely:

 skewness and kurtosis (distribution's shape)

 mean and median cell size (positioning)

 standard deviation (SD) and median absolute deviation (MAD) (data dispersion)

 maximal and minimal cell size (data dispersion)

In order to compare different calibration solutions, a principal component analysis (PCA) was performed on the descriptors of cell distribution arising from each model estimation. The ade4 library of R software was used for this purpose (R development Core Team 2006).

RESULTS

A characteristic right-tailed distribution of cell areas

The distribution of cell sizes at a given stage of fruit development directly depends on the particular cell division and expansion patterns followed by the organ up to the considered time. Any change in the cell division or expansion rate will have a consequence on the shape and position of the observed distribution.

For both tomato genotypes considered in this study, the cell area distribution at mature stage shows a typical right-tailed shape (Figure 2), compatible with a

Weibull or a Gamma distribution (see section S1). The observed cell sizes span up to two orders of magnitude, with cell areas (cross section) ranging from 0.004 to 0.08 mm 2, for Cervil genotype, and from 0.005 to 0.28 mm2 for Levovil (Table 3 and4). The average cell area is calculated to be 0.026 mm2 for the cherry tomato and 0.074 mm2 for the large-fruited genotype, values in agreement with data from other tomato varieties [START_REF] Bertin | Analysis of the tomato fruit growth response to temperature and plant fruit load in relation to cell division, cell expansion and DNA endoreduplication[END_REF][START_REF] Renaudin | Cell layer-specific patterns of cell division and cell expansion during fruit set and fruit growth in tomato pericarp[END_REF]. Data dispersion is higher for the large-fruited genotype, but the shape of the distribution, as measured by its skewness and kurtosis values, is pretty similar for both tomato varieties.

Assuming that cell area distribution is a good descriptor of the underlying interaction patterns among cellular processes, a principal component analysis was performed on 8 statistical descriptors of cell size distribution in order to compare predictions of M0-M7 models (see M&M and Tables 2 and3 ). For both genotypes, the two first principal components explained approximately 90% of observed variance and were able to correctly separate models with cellautonomous control (M0, M5, M6) from models including an organ-wide control of cell expansion (Figure 3 and4). For Cervil genotype, the intra-model variability of cell size distribution was lower than then inter-model differences, whereas for Levovil models including a ploidy-dependent effect resulted in a similar cell size distribution.

Separation was mainly performed by the first principal component on the basis of the width of the distribution (sd, mad and maximal cell size) and its skewness, both generally increased in models including an interaction between endoreduplication-expansion processes (Figure 3 and4). Models without organwide control are characterized by a reduced dispersion and a larger median value. With respect to experimental data, models combining an organ-wide and a ploidy-dependent control gave the best results.

In the following, the effect of specific control mechanisms on the resulting cell area distribution is analysed in details, based on the results obtained for the bestfitting solution. The corresponding statistical descriptors are reported in Table 2 and 3, for Cervil and Levovil genotype respectively. Note that predicted minimal cell sizes for Levovil genotype are systematically lower than experimental measurements and correspond to the size of proliferating cells. This is partly due to the dissociation method employed for cell counting that underestimates small sub-epidermal cells. These cells although quite numerous contribute little to total pericarp volume and mass [START_REF] Renaudin | Cell layer-specific patterns of cell division and cell expansion during fruit set and fruit growth in tomato pericarp[END_REF].

A simple cell-autonomous control scheme leads to unrealistic cell size distribution

As a benchmark model, the case of a simple cell-autonomous control, without ploidy-dependent effect, was first considered (version M0 of the model).

Accordingly, two cells with the same age, even if initiated at different fruit developmental stages, behaved identically in what concerns carbon metabolism, transport and wall mechanical properties. In this scheme, therefore, cell size variations derived exclusively from the dynamics of cell division, which caused a shift in the initiation date for different cohort of cells. When applied to our genotypes, the cell-autonomous model was able to reproduce the observed fruit mass dynamics but the corresponding cell size distribution was extremely narrow (Figure 5 A and D), with standard deviation less than 3e-3, and strongly left-tailed (see Table 2 and3).

Including an organ-wide mechanism that controls cell size (model M1) introduces a source of variance among cells. In this case, two cells of the same age which were initiated at different fruit stages did not behave identically, resulting in different expansion capabilities and growth patterns [START_REF] Baldazzi | In-silico analysis of water and carbon relations under stress conditions. A multi-scale perspective centered on fruit[END_REF]. Following literature information and preliminary studies [START_REF] Baldazzi | Cell division, endoreduplication and expansion processes: setting the cell and organ control into an integrated model of tomato fruit development[END_REF] carbon symplastic transport had been supposed under organ-wide control, via the progressive closure of plasmodesmata with fruit age [START_REF] Zambryski | Cell-to-cell transport of proteins and fluorescent tracers via plasmodesmata during plant development[END_REF]) (see M&M section). As a result, the cell size distribution got larger, and skewness increased towards zero values, indicating a symmetric cell size distribution, both for cherry and large-fruited tomatoes. Indeed, the typical righttail observed in experimental data was absent and the maximum cell size predicted by the model was much smaller than expected. This suggested that a mechanism controlling cell expansion was lacking in the model.

Endoreduplication and cell growth: possible interactions and genotypic effect

A significant correlation between cell size and endoreduplication level has been often reported. However, the molecular mechanism by which ploidy could 2).

In both models, a high ploidy level resulted in a larger cell size, although the maximum predicted cell size was slightly lower than the observed one.

Correlation between ploidy level and cell area was significant with a p-value <0.0001 (Table S5). The heterogeneity of cell sizes usually observed at each ploidy level was correctly captured by the models as a consequence of the asynchrony in cell division and endoreduplication patterns [START_REF] Bourdon | In planta quantification of endoreduplication using fluorescent in situ hybridization (FISH)[END_REF][START_REF] Roeder | Variability in the control of cell division underlies sepal epidermal patterning in Arabidopsis thaliana[END_REF]. In comparison to the M2 and M3 models, the potential effect of endoreduplication onto cell's mechanical properties (M4) failed to increase cell size variance beyond the values already obtained without any ploidy effect (M1).

For Levovil genotype, the interaction between endoreduplication and expansion had less impact on the resulting cell distribution. For both model M2 and M4, the addition of a ploidy-dependent effect on cell expansion, although significant, was not able to produce a right-tailed distribution, as observed in experimental data (Figure 5E). The shape of the distribution was pretty symmetric, with skewness values close to zero, reduced dispersion and mean cell size. Only the inclusion of a ploidy-dependent effect on cell's strategy for carbon allocation (model M3) allowed increasing the skewness value up to a reasonable value, but the width of the distribution remained lower than expected with maximal cell sizes not exceeding 0.13 mm2 (against the 0.28mm2 observed) (Figure 5F).

The mechanisms that contribute to cell size variance are likely genotypedependent

The above results showed that cell distribution can be significantly affected by both an organ-wide control of fruit development and a direct interaction between endoploidy and cell expansion capabilities. In order to better discriminate their respective role, the effect of a ploidy dependent expansion was tested alone, without the contribution of an organ-wide control of cell growth (models M5-M7).

Results confirmed that the best results are obtained by a combined action of both an organ-wide control and a ploidy-dependent modulation of cell expansion but the relative importance of the two mechanisms is likely genotype-dependent.

In the case of Levovil, organ-wide control turned out to be a major regulatory mode. Independently of the interactions between endoreduplication and expansion, indeed, models without organ-control (models M5-M7) completely failed to reproduce the observations, resulting in a very narrow and left-tailed cell size distribution (Table 3). Organ-wide coordination of cell expansion appeared to be the main responsible for positive skewness of cell size distribution whereas the addition of an endoreduplication-mediated modulation of cell expansion capabilities, alone, resulted only in a marginal improvement of model's performances.

The relative roles of ploidy-dependent and organ-wide control of cell growth appear more balanced in cherry tomatoes. Indeed, with the exception of model M6, both the organ-wide and the ploidy-mediated control of cell expansion were able to reproduce the expected right-tailed distribution shape (Table 2). However, their concomitant action was needed in order to get a realistic cell size variance.

The two mechanisms thus seem to act in synergy to increase cell expansion and final cell size.

A combination of ploidy-dependent control of both carbon uptake and allocation better explains data

In spite of a good agreement with pericarp data, all tested models failed to fully reproduce the observed cell area distribution for Levovil genotype. In particular, the maximum reachable cell size predicted by the models was far lower than the observed data. Up to now, for seek of simplicity, interaction between endoreduplication and expansion has been supposed to affect a single process at time but in reality a combination of effects cannot be excluded. We therefore tried to combine a ploidy-dependent effect on both the maximum carbon uptake and the cell's allocation strategy between soluble and non-soluble compounds (model version M23).

Results showed that, for both genotype, the two mechanisms successfully combined together to improve cell's expansion capabilities. As a consequence, the tail of the distribution straightened and the maximal cell size increased towards realistic values (Figure 6). For Cervil genotype, indeed, the predicted cell size distribution approached the experimental one, as showed by the projection of the M23 model onto the first principal plane (Figure 3). For Levovil genotype, in spite of a significant increase of the maximal cell size, the width of the distribution remained much lower than in experimental data, resulting into a very long-tailed distribution shape.

DISCUSSION

The present paper describes an improved version of an integrated cell division-expansion model that explicitly accounts for DNA endoreduplication, an important mechanism in tomato fruit development. The model is used to investigate the interaction among cell division, endoreduplication and expansion processes, in the framework of the neo-cellular theory [START_REF] Beemster | Cell cycle: the key to plant growth control?[END_REF]. To this aim, 8 model variants including or not an organ-wide control of cell development, have been tested and compared to data from two contrasting tomato genotypes. Model simulations showed that a pure cellular control was unable to reproduce the observed cell size distribution in terms of both average cell size and cell variance. In agreement with the neo-cellular theory, the model supported the need for an organ-wide control of cell growth, mediated by cell-tocell communication via plasmodesmata [START_REF] Han | Transcription factormediated cell-to-cell signalling in plants[END_REF][START_REF] Norman | Intercellular communication during plant development[END_REF] and confirmed the role of endoreduplication as an important modulator of the cell expansion potential.

According to the model, organ-wide control was the main responsible of cell-to-cell variance but a ploidy-mediated effect on cell expansion was needed in order to obtain large cell surface as the ones observed in experimental data. A positive significant correlation between cell size and ploidy level was clearly reproduced, independently of model version, in agreement with recent analysis of cell size in both leaf epidermis [START_REF] Kawade | Probing the stochastic property of endoreduplication in cell size determination of Arabidopsis thaliana leaf epidermal tissue[END_REF][START_REF] Roeder | Variability in the control of cell division underlies sepal epidermal patterning in Arabidopsis thaliana[END_REF] and fruit pericarp [START_REF] Bourdon | In planta quantification of endoreduplication using fluorescent in situ hybridization (FISH)[END_REF].

However, the strength of the correlation may be genotype-dependent. For the cherry tomato variety, our modelling approach showed that a ploidy-dependent control of carbon metabolism (uptake, allocation or a combination of the two), in combination to an organ-wide modulation of cell expansion, was able to generate a cell size distribution close to the observed one. For large-fruited tomato genotype, interaction between endoreduplication and cell expansion via a single process did not suffice to get large cells. Independently of the specific interaction mechanism, the model proved unable to correctly reproduce the observed cell sizes, resulting in a too narrow and symmetric distribution. The addition of a double effect of cell's endoploidy on both carbon uptake and allocation was able to increase the maximal cell size close to the correct values, but dispersion remained lower than expected.

Cell stochasticity may be important to explain cell size distribution in fruit

A few reasons may account for the discrepancy observed for the largefruited variety. A first issue may reside in the quality of experimental data for Levovil genotype. A rapid computation of the average cell size as the pericarp volume over the number of cells at maturity, yields a value significantly lower than the mean of the experimental distribution. This means that the proportion of small cells is underestimated in our dataset, resulting in a distribution that is probably a little flatter and less skewed than the real one.

A second, more fundamental reason of discrepancy is rooted in our modelling approach. Our model is an example of population model: the fruit is described a collection of cell groups, each having specific characteristics in terms of number, mass, age and ploidy level, that dynamically evolve during time. Although asynchrony in the emergence of cell groups allowed to capture a considerable part of cell-to-cell heterogeneity, intrinsic stochasticity of cellular processes (Meyer and Roeder, 2014;Robinson et al., 2011b;Smet and Beeckman, 2011) are not accounted for. Variations in the threshold size for division (often associated to a change in the cell cycle duration) as well as asymmetric cell divisions are considered as important determinant of the final cell size [START_REF] Dupuy | Coordination of plant cell division and expansion in a simple morphogenetic system[END_REF][START_REF] Osella | Concerted control of Escherichia coli cell division[END_REF][START_REF] Roeder | Variability in the control of cell division underlies sepal epidermal patterning in Arabidopsis thaliana[END_REF][START_REF] Stukalin | Agedependent stochastic models for understanding population fluctuations in continuously cultured cells[END_REF] they may contribute to significantly spread the size distribution of both dividing and expanding cell groups, from the early stages. Moreover, the degree of additional dispersion introduced by cell expansion is likely to depend on the specificity of the underlying mechanisms, with possible interactions with ploidy-dependent and organ-wide controls. Of course, these mechanisms are intrinsic of the development of any new organ, independently of the genotype, but they can prove more sensitive for Levovil genotype than for Cervil in reason of its higher cell number and longer division windows.

In perspective, the addition of stochastic effects could help to fill the missing variance for both Cervil and Levovil genotypes. To this aim, a novel modelling scheme is needed in which the average cell mass of a group is replaced by a distribution function of cell sizes, whose parameters can evolve with time under the effect of cell expansion processes.

Endoreduplication could regulate cell size through carbon allocation and metabolism

From a biological point of view, the model suggested that endoreduplication may interact with cell's carbon metabolism, increasing the substrate potentially available for cell expansion. This is in line with literature data pointing to ploidy level setting the maximum potential cell size that can be attained or not, depending on internal (hormones) and external (environmental) factors [START_REF] Breuer | Developmental control of endocycles and cell growth in plants[END_REF][START_REF] Chevalier | Elucidating the functional role of endoreduplication in tomato fruit development[END_REF][START_REF] De | Molecular control and function of endoreplication in development and physiology[END_REF]. In particular, the model revealed that high ploidy level may increase both the carbon uptake rate and its relative allocation to soluble compounds, thanks to an overall economy in cell wall synthesis [START_REF] Barow | Endopolyploidy in seed plants[END_REF][START_REF] Pirrello | Transcriptome profiling of sorted endoreduplicated nuclei from tomato fruits: how the global shift in expression ascribed to DNA ploidy influences RNA-Seq data normalization and interpretation[END_REF].

It is important to stress that the molecular basis of the supposed interaction between endoreduplication and expansion are not described in the model and could involve many molecular players. Moreover, the existence of other targets of a ploidy-dependent control cannot be excluded nor the contribution of other mechanisms to the control of the final cell size. In many fruit species including tomato, a negative correlation between average cell size and cell number has been observed, suggesting the existence of a competition for resources [START_REF] Lescourret | A multi-level theory of competition for resources applied to fruit production[END_REF][START_REF] Prudent | Resource competition modulates the seed number-fruit size relationship in a genotype-dependent manner: A modeling approach in grape and tomato[END_REF]. This kind of mechanism may contribute to broaden the range of attainable cell sizes, increasing size variance among first and late-initiated cells (see section S5).The importance of such an effect may vary with genotype and environmental conditions [START_REF] Bertin | Analysis of the tomato fruit growth response to temperature and plant fruit load in relation to cell division, cell expansion and DNA endoreduplication[END_REF][START_REF] Quilot | Is competition between mesocarp cells of peach fruits affected by the percentage of wild species ( Prunus davidiana ) genome?[END_REF]. This may be especially important for largefruited tomatoes for which cell number is large.

Measurement of cell size distribution: promises and challenges to understand the control of fruit growth

Further work is needed in order to identify the mechanisms behind organ growth and cell size determination. To this aim, the analysis of cell size distribution shows up as a promising approach. When looking at our results, indeed, the NRMSE with respect to pericarp fresh and dry mass was always between 20% and 30% indicating a satisfactory agreement with data, independently from the model version and the tomato genotype. This highlights the fact that the dynamics of fruit growth alone is not enough to discriminate between several biologically-plausible models. In this sense, cell size distribution represents a much more informative dataset as it uniquely results from the specific cell division and expansion patterns of the organ [START_REF] Halter | Cell volume distributions reveal cell growth rates and division times[END_REF].

The assessment of cell sizes in an organ is not an easy task though. As illustrated by the case of Levovil, the employed measurement technique may have important consequences on the resulting cell size distribution [START_REF] Legland | Cartography of cell morphology in tomato pericarp at the fruit scale[END_REF]. Indeed, mechanical constraints acting on real tissues as well as vascularisation can largely modify cell shape, resulting in elongated or multi-lobed cells [START_REF] Ivakov | Plant cell shape: modulators and measurements[END_REF]. Thus, if the orientation of 2D slices can potentially affect the resulting cell area estimation, possible differences between in-vivo tissues and dissociated cells, both in number and size, should also be checked.

The use of mutant or modified strains [START_REF] Musseau | Identification of Two New Mechanisms That Regulate Fruit Growth by Cell Expansion in Tomato[END_REF] in combination with recent advancements in microscopy and tomography [START_REF] Mebatsion | A novel method for 3-D microstructure modeling of pome fruit tissue using synchrotron radiation tomography images[END_REF][START_REF] Wuyts | High-contrast three-dimensional imaging of the Arabidopsis leaf enables the analysis of cell dimensions in the epidermis and mesophyll[END_REF] could now permit the acquisition of more reliable datasets, opening the way to an in-depth investigation of cell size variance in relation to the their position within the fruit [START_REF] Renaudin | Cell layer-specific patterns of cell division and cell expansion during fruit set and fruit growth in tomato pericarp[END_REF] and to the underlying molecular processes. At term, improvements in the ability of computation models to integrate the multiple facets of organ development in a mechanistic way can help to evaluate and quantify the contribution of the different processes to the control of cell growth.

Figures

  south of France) in 2004 and 2007 on large-fruited(cv Levovil)

  modulate cell expansion capabilities remain elusive. In this work, three timedependent cell properties have been selected as possible targets of ploidydependent modulation (see M&M section): a) the maximum carbon uptake rate (model version M2), b) carbon allocation between soluble and non-soluble compounds (model version M3) c) cell wall plasticity (model version M4). The three hypotheses were tested independently on both genotypes, in combination with an organ-wide control. Results are shown in Table2 and 3. In most cases, the addition of a ploidy effect on cell expansion resulted in a positive skewed distribution, with comparable or increased cell size dispersion and maximum cell size with respect to the M0 and M1 models. The strength of the effect however strongly depended on the genotype, with different ranking among model versions. For Cervil genotype, a potential control of endoreduplication on both the Vmax (model M2) and cell's allocation strategy (model M3) provided distribution agreement with data, i.e. right-tailed distributions with good dispersion and correct positioning in both mean and median cell area (Figure 5 B and C). The shape of the distribution however was better reproduced by the M2 model, which showed a skewness and kurtosis values close to the observed ones, but the adjustment to fruit growth dynamics, especially for dry mass, was only partially satisfactory (Table

Figure 1 :Figure 2 : 29 Figure 3 :

 12293 Figure 1: Scheme of the integrated model. The fruit is described as a collection of cell populations, each one having a specific age, ploidy and volume. Cells can be either proliferating or expandingendoreduplicating. The number of cells in each class is predicted by the divisionendoreduplication module, assuming a progressive decline of cells' proliferating activity. Expanding cells grow according to the expansion module which provides a biophysical description of the main processes involved in carbon and water accumulation. It is assumed that the onset of endoreduplication coincides with the beginning of the expansion phase. Two timescales are recognizable in the model: the organ age i.e. the time since the beginning of the simulation, and the cell age i.e. the time since the cell left the mitotic cycle and entered the expansionendoreduplication phase. Depending on the model version, cell expansion may be modulated by organ age (organ-wide control) and/or by cell's ploidy.

Figure 4 :Figure 5 :

 45 Figure 4: Principal component analysis (PCA) cell size distributions obtained for the different estimations of models M0-M7 on Levovil genotype. 8 statistical descriptors are used as variables to characterize cell distribution. Measured cell size distribution and results of model M23 are projected as a supplementary observation. Top: Projection of individual distributions on the PC1-PC2 plane. Model variants are tagged with different colors. Bottom: Correlation of the variables with the first two principal components.

Figure 6 :

 6 Figure 6: Predicted cell area distribution at fruit maturity for model M23, combining a ploidydependent effect on both carbon uptake and allocation. Left: Cervil genotype. Right: Levovil genotype.

Table 1 :

 1 Experimental design showing the characteristics of the 8 model versions tested in the paper.

	MODEL	FRUIT MASS			CELL DISTRIBUTION
		FIT QUALITY		SHAPE	POSITIONING		DISPERSION
		NRMSE	NRMSE	Skewness Kurtosis Mean Median Min	Max	SD	MAD
		FM	DM
	Exp Data			0.97	3.13	0.026 0.019	0.0039 0.08	0.019	0.016
	M0	31.49	31.34	-4.06	30.34	0.030 0.031	0.0042 0.033	0.0015 0.0005
	M1	28.32	29.58	0.42	3.07	0.030 0.029	0.0044 0.041	0.0046 0.0042
	M2	23	36	0.98	3.58	0.027 0.025	0.0035 0.062	0.011	0.0099
	M3	20.18	31.24	0.53	2.58	0.026 0.025	0.0033 0.046	0.0077 0.008
	M4	24.97	27.73	0.53	3.45	0.027 0.027	0.0040 0.045	0.005	0.005
	M5	34.35	37.02	0.33	3.44	0.029 0.029	0.0040 0.037	0.0035 0.003
	M6	34.21	33.81	-3.7	24.9	0.036 0.036	0.0051 0.039	0.002	0.0008
	M7	29.53	32.47	0.24	3.34	0.034 0.033	0.0040 0.056	0.005	0.005
	M23	23.33	33.05	2.0	8.9	0.026 0.021	0.0037 0.11	0.015	0.009

Table 2

 2 Statistical descriptors for the measured and predicted cell area distribution for Cervil genotype. The NRMSE scores for predicted pericarp dry and fresh masses corresponding to the selected solution are reported under the columns "Fit Quality". For an easier interpretation, green boxes indicate an agreement within 30% with respect to experimental data, yellow an agreement between 30 and 40%, white between 40% and 70% and red a strong discrepancy (over 70% difference with respect to data).

	34

Table 3 :

 3 Statistical descriptors for the measured and predicted cell area distribution for Levovil genotype. The NRMSE scores for predicted pericarp dry and fresh masses corresponding to the selected solution are reported under the columns "Fit Quality". For an easier interpretation, green boxes indicate an agreement within 30% with respect to experimental data, yellow an agreement between 30 and 40%, white between 40% and 70% and red a strong discrepancy (over 70% difference with respect to data).

	35

Acknowledgments

The authors warmly thank B. Brunel for help with cell measurements. The authors are grateful to Inria Sophia Antipolis -Méditerranée "NEF" computation cluster for providing resources and support. This work was partially funded by the Agence Nationale de la Recherche, Project "Frimouss" (grant no. ANR-15-CE20-0009) and by the Agropolis Foundation under the reference ID 1403-032 through the « Investissements d'avenir » programme (Labex Agro:ANR-10-LABX-0001-01).


Fabien Gandon

    MonaLIA 1.0 preliminary study on the coupling learning and reasoning for image recognition to enrich the records of in the Joconde database Anna Bobasheva

OBJECTIVE

The goal of this project is to exploit the cross-fertilization of recent advances in image recognition and semantic indexing on annotated image databases in order to improve the accuracy and the details of the annotation. The idea would be, at first, to assess the potential of machine learning (including deep learning) and the semantic annotations on the Joconde database (350 000 illustrated artwork records from French museums). Joconde also contains metadata based on a thesaurus. In a previous project (JocondeLab) these metadata was formalized in semantic Web formalism and is linking the iconographic Garnier thesaurus and DBpedia to the data of the Joconde database.

This project enters into the current emerging trend in deep learning: Deep Reasoning (http://www.college-de-france.fr/site/yann-lecun/seminar-2016-04-08-12h00.htm). This topic focuses on combining the strength of two very powerful but different approaches for extracting information and knowledge from data, deep learning from unstructured data and reasoning from the structured data.

Objectives of the project:

This study would assess the interest and feasibility of several experimental objectives:

 Enhance the records with some keywords from the Garnier thesaurus and with some Inferences on the vocabularies used.  Use learning based on semantic queries on JocondeLab database to (1) enhance and complete the database [START_REF] Johnson | Love thy neighbors: Image annotation by exploiting image metadata[END_REF] apply the same process to other corpuses than Joconde.  Use pattern recognition to generate a ranking by iconographic relevance, in the lists of results from Joconde; currently with keyword indexing, we only know if an image contains a topic or not, but we cannot assess the importance of the topic compared to the global content of the image: it is unclear if it is the main theme, or just a detail. Accordingly, the resulting lists are polluted by images whose content reported by a keyword is actually anecdotal.

APPROACH IMAGE SUBSET FOR CATEGORY EXTRACTION

Query (SPARQL) the metadata to extract the subset of images that belong to a certain category or the subcategories below it to train the Neural Network classifier  Joconde dataset is represented by the RDF files containing artwork records metadata and thesauri (Joconde KB) as well as the collection of the JPEG image files. Metadata provides the association between the artwork records and the image file path in the collection. Each artwork record has a unique ID noticeRef that can be used to link the image and the metadata.

 To label the images for the classification I used two fields with hierarchical thesauri: represented subject (REPR thesaurus) and domain (DOMN thesaurus). The former is of the obvious interest and the latter is chosen as an experimental for its relatively simple thesaurus and image quantity sufficiency.  Query the Joconde KB to extract the records for a specified top category (concept, term) and all its subcategories capturing the hierarchy based on the thesauri.  Filter the records with available images

All records

PRE-PROCESS THE QUERY RESULTS

The images come in a proprietary file structure with artwork records containing the reference to the relative path in this structure. The classification framework however requires a different file structure so the image files have to be selected, rearranged and split into training, validation and test sets.

 Identify class (category) subsets with enough labeled images for training (at least 200)  Balance number of images per class (for simplicity reasons) by random choice  Exclude multi-labeled images for training set (for simplicity reasons)  Extract the actual image size information: done once and stored in the RDF file linkable with the image metadata by the noticeRef.  Exclude the images with the aspect ratio over 2.0 -too wide or too tall  Split the image subset on train/validation/test (80%-10%-10%)  Rearrange the image files to the structure suitable for the classifier's consumption.  Rename the image files from the original to the associated unique noticeRef to simplify the post processing.

RUN THE CNN CLASSIFIER

1 Some of the referenced images are not actually present in the provided image set

POST-PROCESS OF THE CLASSIFICATION RESULTS

Query the KB metadata again to find the dependencies between the classification outcome and the artwork properties. Query the metadata only for the test images to reduce the data size.

 Link the acquired image size data with the metadata.  Link the binary classification outcome (correct/incorrect) with the metadata.  Exclude sparsely populated metadata variables ( < 33%)  Reduce the number of classes in the categorical variables (manually selected count thresholds for each variable)  Run statistical analysis of the combined dataset to determine if there is any dependency of the prediction outcome and metadata variables o Cluster map of the confusion matrix to visualize the perceived similarities between the classes o Logistic regressions for the continuous variables (image width, height, aspect ratio) o Logistic regressions for categorical variables (art form, domain, technique, etc.) o Recursive Feature Elimination (down to 6) o Decision tree for selected variables to see if the outcome can be explained by the independent metadata and for the better visualization

IMPLEMENTATION

The steps described in the previous section are implemented as a set of 5 Jupyter Notebooks and a shared library script:

Conclusion:

There is a room for improvement of the Joconde KB in terms of filling the missing data either by experts or by inference or by deep learning algorithms.

Some of the properties would benefit from splitting into several and indexing to make them more searchable and machine understandable. (for ex.: noticeDeno, noticeTech, noticeEtat)

Property noticeDims (artwork dimensions) is in free form and unusable. The dataset would benefit from formalizing the dimensions properties (width, height, depth) and the units (cm, mm, m, inch, etc.).

The number of the subject term is too large to be able to do the deep learning classification by one CNN.

By far not all of the subject terms have enough images to train the CNN to recognize the subjects.

TRANSFER LEARNING

Learned parameters obtained from the PyTorch framework for the VGG16_bn model trained on ImageNet and loaded on the model creation. Ran classifications in three modes:  Calculated normalization vectors for 2 samples (animals, domains).  Post-classification analysis shown that the image aspect ratio over 2 or under 0.5 has a negative effect on the classification accuracy.

Conclusions:

The experiments shown that there is not much difference in the classification accuracy between using the normalization values of the samples or even ImageNet normalization parameters.

The aspect ratio of the image file affects the image classification if the image dimensions are too far from square. Did not use the "long" or "tall" images for training. Filtered out the images with the aspect ratio greater than 2.

IMAGE SET AUGMENTATION

In the Joconde image set the distribution of the images of certain objects is not uniform thus the classes are unbalanced.

Conclusion:

Adding the images from the external datasets for training increases the model quality.

CLASSIFICATION RESULTS ANALYSIS

An attempt of the statistical analysis was made to find the possible explanation of the missclassifications. The results are only preliminary and not exhaustive or conclusive.

We were interested in what are the metadata properties might have an effect on the correctness of the image classification.

To answer this question I've ran logistic regressions and decision tree algorithms on the dependent binary variable (correct/not correct classification) and the preselected metadata properties. The properties were selected based on subjective relevance and sparsity. Some of the properties were split into multiple variables. For the continuous numeric variables, it appears that the size matters. Especially the aspect ratio of the image. With the increase of aspect ratio by 1 the positive outcome is reduced by 20%.

Binary

For the categorical variables, the decision tree analysis shown that the classification outcome may be explained:

 by the art form, museum and the photography agency that took a digital picture. For example, these variables appear the most relevant for the Animals vs. Humans classifier:

 by the material of the artwork (paper, canvas, gold, etc.) . For example, this variable appear the most relevant for the animals and iconographic genre classifier:

 by the denomination (painting, button, etc.) in addition to the material (paper, canvas, gold, etc.) . For example, these variables appear the most relevant for the Animals classifier:

 Investigate the different image transformation techniques available in PyTorch torchvision package https://pytorch.org/docs/stable/torchvision/transforms.html to improve the classification accuracy.

 Build the multi-label classification model to detect multiple objects on the image. The majority of the artworks represent more than one subject therefore it will be interesting to investigate the possibility of detecting multiple classes on the same image using the multi label training techniques like One-hot-encoding. https://towardsdatascience.com/multi-label-classificationand-class-activation-map-on-fashion-mnist-1454f09f5925

 Build a hierarchical classification model. Investigate the depth of the hierarchy that can be useful in image annotation. Investigate the possibility of using the hierarchy in recognition of the different levels of details. For example, if the species of the animal cannot be recognized with high enough probability (cats, dogs, etc.) but the family is more certain (Felidae, Canidae, etc.) then the description of representation can be chosen just on the family level.

 Build a model to recognize the theme and/or iconographic genre and the content of the image by the same CNN. Investigate if the theme recognition can help with the object recognition. For example, train the classifiers for the iconographic genre (human representation, animal representation, etc.) and more specific subjects (man, woman, animal, bird, etc.) would be weighted accordingly.  Investigate the possibility of using the RNN classification layers based on this paper https://link.springer.com/article/10.1007/s11042-017-5443-x.  In the preliminary studies, we only used VGG16 model. Investigate if the other well-known pretrained models can improve the prediction accuracy. https://pytorch.org/docs/stable/torchvision/models.html  For the promising models build in the cross validation to determine the prediction power of a model.



  As a baseline model use VGG16 with batch normalization [3] pre-trained on the ImageNet (PyTorch VGG16_bn model) dataset  Transfer learning from the training of the network on the ImageNet dataset to decrease the training time. Learned parameters are available within the PyTorch framework.  Image transformations o Resize to 256x256 o Center Crop to 224X224 o Normalization where the normalization values for means and standard deviation were calculated over the sample of images.  Hyperparameters: o Mini batch = 4 o Cross-Entropy loss function, o stochastic gradient decent (SGD) as optimizing algorithm o Learning Rate = 0.001, adjusted every 4 epochs o 10 epochs.  Run training/validation in one of three modes: o only the last fully connected layer training o all fully connected layers training o all layers (full) training  Select the final model based on the best validation accuracy of the iteration. Save the parameters in a file.  Test the model on unseen images and record the metrics (precision, recall, f1 score per class and overall accuracy).  Record the top-5 predicted probabilities of the classes for each test image and store as an RDF file.

  Some of the properties that might be useful from the data inference point of view have more than 1 entity in them with entities being entered in free form. That makes them hard to use in the automated processing. Below are the properties that may benefit from the indexing: noticePeriTerm noticeAutrTerm noticeAptn noticeEcol noticePaut noticeDeno noticeReprTerm noticeMill noticePins noticeInsc noticeBibl noticeDmaj noticeLieuxTerm noticeGene noticeEtat noticeDesy noticeDepo noticeDecv noticeDrep noticeDdpt noticePuti noticePdec noticePlieux noticeDation noticeGeohi noticeNsda noticePdat noticeData Some 17426 terms are found that are grouped under the "quidams"@fr category. These terms are personal names and only assigned once or twice. These terms can be ignored.

	0 25 50 75 100 Terms associated with artwork records noticeCopy noticeImage noticeVideo noticeInv noticeTico noticePhot noticeRefim noticeMsgcom Terms associated with artwork records that have images Terms excluding names * Terms associated with more than 200 images The 10 most frequent terms are: REPR Term Frfequency noticeLabel noticeDims noticeContact % Filled homme@fr 31709	29185 22552 12013 790 % of images 90% 70% 37% 2.4% 10,62%
	femme@fr		27754		9,29%
	figure@fr		25323		8,48%
	scène@fr		23755		7,96%
	paysage@fr		23499		7,87%
	noticeLoca portrait@fr	Artwork location (city, 20859		Contains both city and museum. However, 6,99%
	museum) vue d'architecture@fr	14037	99.9%	the noticeMuseo property can be used instead. 4,70%
	noticePhot * ornementation@fr	Artwork photo credit 11457		In many cases this field represent a 3,84%
	en pied@fr		11099	99.2%	company and a photographer 3,72%
	1. MonaLIA.STEP 1.KB Query Result To Dataframe.ipynb 2. MonaLIA.STEP 2.Image Set Preprocessing.ipynb 5. MonaLIA.STEP 5.Classification Analysis.ipynb technique/materials 94,55% separated. 4. MonaLIA.STEP 4.Image Set Postprocessing.ipynb DATA QUALITY ASSESMENT noticeTech * Artwork Can contain many entities inconsistently 3. MonaLIA.STEP 3.Pretraned VGG16_bn Classification.ipynb 97,95% (country, region, department) RESULTS noticeLoca2 Artwork location 2 Contains 2 or 3 entities representing the en buste@fr 10914 3,66%
	6. MonaLIA.py noticeDims	Artwork dimensions		Totally free texted. However this data
				93,44 %	might be very useful in image
					classification.
	noticeDeno *	Artwork denomination		Can contain many entities inconsistently
				62,44%	separated
	 A special attention was paid to the property noticeReprTerm that links the terms of the REPR
	thesaurus inspired by the iconographic thesaurus by Francios Garnier
	http://www2.culture.gouv.fr/documentation/joconde/fr/partenaires/AIDEMUSEES/thesaurus-
	garnier/thesaurus-pres.htm published in 1984.
	All REPR terms				32274	100%
	6

 59% (285144) of all records (483297) have the images.  Joconda ontology defines 76 properties of the artwork o 37% of the properties (28) are filled over 75% o 46% (35) are filled under 25%.  56% (165800) of the images have the represented subject associated with them.  * Fields that were selected as explanatory variables to the image classification results analysis. *  The terms are organized in the hierarchies with 12 top concepts. The levels of the hierarchies may not have the same semantic level across all or the sub-hierarchies. The deepest branches are 11 edges removed from the root.

intérieur Classification: iconographic types Number of classes Train/Val/Test Set Size Test Accuracy Training Time (10 epochs)

  At the beginning used the same image transformations that the VGG16 on ImageNet training used, that is RandomCropResize+ RandomHorizontalFlip+ Normalization. But the experiments shown that for the Resize(256) + CenterCrop(224) + Normalization the accuracy increased by 3%. Used the latter for the majority on the experiments.

			Confusion Matrix 1
			Confusion Matrix 3
	ANIMALS  Aigle		Confusion Matrix 2  Lion Confusion Matrix 4
	Classification: Representation Type Last fully connected layer only training Fully connected layers only training Full training Last fully connected layer only training Fully connected layers only training Full training Confusion matrix : ANIMALS VS. HUMANS  espèce animale  âge de la vie Classification: animals & humans  Cerf  Chat  Cheval  Chien  Chèvre  Colombe ARTISTIC DOMAINS  dessin  estampe  Imprimé  Miniature Classification: Last fully connected Classification: animals Number of Number of classes 4 4 4 8 8 8 Number of classes Number of 2 domains classes classes Last fully connected 7 layer only training Fully connected layers 2 Last fully connected 14 layer only training layer only training Fully connected layers 7 only training Full training 2 Fully connected layers 14 only training only training Full training 7	Train/Val/Test Set Size 1828 228 232 81% Test Accuracy Training Time (10 epochs) 10min 1828 228 232 81% 11min 1828 228 232 85% 54min 5152 640 648 67% 30min 5152 640 648 66% 36min 5152 640 648 69% 2h 35min Train/Val/Test Set Size Test Accuracy  Mouton  Ophidien  Peinture  Papillon  Photographie  Sanglier  sculpture  Vache  âne Training Time Train/Val/Test Test Accuracy Training Time (10 epochs) 23058 2882 2882 82% Train/Val/Test Set Size (10 epochs) Test Accuracy Training Time 2h 19min Set Size (10 epochs) 5159 644 644 77% 25min 23058 2882 2882 85% 7153 893 893 32% 35min 3h 15min 5159 644 644 76% 36min 23058 2882 2882 7153 893 893 40% 1h 2min Not done 5159 644 644 83% 2hr 22min
	Full training	14	7153 893 893	35%	3h 32min

1. Last fully connected layer only training 2. Fully connected layers only training 3. Full training (using the learned parameters as initial parameters) Performance details of the experiments on different image subsets: REPRESENTATION TYPE (THEME)  genre iconographique  ornementation  représentation non figurative  représentation scientifique ICONOGRAPHIC TYPES (THEME/REPRESENTATION TYPES)  le paysage  nature morte  représentation animalière  représentation d'objet  représentation humaine  représentation végétale  vue d'architecture  vue d'

  To elevate the problem I downloaded the Painter by Numbers dataset from Kaggle that has 79434 images of artwork. If a certain class did not have enough training images, I've selected the images which had the class name in the title. For example adding images of the cats to the Cat-Dog-Horse classifier increased the test accuracy by 11%.

	Classification: cats,	Number of	Training Set	Test Accuracy	Test Accuracy
	dogs, horses	classes	Size	(last fully	(full training)
				connected
				layer)
	Training images only	3	636	53%	57%
	from the Joconde set
	Training images	3	1662	64%	68%
	augmented by images
	from Painter By
	Numbers

Team at French Ministry of Culture (MiC) :

-Laurent Manoeuvre (MIC / BDNC), project leader for MIC -Bertrand

The transferred parameters are very good set of initial parameters so the initial learning rate can be chosen at 0.001 for the loss function and convergence reached within 10 epochs.

Full training improved the accuracy 2-3%, which means that the last layer only training gives reasonably good results in the fraction of the time.

The broader (semantically) the class the higher the accuracy due to the number of factors such as larger number of available training images, smaller number of classes. However, what is very interesting that the broad classes like iconographic types and domains do not correspond to the classes of the ImageNet and yet yield a good accuracy.

Another very interesting observation is that the network was able to distinguish between the photography as an art form from photographs of the other art forms.

For the Animals classification, the dense layer training performed the best in terms of both accuracy and time. My explanation is that the ImageNet dataset contains the same classes, so the features are the same and did not need to be retrained. In fact, it might be that the retraining based on the "worse" quality images makes the model less accurate.

IMAGE PRE-PROCESSING

Conclusions:

It seems that there is a dependence of the classification outcome on some of the metadata properties.

Unfortunately, the metadata properties identified are dirty and the concrete conclusion cannot be reached.

FURTHER RESEARCH SEMANTIC INDEXING: JOCONDE KB

The most important decision has to be made on the subset of classes of interest out of 37279 from the REPR Thesaurus.