François Gardes
email: gardes@univ-paris1.fr.

Ali Abbas

Julien Boelaert

Francesco Gerotto

Philip Merrigan

Biases on variances estimated on large data-sets March 8, 2021

Keywords: dataset, estimated variance, spatial autocorrelation, grouped observations JEL Classification: C01, C12, C55

The inverse dependency of the estimated variances over the sample size throws a fundamental question on the validity of the usual statistical methodology, since any hypothesis on the value of a coefficient can be tested negatively by increasing the size of the data-set. I suppose that large data-sets are characterized by a concentration of information on homogenous sub-populations, a spatial autocorrelation of the error terms and the covariates may bias the estimation of variances. Using the corrections of variances under spatial autocorrelation, we obtain variances comparable to an estimation on sub-samples (named efficient sub-samples) the sizes of which are sufficient to contain the information which gives rise to similar estimates to those obtained on the whole population. Moreover, the estimation on efficient data-sets does not necessitate the specification of the spatial autocorrelations which are supposed to bias the estimated variances.

Introduction

Edward [START_REF] Leamer | Specification Searches: Ad Hoc Inference with Non Experimental Data[END_REF] and Deidre [START_REF] Mccloskey | The Loss Function Has Been Mislaid: The Rhetoric of Significance Tests[END_REF], among others, remarked that the variances are probably underestimated on large data-sets, which biases all inferences and tests on microdata. Estimates of the coefficient variances in a linear model are generally very low once the data-set is sufficiently large, since increasing the sample size diminishes the standard error by some factor linked to the square root of the size. This occurs for instance for cross-sections of households or for financial data. That important question have recently been accompagnied by critics on the p-value and the usual 95% confidence limit, which does not take into account the statistical type 2 errors provoked by misspecification [START_REF] Leamer | Specification Searches: Ad Hoc Inference with Non Experimental Data[END_REF][START_REF] Mccloskey | The Loss Function Has Been Mislaid: The Rhetoric of Significance Tests[END_REF][START_REF] Ziliak | The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives[END_REF]; see also [START_REF] Nuzzo | La Malédiction de la Valeur-P[END_REF], for other references).

No plausible explanation of that under-estimation of variances has been proposed yet except the idea that information perhaps does not increase at the same speed as the data size (Leamer discussed by [START_REF] Darnell | The Limits of Econometrics[END_REF], page 101) and the proposal by Leamer to use Bayesian inference (since the Bayes factor is determined in part by the sample size) and to carry out conventional testing with significance levels which are decreasing functions of sample size. I suppose in this article that the supplementary amount of information contained in large data-sets decreases with the concentration of information on restricted areas (homogenous sub-populations), which creates a spatial autocorrelation of the error terms and the covariates. [START_REF] Angrist | Almost Harmless Econometrics[END_REF] note that while "heteroskedasticity rarely leads to dramatic changes in inference, clustering can make all the difference", for instance by an under-estimation of the standard error by a factor 2.65 in the STAR experiment on the score of students in the same class. The estimation on large data-sets provides another evidence of this importance of spatial autocorrelation. Section 1 presents an example and our basic assumptions, section 2 an alternative method to remove the biases due to spatial autocorrelation and section 3 an application using a French survey containing more than ten thousand households.

1 The under-estimation of variances 1.1 An empirical example I consider, first the estimation of an Engel curve for four expenditures (under a Working specification where the budget share is regressed on logarithmic income and other covariates) on a cross-section containing 10251 households, then the estimation of the same model over sub-populations of much more limited size taken at random from that survey. Compare for instance in Table 1 the estimation of a linear model, first on the whole data-set, then on five hundred sub-populations which have been taken at random for a certain relative size (1% or one third of the whole survey in column 2 and 3). The estimated income elasticities are in average very similar for these two types of data if the subpopulations contain, for instance, only 1% of the population (105 households), while the estimated variance is much smaller on large data-sets compared to the estimates on sub-populations. For sub-population corresponding in average 2 Documents de travail du Centre d'Economie de la Sorbonne -2021.22 to 17% of the sample (an average of the size of the efficient sub-populations2 for food, housing and transport and other expenditures), the average inflating factor for the standard error compared to the whole sample is for instance 1.49. Everything acts as if small sub-populations are sufficient to recover the values of the parameters while indicating much more dispersed estimates.

Basic Hypothesis: Borsuk-Ulam theorem

The first possible explanation could be that the increase in the number of observations creates numerous pairs of close observations, the residual distances of which (squared in the loss function) are much smaller than the average distance among a limited number of pairs of observations which differ largely as concerns both the endogenous variable and the covariates. The examination of a case where the data-set is distributed among cells of equal volume in a two-dimensional space shows that this explanation does not help to explain the different values of the residual variances. An analysis on simulated data3 concludes to the same result. Another possible explanation could be based on the Borsuk-Ulam's topological theorem which states that, for any set of observations, there is a tendency to a concentration of these observations on poles acting as barycenters of sub-populations of similar individuals4 . Economic variables (both households' characteristics and the endogenous variables resulting from their choices) could concentrate households on selected parts of the possible location on the surface corresponding to all observed variables in a data-set 5 . Such a concentration of information in an economic data-set would imply a spatial correlation of the residuals which exists only when the number of observations is large. This spatial autocorrelation in turn will create an under-estimation of the variancecovariance matrix.

Borsuk-Ulam's theorem states that for every continuous mapping g from a n-dimensional sphere into R n , there exist a point for which g(x) = g(-x). This means that the second point (-x) can be removed from the sphere without loosing any information. In order to apply the Borsuk-Ulam's theorem to a data-set, consider a curve on a surface (y, x) in R n , with x a set n explanatory variables and y the explained variable. The curve y = g(x) can be considered as immersed in the smooth surface in R n corresponding to all observation of (y, x) given by the data-set. This surface is thus a hypersurface of dimension (n-1) in the space of dimension n which corresponds to an economic model defined by the equation relating y to x. The empirical counterpart of this theorem is thus that considering the sample of observations (y, x) as immersed into a smooth n-dimensional surface, the point in the sample which is the closest to (g(-x), x) could be removed without changing much the size of the sample. This operation can therefore be repeated till the remaining sub-sample cannot be considered as a good approximation of the embedding smooth surface. A classical application of the theorem is known as the Ham Sandwich Theorem : M 1 to M K being K bounded measurable subsets of R n , there exist a n-1 dimensional hyperplane that cuts all of them exactly into half of their measure. Applying this theorem to our statistical sample, the whole population being divided into K homogenous groups (immersed into the subsets M k ), a similar reasoning allows to divide by two the number of observations in each group, leaving K sub-populations M k composed of similar individuals to the half of each groups M " k which have been removed (such that M k ∪M " k = M k ). That means that the original sample can be finally reduced, by c consecutive operations, to a sub-sample containing 2 c less observations, the reduction finishing as soon as the remaining population cannot be considered as a good representation of the smooth surface (that is when the information contained in this surface is not fully represented by the final sub-population).

Our basic assumption can then be formulated: whenever the sample can be grouped into homogeneous sub-populations, there appears a correlation between the covariates within these sub-populations, since homogeneous populations means probably correlated explanatory variables between similar individuals. Therefore, it can be supposed that within these sub-populations, latent (unobserved) variables (which effects are composed into the residuals) are also correlated, which creates a correlation between the residuals within these subpopulations. This spatial correlation has as a consequence a negative bias in the estimated variances and covariances of the linear model, which depends on the size of the data-set.

Estimation of the bias : two methods

The various possible corrections of spatial autocorrelation are presented in the first sub-section. Another method, discussed in the second sub-section, is based on the definition of sub-populations the sizes of which are sufficient to recover all the information contained in the whole population and used to estimate the model. These sub-populations are named efficient sub-populations. The estimation on these sub-population gets rid of the biases in the estimation of variances provoked by the spatial autocorrelation due to the repetition of observations.

Usual Correction methods for the biases issued from a spatial autocorrelation

Bruce Moulton discussed the biases on estimated variances in the linear model when some covariates take a unique value in separate areas of the data-set: for instance the level of unemployment calculated for each state in a sample made over the whole US territory. His articles rely in fact on the formulas obtained by [START_REF] Kloek | OLS Estimation in a Model Where a Microvariable Is Explained by Aggregates and Contemporaneous Disturbances Are Equicorrelated[END_REF] and [START_REF] Greenwald | A General Analysis of the Bias in the Estimated Errors of Least Squares Coefficients[END_REF]. Bruce Greenwald gived a precise analysis of all types of biases as soon as some autocorrelation appears between residuals because of a spatial structure of the data-set (previous researches on the sub-ject by Engle, Kloeck, Pakes and Pfeffermann were dedicated to the calculus of bounds to the variances for extreme cases of spatial correlation). Greenwald's analysis (equation 7 in Appendix A) shows that the inflating factor increases with the product of the intra-class correlations of the covariate and the residual (which vary according to the level of definition of the clusters) and with the clusters average size and size variance. The product of intra-class-class correlations decreases from 0.026 to 0.021 when the number of clusters decreases from 88 to 30, and to 0.001 for 5 clusters, but these movements are compensated by the increase of the average size and size variances of the clusters. These formulas are discussed in Appendix A.

Estimation on efficient sub-samples

Whence the econometrician restricts the size of the sub-sample among a basic large population applying the Ham Sandwich theorem, the amount of reduction of the sample can be regulated by a statistical method: the precision of estimates being measured by the empirical variances of the coefficients among k sub-population of equal size (which is very similar to the empirical estimate of the s.e., see Table 1), a test of a difference between the estimates of more than 5% for more than 5% of this set of sub-sample indicates which minimum size is necessary in order to have less than 5% chances to obtain an incorrect estimate (i.e. remote from the value obtained using the full reference population). This minimum size n* can be obtained for instance for the case of the income coefficient in the Engel curve, resolving the following equation between the empirical standard error σ estimated on the whole population of size N, the relative size of the sub-population compared to the whole population and the value of the coefficient estimated on the whole population. Suppose that the correct value of the variance V ( β) is given by an estimation on a sub-sample containing a sufficient amount of information in order to have less than 5% chance that the estimate on the sub-sample differs from the true value estimated on the whole survey by less than 5%. The estimate of the variance on the whole sample will be under-estimated by a coefficient equal to the ratio of the sizes of the sub-sample and the whole sample:

V β N = n * N V β n * (1)
Indeed, the passage from the sub-sample to the whole sample can be considered as a replication as many times as necessary of sub-population containing the same information. The formula defining the minimum size writes:

1.96 σ N n * N = 0.05 β N (2)
which implies: Estimating on sub-samples having this minimum size thus affords a correct average estimate with a variance corresponding to a sufficient amount of information. The final estimate is the value obtained by the estimation on the full population, with a variance which is the average of variances obtained on multiple efficient sub-populations.

n * = 0.05 β N 1.96 σ N 2 N (3)
Note that the size of the efficient sub-populations differs between commodities, being much greater for food (2982 households) than for the three other aggregates (1856, 1350 and 759 households being necessary for Transport, Housing and All Other expenditures respectively). This size can be interpreted as an indicator of the heterogeneity of the population as concerns the income elasticity of the commodity. 6 The exactness of the estimation of variances can be checked by the absence of clustering effects measured by usual correction methods discussed in Appendix A.

Empirical application

A partition of a French Family Expenditures survey (2000, 10251 households, see a description of the data-set in Appendix B) into cells (defined by age classes, education level and family structure) defines 88 sub-populations containing in average 116 households. The same Working form as in Table 1 is adopted regressing the food budget share over total expenditures, the food price 7 , the number of Unit of consumption, age and its square (all these explanatory variables in logarithm) and the proportion of children. The resulting income elasticity is 0.5884 (s.e. 0.0072) for the whole sample. The intra-group correlations of explanatory variables are 0.503 for total expenditure and 0.117 for the food price. The intra-group correlation of the residuals is 0.046. The resulting variance inflation obtained by Greenwald-Moulton's formula [START_REF] Liang | Longitudinal Data Analysis Using Generalized Linear Models[END_REF] for the income coefficient is 2.74, which inflates the standard error of the income elasticity by a factor 3.49 (considering only the intra-group correlation of total expenditures). A regression taking account of the clustering into groups by the Cluster Robust Variance Estimator (equation ( 6)) gives the same estimate of the income elasticity with a comparable inflating factor of the s.e. (2.99) 8 . A regression on the population aggregated into 88 cells gives a smaller elasticity with a much larger s.e. of 6 A test of the representatbility of information in efficent sub-populations could rely on the representation in these sub-populations of observations pertaining to each cell of a choosen grouping of the population. Note that a different proportion of each group in an efficient sub-population compared to the whole population may not bias the estimation, at least if the grouping is not endogenous, -i.e. made at random as concerns the residuals of the estimation corresponding to the set of latent variables-since it corresponds to a weighted estimation with weight differing between the two data-sets 7 Full prices defined in Gardes, 2019. 8 Note that the inflation factor would be much larger for an estimation on smaller subpopulations of 105 households (1% of the whole sample) which gives also an average income elasticity close to the estimation on the whole population (see Table 1) but with a much larger dispersion.

7

Documents de travail du Centre d'Economie de la Sorbonne -2021.22 0.0636 (see [START_REF] Hannan | Estimation from Grouped Observations[END_REF], for a precise analysis of biases from grouped observations).

These results show that the correction obtained by regressing on efficient subpopulations gives the same inflation factor as the correction of clustering effects by CRVE or the Greenwald-Moulton' formula. Similar results are obtained using the efficient sub-populations method for Housing, Transport and Other expenditures (inflating factors equal to 2.18, 2.10 and 1.68), CRVE (2.76, 2.37 and 3.71) and Greenwald-Moulton's (2.05, 2.89 and 2.00). I conclude that the estimation on efficient sub-populations estimates unbiased coefficients (compared to the estimation on the whole sample) but inflates the estimated variances by the same magnitude than the correction of clustering effects by Greenwald's equation. One can also remark that the correction of clustering effect, for instance by the Greenwald-Moulton's formula, diminishes by 40% when estimating on efficient sub-populations, which indicates that the definition of an efficient size of the data-set eliminates a large proportion of the spatial autocorrelations. On the contrary, grouping the data-set into homogenous cells changes a lot the estimated parameters because of the loss in degrees of freedom, and increases the estimated variances in a proportion much larger than all correction methods applied to individual data. The discussion of the relationship between the number of clusters and the spatial autocorrelation of the residuals must also take into account the change in the autocorrelation of the covariates which could increase together with the autocorrelation of the residuals, both increasing the inflating factors of estimated variances9 .

Conclusion

Two methods have been proposed to estimate correctly the variances of the parameters of a linear model on a numerous data-set. Both give rise to much greater standard errors, with close inflating factors varying between 2 and 3 for the estimation of Engel curves for four aggregate commodities. The correction by means of clustering suffers from the arbitrary choice of the grouping. Also, clustering is not always possible if discriminant variables are not available to operate an efficient grouping. Besides, some among the formulas used in the literature in the correction of estimated variances apply only in case of one explanatory variable. The correlated changes, across different data-sets, of the spatial correlations of the residuals and of the covariates necessitates an important future study in order to estimate the relation between the structure of the population and the inflating factor of estimated variances. Another interesting problem is whether there exists an aggregate effect of various possible homogeneous groupings introducing independently spatial autocorrelations in the estimation.

On the contrary, the estimation on efficient sub-populations containing enough statistical information is based on a statistical choice which does not suppose an a-priori structure for a grouping of the data-set. Also, the second method is valid whatever the number of explanatory variables. Besides, the estimation on similar data-sets as concerns their size and structure allows to compare estimates performed on independent data-sets, giving rise to a normalization of standard errors given by these estimations. Therefore, in order to compare estimates corresponding to independent data-sets, a convenient method consist in taking at random subsets of each independent data-set corresponding to the same statistical rule defining the efficient sub-population. The second method thus applies to any econometric analysis, by OLS or other statistical models, and also to the estimation of a system of equations.

An interesting question is whether it is possible to split optimally the original sample applying the Borsuk-Ullam theorem: this necessitates the estimation of the hyperplans which split the sample into two similar sub-samples (which have been done at random in our empirical application).

Both methods convey the same information about the bias of the estimated variances, so that these corrected variances or the variances obtained on limited sub-populations should be used on micro-data for any test of significance or specification using the variance-covariance matrix.

Appendix A: Correction of the biases issued from a spatial autocorrelation

Kloek's analysis

Kloek considers the case where all regressors are constant within groups and groups have the same size m. In that case, the ratio between the true GLS variance and the OLS is:

f ( β) = 1 + ρ(m -1) (4)
where ρ is the intra-class correlation of the residual. If for example ρ = 0.05 and m = 50, the factor of under-estimation of standard errors is √ f = 1.86.

Greenwald's formulas

Greenwald generalizes the analysis of the linear model: y = Xβ +e and his basic formula evaluates the bias between the true variance-covariance matrix and the usual estimate in the OLS estimation of a linear model Ĉ = σ 2 (X X) -1 :

E( Ĉ) -C = E(σ 2 )(X X) -1 -(X X) -1 X V X(X X) -1 (5)
with V = E(ee ) = σ 2 A, σ 2 = 1 T tr(V ), T the sample size and A a positive definite T × T matrix indicating the spatial correlations between the residuals.

The Cluster-Robust Variance Estimator (CRVE Stata option for clustered data) used in Table 2 uses a similar formula with a normalization of the estimate depending on the degrees of liberty 10 . It applies whatever the number of covariates with a large number of clusters g = 1 to G and u g the vector of residual inside cluster g:

V clusters (β) = (X X) -1 G G -1 N -1 N -k g X g ûg û g X g (X X) -1 ( 6
)
where k is the number of covariates X, G the number of clusters and N the size of the whole population.

Special cases

Greenwald's case

In the case of the variance component model (observations organized into J groups j with size m j ) with multiple explanatory variables indexed by k, the proportional bias in variances writes:

E( Ĉkk ) = σ 2 T j ρ j ρ j (k)(m j -1) (7)
with ρ j the average correlations within groups of the errors and ρ j (k) of the explanatory variables. Therefore, if the groups within which the errors are correlated tend to be also relatively homogeneous in terms of the set of explanatory variables, the standard errors of the coefficients tend to be understated11 .

6

  Documents de travail du Centre d'Economie de la Sorbonne -2021.22

Table 1

 1 Estimates of income elasticities(French INSEE survey, 2000)

	Data	Whole	Sub-Populations	Efficient
		Population	(1%)	Sub-Populations
	Food
	Average Income Elasticity:	0.5884	0.5859	0.5883
	Empirical s.e.	-	0.0864	0.0128
	Estimated s.e.	0.0072	0.0740	0.0134
	Number of households	10251	103	2982
	Housing
	Average Income Elasticity:	0.8339	0.8400	0.8347
	Empirical s.e.	-	0.1051	0.0258
	Estimated s.e.	0.0077	0.0796	0.0213
	Number of households	10251	103	1350
	Transport
	Average Income Elasticity:	0.8143	0.8157	0.8141
	Empirical s.e.	-	0.0898	0.0263
	Estimated s.e.	0.0088	0.1137	0.0208
	Number of households	10251	103	1836
	Other expenditures
	Average Income Elasticity:	1.244	1.284	1.253
	Empirical s.e.	-	0.1033	0.0345
	Estimated s.e.	0.0082	0.0815	0.0303
	Number of households	10251	103	759
	Note: Empirical s.e. are calculated across 500 sub-populations (with a fixed size) taken at
	random. Data-set: French INSEE surveys: the Family Expenditure Survey (FES, INSEE
	BDF 2001) and the Family Time Budget (FTB, INSEE BDT 1999).

Table 2

 2 Various estimates of the income elasticity of food (FES,France, 2000)

	Data	Whole sample	Correction	Cluster-Robust	Grouping	Efficient
			of the s.e.	Variance Estimator	into 88 cells sub-populations
	Elasticity	0.5884	0.5884	0.5884	0.6920	0.5883
	s.e.	0.0072	0.0119	0.0216	0.0636	0.0134
	Sample size	10251	10251	10251	88	2982
	Method	OLS	Greenwald's	Equation (6)	OLS	OLS
			equation (7)
	Note: Estimates of the Opportunity Cost by country, US dollars 2015, Cobb-Douglas specifi-
	cation. data-set: French INSEE surveys: the Family Expenditure Survey (FES, INSEE BDF
	2001) and the Family Time Budget (FTB, INSEE BDT 1999).

The efficent sub-populations are defined in section 2.2 as giving rise to a difference in the parameter smaller than 5% for at least 95% of samples having a definite size.

Documents de travail du Centre d'Economie de la Sorbonne -2021.22

Performed by Julien Boelaert.

This theorem has been for the first time used for a similar purpose by the French-american astrophysicist, Gérard de Vaucouleurs, who discovered, in the

50's, an unexpected concentration of galaxies into big galaxy clusters which cannot be explained within the traditional big-bang model: the universe is not as homogenous as was supposed by models based on the big-bang event sending all the matter uniformally in all directions. Tiny initial dissymetries (observed lately as waves in the initial states of the universe) were sufficient to create this concentration by the laws of celestial mechanics. As a consequence, these galaxy clusters exert gravitational effects which accelerate the galaxies.[START_REF] Hannan | Estimation from Grouped Observations[END_REF] The concentration of similar individuals taking similar choices may exert an influence on other individuals by changing the costs corresponding to different socio-economic locations, and eventually creates mimetic behavior which increases the concentration. 4 Documents de travail du Centre d'Economie de la Sorbonne -2021.22

The product of the intra-correlation of the covariate and the residuals decreases much (from 0.114 to 0.013 for Food expenditure) when the number of clusters is diminished from 88 to 5 (grouping according to the head's age).

This formula is attributed to[START_REF] Liang | Longitudinal Data Analysis Using Generalized Linear Models[END_REF] by MacKinnon (2016).

Note that in this formula the covariance matrix corresponding to an explanatory variable depends only on the spatial autocorrelation of that variable, and not on its relations with other covariates. 10 Documents de travail du Centre d'Economie de la Sorbonne -2021.22

Moulton's formulas

Moulton applies in his empirical studies (1986,1987,1990) various simpler forms of these formulas and formulate tests of the biases. These correspond to the formula obtained by [START_REF] Scott | The Effect of Two-Stages Sampling on Ordinary Least Squares Methods[END_REF] for a two-stage sampling in the simple linear model and Greenwald formula [START_REF] Leamer | Specification Searches: Ad Hoc Inference with Non Experimental Data[END_REF] for clusters of equal size (variance of the clusters size V (m j ) = 0:

with m the average clusters size and ρ x the intra-correlation of the unique regressor:

where the indices indicates individual j in group i. Moulton analyses (section 3 of his 1990 paper) the model y = Xβ + Zδ + u where Z is a set of dummies indicating the groups. He proposes to apply a Breuch-Pagan test for the hypothesis that variances of the dummy coefficients V (δ i ) = σ i are non-nul and uses the correct variance of the OLS estimate of equation ( 2

He concludes to an under-estimation of standard errors by 1.3 to 7.

His 1990 article writes the true covariance matrix of the OLS estimates corresponding to equation (2):

where N = X ZZ X(X X) -1 which simplifies as in the Kloek case to the formula of the true OLS variance-covariance matrix :

with m the common size of all groups and ρ the common value of the correlation between residuals in each group. These formulas show that the bias in variances increases with this spatial correlation and with the group size, which depend on the total number of observations in the data-set and on the number of groups. cooking time (FTB) and food consumption (FES), cleaning and home maintenance and dwelling expenditures (including imputed rent), clothing maintenance and clothing expenditures, education time and education expenditures, health care time and health expenditures, leisure time and leisure expenditures, transport time and transport expenditures, miscellaneous time use and miscellaneous goods and services. Time uses for all selected activities are regressed on the households' characteristics for all observation units in FTB survey and these estimations serve to predict the time spent on these activities for the corresponding units in the FES survey.

Appendix C: Results for other commodities