Paolo Delle Site
email: paolo.dellesite@unicusano.it

Karim Kilani

Valerio Gatta
email: valerio.gatta@uniroma3.it

Edoardo Marcucci
email: edoardo.marcucci@himolde.no

André De Palma
email: andre.de-palma@ens-paris-saclay.fr

Estimation of Logit and Probit models using best, worst and best-worst choices

Keywords: Best-worst choices, Congestion charge, Logit, Probit, Random utility model, Strict log-concavity

The paper considers models for best, worst and best-worst choice probabilities, that use a single common set of random utilities. Choice probabilities are derived for two distributions of the random terms: i.i.d. extreme value, i.e. Logit, and multivariate normal, i.e. Probit. In Logit, best, worst and best-worst choice probabilities have a closed form. In Probit, worst choice probabilities are simply obtained from best choice probabilities by changing the sign of the systematic utilities. Strict log-concavity of the likelihood, with respect to the coefficients of the systematic utilities, holds, under a mild necessary and sufficient condition of absence of perfect multicollinearity in the matrix of alternative and individual characteristics, for best, worst and best-worst choice probabilities in Logit, and for best and worst choice probabilities in Probit. The assumption of substitutability between best and worst choices is tested with data on mode choice, collected for the assessment of user responses to urban congestion charging policies. The numerical results suggest significantly different preferences between best and worst choices, even accounting for scale differences, in both Logit and Probit models. Worst choice data exhibit coefficient attenuation, less pronounced in Probit than in Logit, and higher mean values of travel time savings with larger confidence intervals.

Introduction

Motivation

Researchers use increasingly stated preference data providing partial or full rankings, because they can provide augmented information with respect to methods based on asking the favorite alternative only.

When data on full rankings are available, these are frequently analyzed, for the higher ease of estimation, with the use of the rank-ordered Logit model, introduced by [START_REF] Beggs | Assessing the potential demand for electric cars[END_REF]. An alternative model, of more recent use, is the rank-ordered Probit, introduced by [START_REF] Hajivassiliou | Classical estimation methods for LDV models using simulation[END_REF]. Examples in transportation of the rank-ordered Logit are found in [START_REF] Beggs | Assessing the potential demand for electric cars[END_REF] and [START_REF] Ben-Akiva | Analysis of the reliability of preference ranking data[END_REF]. An example of the use of the rank-ordered Probit is in Nair et al. (2018a).

However, ranking data can be based on best and worst choices only. [START_REF] Louviere | A comparison of importance weights and willingness-to-pay measures derived from choice-based conjoint, constant sum scales and best-worst scaling[END_REF] suggest that judging items at extremes provides advantages in cognitive efforts on discriminating among items of intermediate importance. In addition, the questionnaire task appears easier for the respondent. Thus, it is also suggested the approach to be of interest to applications concerned with complete rankings, obtained by iterating best-and-worst tasks. Examples in transportation of the use of best and worst data are rare [START_REF] Beck | I can't believe your attitude: a joint estimation of best worst attitudes and electric vehicle choice[END_REF]. The present paper is concerned with this type of partial ranking. [START_REF] Louviere | Best-worst scaling: a model for largest difference judgments[END_REF] and [START_REF] Finn | Determining the appropriate response to evidence of public concern: the case of food safety[END_REF] were the first to propose a discrete choice task in which an individual is asked to indicate the least preferred item in a choice set, in addition to indicating the traditional most preferred one. This data collection approach is now called best-worst scaling and is applied in many fields [START_REF] Scarpa | Exploring scale effects of best/worst rank ordered choice data to estimate benefits of tourism in Alpine grazing commons[END_REF], provide several references). Items can have different nature [START_REF] Flynn | Best-worst scaling: theory and methods[END_REF][START_REF] Louviere | Best-Worst Scaling: Theory, Methods and Applications[END_REF]Marley & Flynn, 2015). Initially, they were restricted to attitudes, public policy goals, brands, or anything that does not require a detailed description. Later, they were extended to more complex items, such as attributes and levels describing a single alternative, or complete profiles of multiple alternatives that are standard in choice modeling. The paper deals with this latter case.

Best and worst choices have long been analyzed using random utility models (RUM). Traditionally, distinct models have been considered for the three types of choice: best only, worst only, and best-worst. Which model should be used for best choices, which one for worst choices and which one for best-worst choices remains mainly an empirical question. However, the models used in empirical applications are often mainly driven by tractability reasons.

An i.i.d. extreme value (EV) distribution of the random terms is assumed. For best choices, the ordinary multinomial Logit (MNL) is used. For worst choices, the so-called reverse Logit model is used (see [START_REF] Anderson | Reverse discrete choice models[END_REF]. This model, which provides worst choice probabilities in MNL form with systematic utilities having a minus sign, originates from a form of the total utility where the random term is subtracted (instead of being added) to the systematic part. For best-worst choices, [START_REF] Finn | Determining the appropriate response to evidence of public concern: the case of food safety[END_REF] have proposed the so-called maximum difference (max-diff) model. In this model, the choice maker maximizes the difference in the utilities, with random terms specific of the alternative pair.

As elucidated by [START_REF] Louviere | Best-Worst Scaling: Theory, Methods and Applications[END_REF], one should estimate model forms coherent with the process used by individuals to make choices when asked. Traditionally, two such processes are considered: (i) simultaneous best and worst choice, and (ii) sequential (best-then-worst, or, less frequently, worst-then-best) choice. Thus, one should pay attention to task structures, and decide whether, for an interviewer-administered or a web-based survey, an assumption of simultaneous best and worst choices is feasible and meaningful, or the survey uses a sequential task administration process that consequently requires sequential modeling.

In practice, in the case of simultaneous best and worst choice, estimation is carried out using a maxdiff model, or by data pooling. In the latter approach, one expands the best choice dataset with worst choice data where a minus sign precedes the value of each attribute of the utilities. One estimates then a MNL with the joint likelihood. Both best only and worst only choices are modeled with reference to the complete choice set.

In the case of sequential best-then-worst choices, data pooling is common. This approach is similar to the simultaneous case, but one considers, in constructing the joint likelihood, that the worst choice is performed from a different set, because only the alternatives that remain after the best choice are considered when deciding which alternative is worst. Worst only choices are modeled with reference to this reduced choice set. The case of worst-then-best choices is analogous.

In all of the above cases, estimation is accomplished using standard software for MNL. Data pooling machinery, for both the simultaneous and the sequential case, is implemented in the most recent version of one econometric software 1 .

In the paper, we restrict our attention to simultaneous best and worst choice. We consider two consistent RUMs according to the definitions given in [START_REF] Marley | Some probabilistic models of best, worst, and best-worst choices[END_REF]. Consistency implies that the best, worst and best-worst choices are all modeled according to a single common set of utilities. In the first model -the additive model -the random terms have a plus sign, while in the second model -the reverse model -they have a minus sign.

In best-worst scaling, there are a number of open issues. Some are theoretical. Some are empirical. The following are tackled here. On the theoretical side, results on worst and best-worst probabilities need to be extended to the multivariate normal distribution of the random terms, i.e. the Probit case. Conditions for strict log-concavity of the likelihood associated with best, worst and best-worst choice probabilities is another issue. Strictness of log-concavity is important since it ensures uniqueness of the maximum. On the empirical side, the data augmentation perspective reckons that best and worst choices have the potential to provide better estimates of the utility coefficients than best choices only. The substitutability of best and worst data, that is taken for granted in 1 NLogit 6, by Econometric Software, Inc.

3

this perspective, has been called into question. The following section provides a review of the relevant literature on these issues.

Literature background

For the i.i.d. EV, i.e. Logit, distribution of the random terms, the closedform expressions of the best, worst and best-worst choice probabilities in the additive and reverse models are known from [START_REF] Marley | Some probabilistic models of best, worst, and best-worst choices[END_REF] and de [START_REF] De Palma | Relations between best, worst, and best-worst choices for random utility models[END_REF]. In the additive model, the best choice probabilities are ordinary MNL. The worst choice probabilities are expressed as an alternating sum of best choice probabilities (also derived by [START_REF] Fok | A rank ordered logit model with unobserved heterogeneity in ranking capabilities[END_REF][START_REF] Vann Ophem | Multichoice logit: modeling incomplete preference rankings of classical concerts[END_REF]. The best-worst choice probabilities are a product of best choice and worst choice probability.

For the multivariate normal, i.e. Probit, distribution of the random terms, no investigation has been carried out yet on the expressions of worst and bestworst choice probabilities. Necessary and sufficient conditions for strict logconcavity of the likelihood are available, to the best of our knowledge, only for the two cases of best probabilities of the multinomial Logit [START_REF] Mcfadden | Conditional logit analysis of qualitative choice behavior[END_REF], and best probabilities of the binomial Probit [START_REF] Wedderburn | On the existence and uniqueness of the maximum likelihood estimates of certain Generalised Linear Models[END_REF][START_REF] Daganzo | Multinomial Probit: the Theory and its Application to Demand Forecasting[END_REF][START_REF] Amemiya | Advanced Econometrics[END_REF]. For both distributions (Logit and Probit), there is no example, so far, of estimation that exhibits consistency between best, worst and bestworst choices.

The substitutability assumption between best and worst choice data has been questioned by a few authors. Louviere et al. (2015, Ch. 14), with examples in the marketing area, and [START_REF] Ben-Akiva | Analysis of the reliability of preference ranking data[END_REF] and [START_REF] Giergiczny | Testing and stability of utility parameters in repeated best, repeated best-worst, and one-off best-worst studies[END_REF], with examples in the transportation area, find statistically significant differences between depth-specific coefficients, i.e. coefficients based on best choice data only and coefficients based on worst choice data only. The results restrict to the Logit.

The approaches used by these authors are different. [START_REF] Louviere | Best-Worst Scaling: Theory, Methods and Applications[END_REF] estimate a reverse Logit model with worst choices. [START_REF] Ben-Akiva | Analysis of the reliability of preference ranking data[END_REF] and [START_REF] Giergiczny | Testing and stability of utility parameters in repeated best, repeated best-worst, and one-off best-worst studies[END_REF] use data based on full rankings, and compare the results of rank-ordered Logit models estimated with the full dataset and with conditional datasets, i.e. datasets obtained by eliminating the first-best alternative, the first-and the second-best alternative, and so on. The main result common to these studies is that the attenuation of the values of the coefficients, from best to worst choices, is a manifestation of the higher uncertainty (lower scale parameter) that affects ranking preferences of the individuals at lower depths (i.e. for worst choices). [START_REF] Giergiczny | Testing and stability of utility parameters in repeated best, repeated best-worst, and one-off best-worst studies[END_REF] provide the only analysis so far of the differences between best and worst choices in terms of value of travel time savings (VTTS). The case they address relates to rail services.

Yan & Yoo (2014) find, with simulation experiments, that the estimates produced by the rank-ordered Logit model can show coefficient attenuation if the true distribution of the random terms deviates even slightly from the i.i.d. EV distribution. This result has motivated the papers by Nair et al. (2018a,b), where the performances of the rank-ordered Probit are investigated. Nair et al. (2018b) find the rank-ordered Probit more robust than the rank-ordered Logit in terms of stability of the coefficients across ranking depths.

Contribution

In the paper, we consider consistent additive and reverse models under two distributions of the random terms: i.i.d. EV, i.e. Logit, and multivariate normal, i.e. Probit.

In the theoretical part of the paper, we address two issues. The first is the derivation of probabilities for best, worst and best-worst choices. After a presentation of preliminary results extending to any additive RUM, we review the best, worst and best-worst choice probabilities of the two models (additive and reverse) in the Logit case. For the Probit, we provide new results for worst choice probabilities.

The second issue is the strict log-concavity, with respect to the coefficients of the systematic utilities, of the choice probabilities and of the associated likelihood functions. In particular, for the Logit, we prove strict log-concavity of the likelihood function of MNL under milder assumptions than those in [START_REF] Mcfadden | Conditional logit analysis of qualitative choice behavior[END_REF]. For the Probit, we provide (mild) conditions for strict log-concavity of the likelihood function of multinomial Probit (MNP). Clearly, these results have a wider impact than that on best-worst scaling only.

At the same time, the paper aims to test, with a numerical example of practical relevance in transportation, the implict assumption about the substitutability of best and worst data that is taken for granted in the traditional data augmentation perspective. The test extends to Logit and Probit, and is carried out in terms of coefficients of the systematic utilities and VTTS alike.

In the empirical part, the data are presented. They relate to a statedpreference survey carried out in Rome in 2015. The choice of the transportation mode among three alternatives -car, powered-two-wheeler (PTW) and public transportation -is considered. Respondents in the sample were asked to state their best and worst choice. The survey was aimed to assess the responses of car users to charging measures, similar to those implemented in London, Stockholm and Milan. This policy is currently being planned by the municipality. Then, we present the results of the estimation of additive and reverse Logit and Probit models, each with best choice data only, worst choice data only, and best-worst data.

The remainder of the paper is organised as follows. Section 2 and 3 are theoretical, and present general assumptions and identities. Section 4 and 5 deal with specific theoretical results for Logit and Probit models, respectively. Section 6 presents the data and the estimation results. Section 7 concludes. Proofs of lemmas, propositions and theorems are relegated to appendices.

Main assumptions

Consider an individual facing a choice of the best and/or the worst alternative among a finite set of alternatives X ≡ {1, • • • , n}, where n ≡ |X| denotes the cardinality of X, with n ≥ 3. Individual preferences are represented by an additive random utility model (ARUM). We assume there exists a vector of systematic utilities v ≡ (v 1 , • • • , v n ), and a vector of random utilities ≡ ( 1 , • • • , n ) such that the utility of alternative z takes the following additive form

U z = v z + z , z ∈ X.
(1)

We assume that admits a positive and continuous probability density function with respect to Lebesgue measure on R n .2 Let F (•) be the cumulative distribution function of the vector , whose arguments (realizations) are denoted by t

≡ (t 1 , • • • , t n ) ∈ R n .
With the above assumptions, non-coincidence holds, i.e. P(U z = U z ) = 0, z, z ∈ X, z = z, where P(•) is a probability measure. In words, ties among utilities occur with zero probability.

Identities

Best choice probabilities

For any non-empty subset Y of X and any alternative x of Y , the probability that alternative x is the best choice in Y is given by

B Y (x; v) ≡ P (U x ≥ U z , z ∈ Y ) , x ∈ Y ⊆ X, v ∈ R n .
(2)

All these probabilities are forming a system of best choice probabilities. The probabilities in the whole set can be obtained by performing the following single integration (S. P. et al., 1992, Eq 2.27)

B X (x; v) = +∞ -∞ ∂F (u1 X -v) ∂t x du, x ∈ X, v ∈ R n , ( 3
)
where 1 X is the all-ones n-vector. The best choice probabilities on any set Y ⊆ X can be obtained by performing the following limit3

B Y (x; v) = lim vz→-∞ ∀z∈X\Y B X (x; v), x ∈ Y ⊆ X, v ∈ R n . (4)

Worst choice probabilities

Consider now the system of worst choice probabilities defined by

W Y (x; v) ≡ P (U z ≥ U x , z ∈ Y ) , x ∈ Y ⊆ X, v ∈ R n . ( 5
)
A general identity provided in de Palma et al. (2017, Theorem 5) allows to recover the worst choice probabilities from the best choice probabilities. Accordingly, worst choice probabilities are given by an alternating sum of best choice probabilities

W X (x; v) = {x}⊆Y ⊆X (-1) |Y |-1 B Y (x; v), x ∈ X, v ∈ R n .
(6)

Best-worst choice probabilities

Now, for two different alternatives x and y of X, consider the probability that, jointly, x is the best in X and alternative y is the worst in X. Formally, best-worst choice probabilities are defined by

BW X (x, y; v) ≡ P (U x ≥ U z ≥ U y , z ∈ X) , x, y ∈ X, x = y, v ∈ R n . ( 7
)
Let 1 Y denotes the n-vector with zth component being one if z ∈ Y and zero if z ∈ X \ Y , with Y ⊆ X. We have established this new identity:

Lemma 1. The best-worst choice probabilities are, for any x, y ∈ X, x = y, and any v ∈ R n , given by:

BW X (x, y; v) = {y}⊆Y ⊆X\{x} (-1) |Y |-1 B X,Y (x, y; v), (8)
where:

B X,Y (x, y; v) ≡ ∞ 0 -∂B X (x; v + t1 Y ) ∂v y dt. ( 9
)
Proof. See Appendix A.

B X,Y (x, y; v) represents the probability that, jointly, x is the best alternative in X and y is the best alternative in Y (see [START_REF] De Palma | Relations between best, worst, and best-worst choices for random utility models[END_REF].

Log-likelihood identities and properties

We consider the case where the systematic part of the utility of alternative z has the following linear specification

v z = ξ z θ, z ∈ X, ( 10
)
where ξ z is a row vector of alternative and individual characteristics of R K , and θ is a column vector of coefficients of R K to be estimated. The log-likelihood associated with the observation of x as the best (resp.

worst) choice in Y is L B Y (x) ≡ ln B Y (x; v) (resp. L W Y (x) ≡ ln W Y (x; v)), x ∈ Y ⊆ X. In the sequel, ∇ θ L B Y (x) (resp. ∇ θ L W Y (x)) denotes the row gradient of L B Y (x) (resp. L W Y (x)) with respect to θ. Moreover, we denote by ∇ 2 θ L B Y (x) and ∇ 2 θ L W X (x)
the corresponding Hessian matrices. The vector argument v is omitted for the sake of notational convenience. Evaluating the gradient vector and the Hessian matrix of the log-likelihood functions is critical for estimation purpose.

For any row vector ξ of R K , we shall use the vectorial notation ξ 2 ≡ ξ ξ, where the prime denotes vector transpose. Using the following alternating weights

4 ω X,Y (x) ≡ (-1) |Y |-1 B Y (x; v) W X (x; v) , x ∈ Y ⊆ X, ( 11
)
we establish a result which provide identities relating to the gradient vectors and the Hessian matrix of the log-likelihood functions associated with best and worst choices.

Lemma 2. For any ARUM with linear specification ( 10), for any x ∈ X, we have:

∇ θ L W X (x) = {x}⊆Y ⊆X ω X,Y (x)∇ θ L B Y (x). ( 12
)
Moreover,

∇ 2 θ L W X (x) + ∇ θ L W X (x) 2 = {x}⊆Y ⊆X ω X,Y (x) ∇ 2 θ L B Y (x) + ∇ θ L B Y (x) 2 . ( 13
)
Proof. See Appendix A.

This lemma shows that, for general ARUMs, the gradient of the log-likelihood associated with worst choice is an alternating weighted sum of gradients of the log-likelihood associated with best choice. The Hessian associated with worst choice involves a similar alternating weighted sum of best choice Hessians plus square matrices obtained from differences between best and worst choice gradients.

Reverse models

Now, we introduce a parallel RUM where the roles of best and worst choice probabilities are interchanged. Following [START_REF] Anderson | Reverse discrete choice models[END_REF], we associate to every RUM its corresponding reverse ARUM, where the utilities are now given by

U R z = v z -z , z ∈ X. ( 14
)
Note the termz , whereas in the previous model (see Eq. ( 1)) we had + z .

Clearly, the best choice best choice probabilities, in the reverse model, de-

fined by B R X (x, v) ≡ P U R x ≥ U R z , z ∈ X , satisfy the following B R X (x, v) = W X (x, -v), x ∈ X, v ∈ R n . ( 15
)
Conversely, the worst choice probabilities in the reverse model, defined by

W R X (x, v) ≡ P U R z ≥ U R x , z ∈ X , satisfy W R X (x, v) = B X (x, -v), x ∈ X, v ∈ R n . ( 16
)
Finally, the best-worst choice probabilities in the reverse model, defined by BW R X (x, y; v)

≡ P U R x ≥ U R z ≥ U R y , z ∈ X , are related to the best-worst choice probabilities in the original model by BW R X (x, y; v) = BW X (y, x; -v), x, y ∈ X, x = y, v ∈ R n . ( 17
)
4. Logit

Best, worst and best-worst choice probabilities

Assume the random variables x , x ∈ X, follows independent standard Gumbel distributions. That is, the cumulative distribution function of is given by

F (t) = exp(- z∈X e -tz ), t ∈ R n . ( 18
)
Differentiating ( 18), then applying ( 3) and ( 4), best choice probabilities are obtained by integration yielding the ordinary multinomial Logit model (MNL) formula5

B Y (x; v) = e vx z∈Y e vz , x ∈ Y ⊆ X, v ∈ R n . ( 19
)
It is worth to rewrite the above Logit expression as follows

B Y (x; v) = e vx-Λ Y (v) , x ∈ Y ⊆ X, v ∈ R n , ( 20
)
where Λ Y (v) ≡ ln z∈Y e vz is the Log-Sum-Exp function. Note the following property (Williams-Daly-Zachary theorem; [START_REF] Mcfadden | Econometric Models of Probabilistic Choice[END_REF])

∂Λ Y (v) ∂v z = B Y (z; v) , z ∈ Y ⊆ X; 0, z ∈ X \ Y. ( 21
)
Applying Identity ( 6), the corresponding worst choice probabilities are obtained (also found in [START_REF] Fok | A rank ordered logit model with unobserved heterogeneity in ranking capabilities[END_REF][START_REF] Vann Ophem | Multichoice logit: modeling incomplete preference rankings of classical concerts[END_REF])

W X (x; v) = {x}⊆Y ⊆X (-1) |Y |-1 e vx z∈Y e vz , x ∈ X, v ∈ R n . ( 22
)
Note that the alternating weights defined by ( 11) are explicitly given by

ω X,Y (x) ≡ (-1) |Y |-1 e -Λ Y (v) {x}⊆Z⊆X (-1) |Z|-1 e -Λ Z (v) , x ∈ Y ⊆ X. ( 23
)
Best-worst choice probabilities are, for any x, y ∈ X, x = y, and any v ∈ R n , given by6

BW X (x, y; v) = e vx z∈X e vz {y}⊆Y ⊆X\{x} (-1) |Y |-1 e vy z∈Y e vz . ( 24
)
When we have three alternatives only, it is easily shown that the additive Logit with B&W choices is exactly the same as the rank-ordered Logit. In fact, with reference to alternatives 1 and 2, we have:

BW {1,2,3} (1, 2) = e v1 e v1 + e v2 + e v3 1 - e v2 e v2 + e v3 = e v1 e v1 + e v2 + e v3 e v3 e v2 + e v3 = B {1,2,3} (1) B {2,3} (3) , ( 25
)
which is the rank-ordered Logit model, where best choice probabilities are exploded.

Our results can be put into perspective with the impossibility theorem by [START_REF] Luce | Preference, utility and subjective probability[END_REF]. The theorem assumptions are that: (i) ranking probabilities are derived from a common set of utilities, (ii) the probability of the ranking from best to worst is obtained from explosion of best choice probabilities, and the probability of the ranking from worst to best is also obtained from explosion of worst choice probabilities. The theorem states that, under these assumptions, best choice probabilities should be equal across alternatives, and worst choice probabilities should also be equal across alternatives, an impossibility result. Notice that not all the assumptions of this theorem apply to the framework here. In the additive Logit model, explosion of best probabilities, for computation of the probability of the ranking from best to worst, holds because best probabilities are MNL, but explosion of worst probabilities, for computation of the probability of the ranking from worst to best, does not because they are not MNL. With three alternatives, we do not compute the probability of the ranking from worst to best by exploding worst probabilities, but we use the formula of best-worst choice probability.

Log-likelihood

Let us recall a classical result in the econometrics of the MNL (see, e.g., [START_REF] Mcfadden | Conditional logit analysis of qualitative choice behavior[END_REF][START_REF] Ben-Akiva | Discrete Choice Analysis: Theory and Application to Travel Demand[END_REF]. Note that the log-likelihood corresponding to the observation of the best choice is, using (20), given by Therefore,using (21), its gradient satisfies

L B Y (x) = v x -Λ Y (v), x ∈ Y ⊆ X.
∇ θ L B Y (x) = ξ x -ξ Y , x ∈ Y ⊆ X, ( 26
)
where

ξ Y ≡ z∈Y B Y (z; v) ξ z , Y ⊆ X. ( 27
)
The corresponding Hessian matrix is given by

∇ 2 θ L B Y (x) = H Y (v) ≡ - z∈Y B Y (z; v) (ξ z -ξ Y ) 2 , Y ⊆ X. ( 28
)
It is denoted by H Y (v) since it does not depend on the alternative x. From (28), the Hessian matrix of the best choice log-likelihood function is a semi-definite negative matrix, because it is, formally, the negative of a covariance matrix.

For the worst choice, the formulas are more intricate. We shall use the following notation

ξ W X (x) ≡ {x}⊆Y ⊆X ω X,Y (x)ξ Y , x ∈ X. ( 29
)
Note that an alternative form of ξ

W X (x) is ξ W X (x) = z∈X θ X (x, z; v)ξ z , x ∈ X, ( 30
)
where

θ X (x, z; v) ≡ {x,z}⊆Y ⊆X (-1) |Y |-1 B Y (x; v)B Y (z; v) W X (x; v) , x, z ∈ X. ( 31
)
We have:

Proposition 1. For a MNL with linear specification (10), for any x ∈ X, we have:

∇ θ L W X (x) = ξ x -ξ W X (x). ( 32
)
Moreover, the corresponding Hessian matrix is:

∇ 2 θ L W X (x) = {x}⊆Y ⊆X ω X,Y (x) H Y (v) + ξ Y -ξ W X (x) 2 . ( 33
) Proof. See Appendix B.
It is far from being clear from (33) whether the Hessian matrix is semidefinite. At this stage, we will go even further and establish Proposition 2 which will be helpful to find some mild conditions ensuring that the log-likelihood functions are strictly concave (Theorem 1). Strict concavity ensures uniqueness of the maximum.

We denote by d

≡ (d 1 , • • • , d n-1 ) ∈ R n-1 the vector whose components are d z = v z -v n , z = 1, • • • n-1.
We are using a Prékopa's theorem (Prékopa, 1973) which states that if the probability density function f (•) is positive and strictly logarithmic concave function in an open convex set of R n-1 , then the associated cumulative distribution function F (•) is also strictly logarithmic concave in the same set. We have: Proposition 2. The MNL best choice probabilities B X (x; v), given by the Logit formula ( 19), and the worst choice probabilities W X (x; v) given by ( 22), are strictly log-concave in d, for any x ∈ X.

Proof. See Appendix B.

We consider now a sample of N observations indexed by i = 1, • • • , N . The observation index will be in general omitted in the notations below. Each observation consists of a best and a worst choice within the given choice set X (not necessarily the same for all the observations in the sample). Every observation provides log-likelihood functions, L B X (x) and L W X (y), where x, y ∈ X are the observed best and worst choices, respectively. They contribute to the total best and worst log-likelihood functions, L B and L W , which are obtained by summing-up, over the whole sample, those terms.

We denote by M the matrix obtained by concatenating the rows of the N observation-specific (n i -1) × K matrices whose rows are ξ z -ξ n i , z ∈ X i \ {n i }, where X i denotes the choice set of i and n i its cardinality. The matrix M has N i=1 n i -N rows and K columns.

Theorem 1. In the linear-in-the-coefficients MNL model, the total log-likelihood functions of the best choices L B and of the worst choices L W are strictly concave in θ iff the matrix M has full rank.

Proof. See Appendix B.

Note the high simplicity of these necessary and sufficient conditions for strict log-concavity of both L B and L W . Recall that the conditions in McFadden (1974, Axiom 5) for the global concavity of L B require the full rank of the N i=1 n i × K matrix whose rows are ξ z -ξ X i . Since ξ X i depends on θ, McFadden's conditions are less easy to check.

We have therefore proved that the likelihood functions associated with best and worst choices have, under the stated conditions, a unique maximum, provided one exists. Similar results can be derived for best-worst choices, on the basis of (24), by preservation of strict log-concavity under product.

Reverse Logit

Consider now the reverse MNL. Applying ( 15) and ( 22), the best choice probabilities are given by (see also the derivation of [START_REF] Anderson | Reverse discrete choice models[END_REF]:

B R X (x; v) = {x}⊆Y ⊆X (-1) |Y |-1 e -vx
z∈Y e -vz , x ∈ X.

(34)

Likewise, the worst choice probabilities are given by the ordinary MNL formula with systematic utilities preceded by a minus sign [START_REF] Anderson | Reverse discrete choice models[END_REF][START_REF] Marley | Some probabilistic models of best, worst, and best-worst choices[END_REF]:

W R X (x; v) = e -x z∈X e -vz , x ∈ X. ( 35
)
Best-worst choice probabilities are, for x, y ∈ X, x = y, and on the basis of ( 17) and ( 24), given by (see also [START_REF] Marley | Some probabilistic models of best, worst, and best-worst choices[END_REF]:

BW R X (x, y; v) = e -vy z∈X e -vz {x}⊆Y ⊆X\{y} (-1) |Y |-1 e -vx z∈Y e -vz .
(

) 36
The results for the log-likelihood function are specular and are, for brevity, omitted.

Probit

Best, worst and best-worst choice probabilities

Assume that the vector of random components has a multivariate normal (MVN) distribution with zero mean and non-singular covariance matrix Σ. This yields the multinomial Probit (MNP) model. If the covariance matrix Σ were singular, the distribution of the random components would be degenerate, and would not admit a probability density function with respect to Lebesgue measure, contravening our initial assumption.

In the case of MNP, worst choice probabilities can be directly obtained from best choice probabilities. Proposition 3. In the MNP, we have:

W X (x, v) = B X (x, -v) , v ∈ R n , x ∈ X (37) Proof. See Appendix B.
In words, the worst choice probabilities are equal to the best choice probabilities computed with systematic utilities preceded by a minus sign.

When we have three alternatives only, the Probit with best-worst choices is the ordinary rank-ordered Probit.

Log-likelihood

In the literature, we find the following result: in the binomial Probit, the total likelihood is strictly log-concave in the estimation coefficients (see [START_REF] Wedderburn | On the existence and uniqueness of the maximum likelihood estimates of certain Generalised Linear Models[END_REF][START_REF] Daganzo | Multinomial Probit: the Theory and its Application to Demand Forecasting[END_REF][START_REF] Amemiya | Advanced Econometrics[END_REF]. For this result to hold, it suffices to assume that the matrix of observations has column full rank, otherwise there would be a non-uniqueness. No result is found relating to the strict log-concavity of total likelihood of MNP. We are able to prove the strict logconcavity, with respect to the coefficients of the systematic utilities, of the total likelihood of MNP under the same assumptions as those for strict log-concavity of total likelihood of MNL.

Proposition 4. The MNP best choice probabilities B X (x, v) and worst choice probabilities W X (x, v) are strictly log-concave in d, for any x ∈ X.

Proof. See Appendix B.

As in the Logit, recall the M matrix obtained by concatenating the rows of the N observation-specific (n i -1) × K matrices whose rows are ξ z -ξ n i , z ∈ X i \ {n i }. It plays a key role to obtain well behaved likelihood functions.

Theorem 2. In the linear-in-the-coefficients MNP model, the total log-likelihood functions of the best choices L B and of the worst choices L W are strictly concave in θ iff the matrix M has full rank.

Proof. See Appendix B.

Reverse Probit

Consider now the reverse MNP. The following equalities hold between reverse and additive choice probabilities.

Proposition 5. In the MNP, we have:

B R X (x, v) = B X (x, v) , v ∈ R n , x ∈ X , ( 38
)
and

W R X (x, v) = W X (x, v) , v ∈ R n , x ∈ X . ( 39
)
Proof. See Appendix B.

Empirical illustration

Data

In Rome, an access charging policy for a large central area, delimited by the railway ring ("Anello ferroviario"), with an extension of 33.7 square km, has been included in the new urban traffic master plan ("Piano Generale del Traffico Urbano" -PGTU) passed in 2015. The planned policy, not yet put into action, is similar to the congestion charge implemented since 2003 in London (ca 21 square km), the charging scheme implemented since 2006 in central Stockholm (ca 30 square km), and the so-called "area C" implemented since 2012 in central Milan (ca 8 square km), since it is based on a daily access charge to enter the inner city area.

A combined revealed-preference (RP) and stated preference (SP) survey was conducted, by means of computer-assisted face-to-face interviews, in Spring 2015. The sample is drawn from a population including individuals who travel by car, at least once a week, from an origin located in the area between the road ring ("Grande raccordo anulare", GRA) and the railway ring, to a destination located inside the railway ring. The RP part investigates current travel habits and preferences for transportation modes.

The SP part considers hypothetical scenarios characterised by access charges for car and powered two wheelers (PTW), the improvement of bus service supply, and the ban of more polluting vehicles in the charged area. The choice experiment proposes three alternatives (car, PTW, public transportation) with the attributes and levels reported in Table 1.

In Rome, a significant shift towards PTW is expected from policies that create negative incentives to the use of the car. Therefore, the charging policy targets PTW as well, because shifting demand towards public transportation is a main policy goal of the municipality. To increase the realism of the interview, the travel time used for car and PTW is the perceived value stated in the RP part of the interview. For public transportation, the attribute travel time is pivoted around the revealed value with variations of -20% and -40%. The experimental design is a full factorial of 48 combinations. Four combinations are excluded given the aim of the survey (dominated alternatives). After a first pilot, choice combinations were administered, for practical purposes linked to a maximum time span that had to be guaranteed in order for interviewees to complete the questionnaire, using a blocking strategy. The entire design has been subdivided into eleven blocks each including four choice tasks. This made the administration of the interview possible also in logistically unfavorable conditions, e.g. on street, at office entrance/exit, in shopping malls during regular opening hours. Figure 1 shows a choice task example. The interviewee is invited to select, among the three proposed alternatives, the two that she/he would choose with highest probability (best alternative) and lowest probability (worst alternative).

We acquired 146 interviews, each including four choice tasks. The minimum age of the interviewees is 23 years, maximum 79, mean 46.2 and standard deviation 11.3. Table 2 reports sample statistics for other socio-economic characteristics.

Estimation

Table 3 reports estimation results for the three models: additive Logit, reverse Logit and Probit, with B only, W only and B&W choice data. Recall that additive Probit is identical to reverse Probit. Probit also considers independent random terms across alternatives. Alternative 1 is car, alternative 2 is PTW, alternative 3 is public transportation. The systematic utilities have the standard linear-in-the-coefficients structure. Remember that, since we have three alternatives only, the additive Logit with B&W choices is exactly the same as the rank-ordered Logit. Similarly, the Probit with B&W choices is the ordinary rank-ordered Probit.

In all three models with all choice data we observe the following common traits.

• The congestion charge and travel time coefficients have the expected sign and are statistically significant (1% level).

• The alternative specific constant of alternative 2 (PTW) is negative and statistically significant (1% level) (with the exception of Probit with W choice data which shows a sign reversal), the reference alternative being fixed as alternative 1 (car); the negative sign can be explained with the lower levels of comfort and safety.

• The coefficient associated with the travel time-gender interaction effect is positive and statistically significant (1% level), denoting an attitude of females to accept longer travel times than males.

• The coefficient associated with the charge-income interaction effect is positive, denoting that, as expected, income increases reduce the disutility of the congestion charge.

The first analysis relates to the stability of the coefficients across B only and W only choice data. The magnitude of most coefficients is affected by the choice data, something which would have significant impacts on best choice shares, if data aggregation were post-performed upon estimation results. The estimates with W choices appear attenuated in comparison with the B choice ones. The statistical significance, measured by the absolute value of the t-statistic, of the coefficients with W data is lower than with B data. In this regard, one notes that scale differences can exist between B and W data [START_REF] Swait | The role of the scale parameter in the estimation and comparison of multinomial logit models[END_REF]. This makes direct comparison between coefficients, and the associated statistical significance, not possible.

Figure 2, 3 and 4 show for the additive Logit, reverse Logit and Probit respectively, the plot of the W choice versus B choice coefficients. For the additive Logit, the linear regression is worst = .472 • best + .078, with R 2 = .84; for the reverse Logit, the linear regression is worst = .667 • best + .113, with R 2 = .79; for the Probit the linear regression is not estimated due to a sign reversal in one coefficient. The slope of the best-fit linear regression line is less than unity, another manifestation of the attenuation in the coefficients. It is as if, when moving from B to W, the random term variance increased, because individuals are less certain about their least preferred alternative. This outcome is an empirical regularity in rank-ordered data analysis (see the results in [START_REF] Ben-Akiva | Analysis of the reliability of preference ranking data[END_REF][START_REF] Louviere | Best-Worst Scaling: Theory, Methods and Applications[END_REF], Ch. 14, and Giergiczny et al., 2017), which is explained with the relative cognitive difficulties of identifying best and worst alternatives. The equality of individual coefficients between B choice data and W choice data is tested by the asymptotically normal test statistic:

β kB -β kW V ar(β kB ) + V ar(β kW ) (40)
The results of this test are in Table 4. Most differences are significant at 5% level. Statistical significance of differences results to be lower in Probit than in additive Logit (with the exception of one coefficient), a result in agreement with Nair et al. (2018b), who find the rank-ordered Probit a superior option to the rank-ordered Logit in terms of robustness of coefficients across ranking depths. We have conducted additional simulations to assess whether the coefficient attenuation is attributable to the model's mathematical characteristics or to the individuals' behaviour. To this aim, a vector of coefficients of the systematic utilities is assumed. Then the random terms are drawn and the B and the W alternative identified. Two models are estimated with these simulated choices: one with B choices and one with W choices. Three cases are considered: additive Logit, reverse Logit and Probit. As a result, no systematic bias, i.e. no coefficient attenuation from B to W, is found with simulated choices. We can infer this empirical finding to be a behavioural effect, rather than a mathematical artefact.

A second analysis relates to the values of time. The values of the marginal rate of substitution between travel time and the congestion charge (from the utilities of the car and PTW alternatives) can be assumed, in the context of the example here, as a measure of the marginal willingness to pay (WTP). This WTP expresses the value of the travel time savings (VTTS). WTP values, and therefore VTTS, are not affected by the scale value.

The mean VTTS are shown in Table 5 for the additive Logit, reverse Logit and Probit. In all three models, mean VTTS with W choices are higher than mean VTTS with B choices (except for the highest male income class). In contrast, [START_REF] Giergiczny | Testing and stability of utility parameters in repeated best, repeated best-worst, and one-off best-worst studies[END_REF] find for the choice of rail services database the opposite result.

Approximated standard errors of the VTTS are also computed, using the delta method [START_REF] Greene | LIMDEP Version 10 Reference Guide[END_REF]. The associated t-statistics are shown in Table 5. The VTTS confidence intervals with W data are relatively larger than those with B data, except for the highest income class.

Alternative methods can be used to compute WTP confidence intervals. [START_REF] Gatta | On finite sample performance of confidence intervals methods for willingness to pay measures[END_REF] provide a systematic comparison of eleven methods investigating the performance under different conditions. They suggest using the likelihood ratio test inversion method, which is not affected by the cost coefficient approaching zero and is robust to small departures from correct specification, especially with small samples. The delta method, however, produces similar results when relatively large samples are involved, as in the case under consideration.

The equality of VTTS values between B choice data and W choice data is tested by the asymptotically normal test statistic. Table 6 reports the results of the test. Statistically significant differences between B and W choice data tend to be shown by lower income classes and the female population. Statistical significance of differences results to be lower in Probit than in additive Logit, a result reinforcing the agreement with Nair et al. (2018b), who find, as observed, the rank-ordered Probit a superior option to the rank-ordered Logit in terms of robustness of coefficients across ranking depths. -0.18 -0.12 -0.22 Key: ***,**,* = statistical significance at 1%, 5%, 10% level A third analysis relates to the evaluation of the predictive abilities of the models. This is also a test for similarities and differences among models estimated with B, W and B&W choices. We use the root likelihood as a scoring function to measure predictive accuracy. A scoring function is a measure of model prediction performance that relates the probabilistic model predictions to the stated choices [START_REF] Louviere | Best-Worst Scaling: Theory, Methods and Applications[END_REF]. The root likelihood is preferred to the hit rate because it takes into account the probabilistic nature of the choice model, and it penalizes poor predictions in addition to rewarding accurate ones.

The root likelihood for prediction of B choices RL B of the model estimated with choice data C is defined as

RL B = N i=1 x∈X i B i X i x i , v i C i BX i (x i ,v i C ) 1/N , ( 41
)
where i is the observation superscript, N the number of observations in the sample, i BX i x i , v i C the best choice indicator function for observation i and alternative x ( i BX i x i , v i C = 1 if in observation i the alternative x is chosen as best,= 0 otherwise).

In the case where the model is estimated with B data, since

ln RL B = 1 N N i=1 x∈X i i BX i x i , v i B ln B i X i x i , ( 42
)
we also have

RL B = e 1 N LL B , ( 43
)
where LL B is the total log-likelihood with best choices. Similarly, the root likelihood for predicting W choices RL W of the model estimated with choice data C is defined as

RL W = N i=1 x∈X i W i X i x i , v i C i W X i (x i ,v i C ) 1/N , ( 44
)
where i W X i x i , v i C is the worst choice indicator function for observation i and alternative x ( i W X i x i , v i C = 1 if in observation i the alternative x is chosen as worst,= 0 otherwise), and LL W is the total log-likelihood with worst choices.

In the case where the model is estimated with W data we have

RL W = e 1 N LL W , ( 45
)
where LL W is the total log-likelihood with worst choices. Table 7 shows, for the additive Logit, the reverse Logit and the Probit, the values of the root likelihood for predicting B and W choices with different types of choice data (B, W, B&W). The models estimated with B choice data best predict B choices (in the light of the highest root likelihood values), the models estimated with W choice data best predict W choices.

Notwithstanding the additional information, the models estimated with B&W data fail to improve prediction for either B and W choices relative to the models estimated with single-choice data.

In addition, the expectation that the models estimated with B&W data should provide better estimates, i.e. higher values of the t-statistic, than with B choice data or with W choice data only is not confirmed. These results are in agreement with Louviere et al. (2015, Ch. 14), who investigates Logit, using a reverse model for worst choices.

A fourth and final analysis relates to a comparison of the goodness of fit between the three models: additive Logit, reverse Logit and Probit. The analysis is carried out with reference to the AIC (Table 8). The model with the lowest AIC is the one with the best goodness of fit.

Conclusion

The paper has dealt with the estimation of RUMs using best, worst and best-worst data. Two distributions of the random terms are considered: Logit and Probit. Recent literature has provided closed-form expressions for best, worst and best-worst choice probabilities for Logit. The paper has proved that, in the case of Probit, worst choice probabilities are obtained from best choice probabilities by simply changing the sign of the systematic utilities. The paper has also proved the equality of best and worst choice probabilities between the additive and the reverse Probit. These properties of Probit are attributable to the symmetry of the univariate standard normal distribution.

A remarkable result of the paper is the proof, under mild conditions, of the strict log-concavity, with respect to the coefficients of the systematic utilities, of the likelihood functions associated with best, worst and best-worst choice probabilities in the case of Logit, and with best and worst choice probabilities in the case of Probit. The conditions boil down to the absence of perfect multicollinearity in the matrix of the alternative and individual characteristics, similar to linear regression models. The strict log-concavity of the likelihood associated with best-worst choice probabilities in Probit remains an open problem.

In the transportation mode choice example here, worst choice preferences, when compared with best choice preferences, show, in both Logit and Probit models, coefficient attenuation, and higher mean VTTS with larger confidence intervals. The results on VTTS suggest different best choice and worst choice preferences, even accounting for scale differences. This is, for Logit, in agreement with other authors [START_REF] Ben-Akiva | Analysis of the reliability of preference ranking data[END_REF][START_REF] Louviere | Best-Worst Scaling: Theory, Methods and Applications[END_REF][START_REF] Giergiczny | Testing and stability of utility parameters in repeated best, repeated best-worst, and one-off best-worst studies[END_REF] Only the result on the sign of the difference of the VTTS between best and worst choice data is conflicting with the paper by [START_REF] Giergiczny | Testing and stability of utility parameters in repeated best, repeated best-worst, and one-off best-worst studies[END_REF], who find lower mean VTTS with worst choice data. The finding on the lower statistical significance of differences in coefficients and VTTS in Probit than in Logit is in agreement with Nair et al. (2018b), who suggest rank-ordered Probit to be a superior option to rank-ordered Logit in terms of robustness of preferences across ranking depths.

The reasons for these results remain to be more extensively understood. Part of the explanation may be due to the behavioral shift: the respondent is usually more familiar with the best choice than with the worst choice.

In the empirical example, the benefit of collecting and using both best and worst choices as a data augmentation strategy for a best choice model is not confirmed. Therefore, at least in this case, the benefit of collecting best and worst choice data lies in the additional insights into the preferences of the individuals.

Data augmentation with worst choices is in all cases useful to estimate the probability of a given alternative ranking last. This would be of interest to marketing and the transportation field alike, because policy makers might wish to know if a given product or alternative had an unacceptable feature (when best-worst scaling is used for alternatives constituted by attributes of a single product or service). Future research might deal with an extension of this, estimating the probabilities, available in closed form from de Palma & Kilani (2015), of a given alternative ranking k-th, k = 1, 2, • • •, n, in the choice set X (first, second,..., last).

Appendix A. Proofs of lemmas

Proof of Lemma 1. Using Theorem 10 in de [START_REF] De Palma | Relations between best, worst, and best-worst choices for random utility models[END_REF], for any x, y ∈ X, x = y, and any v ∈ R n , the best and worst choice probabilities BW X (x, y; v) take the form given in (8) where

B X,Y (x, y; v) = ∞ 0 +∞ -∞ ∂ 2 F (u1 X -t1 Y -v) ∂t x ∂t y dudt.
Using (3), we have

B X (x; v + t1 Y ) = +∞ -∞ ∂F (u1 X -v -t1 Y ) ∂t x du.
Differentiation of the above equation with respect to v y , yields

-∂B X (x; v + t1 Y ) ∂v y = +∞ -∞ ∂ 2 F (u1 X -v -t1 Y ) ∂t x ∂t y du.
Therefore, we get the alternative expression of B X,Y (x, y; v) provided in (9).

Proof of Lemma 2. Differentiating (6), we obtain

W X (x; v)∇ θ L W X (x) = {x}⊆Y ⊆X (-1) |Y |-1 B Y (x; v)∇ θ L B Y (x).
Dividing then both members of the above equation by W X (x; v), we obtain (12). Differentiating Expression (12), we get

∇ 2 θ L W X (x) = {x}⊆Y ⊆X ω X,Y (x)∇ 2 θ L B Y (x) + ∇ θ L B Y (x) ∇ θ ω X,Y (x) ,
where ∇ θ ω X,Y (x) is the row gradient of ω X,Y (x) with respect to θ. It satisfies

∇ θ ω X,Y (x) = ω X,Y (x)∇ θ L B Y (x) -ω X,Y (x)∇ θ L W X (x).
Combining the two above displayed equations and using (12), Eq. ( 13) is obtained.

Note that w is related to d by a reversible linear transformation. Indeed, setting d n = w x = 0, we have: w z = d x -d z , z ∈ X \ {x}, and conversely: d z = w n -w z , z ∈ X \{n}. Strict concavity being preserved under any reversible linear transformation (Daganzo, 1979, Appendix D), strict log-concavity in d and w are equivalent. We therefore need to prove the strict long-concavity of F δ (•).

In the case of the MNL, we have

F δ (w) = z∈X e -wz -1 , w ∈ R n-1 ,
which is the distribution of the classical (n -1)-variate logistic distribution (see Arnold, 1992, p. 238). Since,

ln F δ (w) = -ln z∈X e -wz , w ∈ R n-1 ,
we get, for z 1 , z 2 ∈ X \ {x}, the following second-order partial derivatives

∂ 2 ln F δ (w) ∂w z1 ∂w z2 = e -wz 1 z∈X e -wz 2 -z∈X\{z1} e -wz , z 2 = z 1 ; e -wz 2 , z 2 = z 1 .
The Hessian of ln F δ (•) is a symmetric matrix which has a negative dominant diagonal since z∈X\{z1} e -wz > z∈X\{z1,n} e -wz , z 1 ∈ X \ {x}.

Accordingly (see, e.g., [START_REF] Mas-Colell | The Theory of General Equilibrium: a Differentiable Approach[END_REF], the Hessian of ln F δ (•) is negative definite so that F δ (•) is strictly log-concave.

Similarly, for the worst choice, we have

W X (x; v) = P   z∈X\{x} { x -z ≤ v z -v x }   = F -δ (-w), (47)
where F -δ (•) is the c.d.f. of -δ. We need to prove that F -δ (•) is strictly logconcave. As well known, the p.d.f. f -δ (•) of -δ and the p.d.f. f δ (•) of δ are related by f -δ (-w) = f δ (w). The p.d.f. of the classical (n -1)-variate logistic distribution satisfies

f δ (w) = (n -1)!e - z∈X wz z∈X e -wz -n , so that ln f δ (w) = ln [(n -1)!] - z∈X w z + n ln F δ (w).
The RHS is the sum of an affine function and a strictly concave expression in w (recall that F δ (•) is stricly log-concave), so that f δ (•) (and therefore f -δ (•)) is strictly log-concave. Applying (Prékopa, 1973, Theorem 5), F -δ (•) is also strictly log-concave, so that W X (x; v) is strictly log-concave in d.

Proof of Theorem 1. Consider one observation (the index i is omitted for simplicity). Based on Proposition 2, the best choice probabilities B (x; v) and the worst choice probabilities W (x; v) are strictly log-concave in d for any x ∈ X. We have

d z = w n -w z , z ∈ X \ {n}, with w z = v x -v z . Therefore, d z = v z -v n , z ∈ X \ {n}.
Recall that the systematic utilities are assumed linear in the coefficients:

v z = ξ z θ, z ∈ X.
Based on Daganzo (1979, Appendix D), if F (x) is strictly concave in x = (x 1 , ..., x n ), A is an arbitrary m×n matrix, and y is a m-dimensional row vector, then G (y) = F (yA) is strictly concave in y. Therefore, the natural logarithm of B (x; v) and of W (x; v) is strictly concave in θ. The strict log-concavity of the total likelihood follows from preservation of strict log-concavity under product.

If the matrix M has not full rank, i.e. if at least one column is linear combination of the others, it is not possible to estimate the independent impact of the change of all the alternative and individual characteristics. This circumstance must be excluded, or, as underlined by [START_REF] Wedderburn | On the existence and uniqueness of the maximum likelihood estimates of certain Generalised Linear Models[END_REF], there would be non-uniqueness. An example clarifies this point. Assume we have for all observations (related index omitted) ξ 1 = αξ 2 . Clearly M has not full rank. Then we have identical values of probabilities for different combinations of θ 1 and θ 2 , i.e. non-uniqueness. This is because we have:

v z = αξ 2,z θ 1 + ξ 2,z θ 2 + h =1,2
ξ h,z θ h , z ∈ X Computation of utility, and probability, depends on αθ 1 + θ 2 . There are infinite combinations of θ 1 and θ 2 yielding the same value of αθ 1 + θ 2 .

Proof of Proposition 3. In the light of Eqs (2) and ( 5), we only need to prove that the vector -, which is MVN since it is an affine transformation of , has zero mean and covariance matrix Σ. Recall the following property (Brockwell & Davis, 1991, Proposition 1.6.1.): if a is a m-component column vector, B is a m × n matrix, and X a n-component random vector with mean E (X) and covariance matrix Σ XX , then the random vector Y = a + BX, has mean E (Y) = a + BE (X), and covariance matrix Σ YY = BΣ XX B . We only need to apply this property to , with a = 0 and B = -I n , where I n is the n × n identity matrix.

Proof of Proposition 4. The proof goes along the same lines as the proof of Proposition 2. We use Eq. ( 46), which holds for any ARUM. In the Probit case, the random vector δ is a MVN with p.d.f.

f δ (w) = 1 (2π) (n-1)/2 [det(∆)] 1/2 exp - 1 2 w ∆ -1 w ,
where w ∈ R n-1 and ∆ is the covariance matrix of δ, provided that this matrix is non-singular, which is proved below. Therefore, we get

ln f δ (w) = - 1 2 w ∆ -1 w - n -1 2 ln (2π) - 1 2 ln [det(∆)] .
The first term of the RHS is a symmetric bilinear form with matrix (-1/2)∆ -1 , the remaining terms being constants. Since the covariance matrix ∆ is positive definite, ∆ -1 is also positive definite (Cambini & Martein, 2009, page 233). Hence (-1/2)∆ -1 is negative definite so that f δ (•) is stritly log-concave. By Theorem 5 of Prékopa (1973), F δ (•) also is strictly log-concave implying that B X (x, v) it is also strictly log-concave in d.

For the worst choice probability in the Probit case, using (47), recall that W X (x; v) = F -δ (-w). Since F -δ (•) = F δ (•), the worst choice probabilities also are strictly long-concave.

We now prove that ∆ is non-singular, therefore positive definite, if Σ, the covariance matrix of , is positive definite. W.l.o.g., and for the sake of notational simplicity, we set x = n. We denote by ∆ z1z2 (resp. σ z1z2 ) the (z 1 , z 2 )-th element of ∆ (resp. Σ), for z 1 , z 2 belonging to X \ {n} (resp. X). They are related by ∆ z1z2 = σ z1z2 -σ z1n -σ z2n + σ nn , z 1 , z 2 = 1, . . . , n -1.

For any w non-null vector of R n-1 , let define Therefore, w ∆w = x Σx, where x = (w, w n ) ∈ R n . Since x is non-null and Σ is definite positive, we have x Σx > 0 so that w ∆w > 0. Hence ∆ is definite positive and therefore non-singular.

Proof of Theorem 2. Consider one observation (the index i is omitted for simplicity). Based on Proposition 4, the best choice probabilities B (x; v) and the worst choice probabilities W (x; v) are strictly log-concave in d for any x ∈ X. We have d z = w n -w z , z ∈ X \ {n}, with w z = v x -v z . Therefore,

d z = v z -v n , z ∈ X \ {n}.
Recall that the systematic utilities are assumed linear in the coefficients: v z = ξ z θ, z ∈ X.

Based on (Daganzo, 1979, Appendix D), if F (x) is strictly concave in x = (x 1 , ..., x n ), A is an arbitrary m×n matrix, and y is a m-dimensional row vector, then G (y) = F (yA) is strictly concave in y. Therefore, the natural logarithm of B (x; v) and of W (x; v) is strictly concave in θ. The strict log-concavity of the total likelihood follows from preservation of strict log-concavity under product.

If the matrix M has not full rank, i.e. if at least one column is linear combination of the others, it is not possible to estimate the independent impact of the change of all the alternative and individual characteristics. This circumstance must be excluded, or, as underlined by [START_REF] Wedderburn | On the existence and uniqueness of the maximum likelihood estimates of certain Generalised Linear Models[END_REF], there would be non-uniqueness. An example clarifies this point. Assume we have for all observations (related index omitted) ξ 1 = αξ 2 . Clearly M has not full rank. Then we have identical values of probabilities for different combinations of θ 1 and θ 2 , i.e. non-uniqueness. This is because we have:

v z = αξ 2,z θ 1 + ξ 2,z θ 2 + h =1,2 ξ h,z θ h , z ∈ X
Computation of utility, and probability, depends on αθ 1 + θ 2 . There are infinite combinations of θ 1 and θ 2 yielding the same value of αθ 1 + θ 2 .

Proof of Proposition 5. The proof is along the same lines of the proof of Proposition 3. It is left to the reader.

Figure 1 :

 1 Figure 1: Choice task example

Figure 2 :

 2 Figure 2: W choice coefficients plotted against B choice coefficients: additive Logit

Figure 4 :

 4 Figure 4: W choice coefficients plotted against B choice coefficients: Probit

∆σ

  z1z2 w z1 w z2 .Let w n = -n-1 z=1 w z . Combining the two above expressions, we get zn w z w n + σ nn w 2 n .

Table 1 :

 1 Attributes and levels in the choice experiment

	Attribute			Alt.
		car	PTW	public transportation
	access charge (EUR/day) 0,2,4,6 0,2,4,6	-
	travel time (minutes)	RP	RP	RP, -20% RP, -40% RP

Table 2 :

 2 Sample statistics

	Abs. freq. Percent

Table 3 :

 3 Estimation results: coefficients

	B W B&W	(t-stat) (t-stat) Coefficient Alt. Coefficient (t-stat.) Coefficient	Additive Logit	-.505*** (-7.75) (-7.85) -.375*** (-5.32) charge (EUR/day) 1,2 -.947***	-.070*** (-12.55) (-9.82) -.060*** (-9.74) travel time (minutes) 1,2,3 -.095***	-1.282*** (-10.93) (-11.06) -1.029*** (-8.21) alt. specific constant 2 -2.186***	.094 (.78) (-2.69) .220 (1.36) alt. specific constant 3 -.453***	.023*** (4.49) (4.81) .017*** (3.23) travel time × gender (female = 1) 1,2,3 .043***	.053*** (3.12) (4.20) .035* (1.90) charge × income (six income classes: 1,...,6) 1,2 .121***	-825.77 -501.35 583 (pseudo)observations, log-likelihood function = -421.76	Reverse Logit	-.576*** (-8.08) (-7.69) -.432*** (-5.27) charge (EUR/day) 1,2 -.796***	-.071*** (-11.70) (-9.80) -.072*** (-9.92) travel time (minutes) 1,2,3 -.081***	-1.317*** (-11.15) (-11.03) -1.164*** (-8.31) alt. specific constant 2 -1.778***	-.028 (-.22) (-2.20) .384* (1.86) alt. specific constant 3 -.333**	.029*** (5.29) (5.06) .023*** (3.60) travel time × gender (female = 1) 1,2,3 .037***	.065*** (3.62) (4.17) .040* (1.83) charge × income (six income classes: 1,...,6) 1,2 .103***	-804.78 -503.42 583 (pseudo)observations, log-likelihood function = -422.74	Probit	-.443*** (-6.69) (-8.21) -.328*** (-5.32) charge (EUR/day) 1,2 -.693***	-.060*** (-11.89) (-10.24) -.055*** (-10.39) travel time (minutes) 1,2,3 -.073***	-1.129*** (-12.03) (8.55) (-11.56) .940*** alt. specific constant 2 -1.644***	(0.24) 0.025 (-1.48) (-2.39) -.229 alt. specific constant 3 -.323**	(4.54) 0.022*** (3.38) (5.03) .016*** travel time × gender (female = 1) 1,2,3 .033***	(2.57) .045*** (1.67) (4.16) .027* charge × income (six income classes: 1,...,6) 1,2 .085***	-811.75 -501.59 583 (pseudo)observations, log-likelihood function = -421.11	Key: ***,**,* = statistical significance at 1%, 5%, 10% level

Table 4 :

 4 Asymptotically normal test statistic: B-W difference of coefficients

		Alt.	Additive	Reverse	Probit
			Logit	Logit
	charge	1,2	-4.09***	-2.76***	-3.50***
	travel time	1,2,3	-3.02***	-0.75	-1.99**
	alt. specific constant 2	-4.94***	-2.87***	-14.38***
	alt. specific constant 3	-2.88***	-2.80***	-0.46
	travel time × gender 1,2,3	2.43**	1.47	2.02**
	charge × income	1,2	2.50**	1.91*	2.18**
	Key: ***,**,* = statistical significance at 1%, 5%, 10% level

Table 5 :

 5 Estimation results: VTTS (EUR/h)

	Additive Logit Reverse Logit Probit	Net income B W B W B W	(EUR/month) Mean (t-stat.) Mean (t-stat.) Mean (t-stat) Mean (t-stat) Mean (t-stat) Mean (t-stat)	Male	≤ 500 6.9 (6.74) 10.7 (5.52) 7.0 (6.58) 11.1 (5.44) 7.28 (6.85) 11.17 (5.51)	501-1000 8.1 (7.46) 11.9 (6.54) 8.2 (7.33) 12.4 (6.46) 8.46 (7.58) 12.30 (6.54)	1001-1500 9.8 (8.30) 13.5 (7.51) 10.0 (8.30) 14.0 (7.43) 10.11 (8.43) 13.69 (7.47)	1501-2500 12.4 (8.35) 15.6 (6.92) 12.6 (8.61) 16.0 (6.85) 12.54 (8.53) 15.44 (6.90)	2501-5000 16.9 (5.91) 18.4 (4.75) 17.3 (6.12) 18.8 (4.73) 16.52 (6.32) 17.69 (4.92)	>5000 26.5 (2.89) 22.4 (2.93) 27.3 (2.89) 22.8 (2.94) 24.20 (3.37) 20.71 (3.21)	Female	≤ 500 3.7 (7.02) 7.5 (5.19) 3.7 (7.04) 7.6 (5.04) 3.93 (7.28) 7.78 (5.09)	501-1000 4.4 (7.56) 8.4 (5.94) 4.4 (7.63) 8.4 (5.75) 4.57 (7.85) 8.57 (5.82)	1001-1500 5.3 (7.96) 9.5 (6.52) 5.3 (8.12) 9.5 (6.29) 5.46 (8.23) 9.54 (6.36)	1501-2500 6.7 (7.39) 11.0 (5.99) 6.8 (7.60) 10.9 (5.81) 6.78 (7.63) 10.75 (5.91)	2501-5000 9.2 (5.18) 12.9 (4.35) 9.2 (5.27) 12.8 (4.28) 8.93 (5.51) 12.32 (4.48)	>5000 14.4 (2.69) 15.8 (2.80) 14.6 (2.66) 15.5 (2.80) 13.01 (3.09) 14.43 (3.05)

Table 6 :

 6 Asymptotically normal test statistic: B-W difference of VTTS

		Additive Logit Reverse Logit	Probit
	Male
	≤ 500	-1.71*	-1.78*	-1.70*
	501-100	-1.79*	-1.87*	-1.76*
	1001-1500 -1.70*	-1.80*	-1.63
	1501-2500 -1.16	-1.23	-1.08
	2501-5000	0.30	-0.32	-0.26
	>5000	0.34	-0.37	0.36
	Female
	≤ 500	-2.43**	-2.39**	-2.38**
	501-1000	-2.61***	-2.56**	-2.53**
	1001-1500 -2.60***	-2.53**	-2.49**
	1501-2500 -2.06**	-2.00**	-1.96**
	2501-5000 -1.09	-1.03	-1.06
	>5000

Table 7 :

 7 Predictive accuracy of models estimated with different choice data: root likelihood

	Additive Logit Reverse Logit Probit	Predict W Predict B Predict W Predict B Predict W Estimation data Predict B	choices RL B choices RL W choices RL B choices RL W choices RL B choices RL W	.485°.384 .484°.402 .483°.371 B	.421°.416 .423°.466 .454 W .423°B	.416 .473 .418 .478 .419 .467 &W	°Highest value

Table 8 :

 8 Goodness of fit of different models: AIC

	Model Predict Predict Predict	B choices W choices B&W	choices	1014.7°1663.5 additive Logit 855.5	857.5 1018.8 reverse Logit 1621.6°P	1635.5 854.2°1015.2 robit	°Lowest value

That is, the support of is the whole space R n .

A proof of this equality requires the use of Fubini's theorem to interchange the order of integrals and Lebesgue's dominated convergence theorem to interchange the limit with an integral sign.

The vector argument v is voluntarily omitted from the notation of the alternating weights. Thanks to Identity (6), we can verify that:{x}⊆Y ⊆X ω X,Y (x) = 1, x ∈ X.

The integrand of the RHS of (3) is given by: ∂F (u1 X -v)/∂tx = e vx-u exp(-e Λ X (v)-u ). By integration, we get: B X (x, v) = e vx-Λ X(v) . The remaining MNL best choice probabilities are then obtained by applying (4).

Since ∂B X (x; v + t1 Y )/∂vy = -e vx+vy +t /[e Λ X\Y (v) + e t+Λ Y (v) ] 2 , integration of that expression yields (notice that e Λ X\Y (v) + e Λ Y (v) = e Λ X (v) ): B X,Y (x, y; v) = e vx+vy -Λ X (v)-Λ Y (v) . Applying then Lemma 1 yields the expression of the MNL best-worst choice probabilities.

Acknowledgments

The authors would like to thank the Associate Editor for helpful comments on an earlier version of the paper. The authors are also grateful to Alfredo Donno, Julien Monardo and to the participants of the 2018 Annual Conference of the International Transportation Economics Association, Hong Kong.

Appendix B. Proofs of propositions and theorems

Proof of Proposition 1. For the gradient computation, thanks to Lemma 2, Eq. ( 12), we have

By expansion, and using the fact that {x}⊆Y ⊆X ω X,Y (x) = 1 and the Definition (29), we obtain (32).

For the expression of the Hessian matrix, we are using Lemma 2, Eq. ( 13). Notice first that using ( 26) and ( 28), for any x ∈ Y ⊆ X, we have

Therefore, using ( 13) and ( 32), we get

where

The above expression can be rewritten as

By expansion, we obtain

which can be rewritten as follows

obtaining the required expression.

Proof of Proposition 2. The best choice probabilities, for any v ∈ R n and any x ∈ X, can be written as

where F δ (•) is the c.d.f. of δ whose components are zx , z ∈ X \ {z}; the vector w ∈ R n-1 has components: w z = v x -v z , z ∈ X \ {x}.


MD Solene-Florence Kammerer-Jacquet

MBBS Angelique Brunot

Adelaide Pladys

MD Guillaume Bouzille

MBBS Julien Dagher

MBBS Sarah Medane

M A N U S C R I P T A C C E P T E Benoit Peyronnet

Synchronous metastatic clear cell renal cell carcinoma: a distinct morphological, immunohistochemical and molecular phenotype

Keywords: clear cell renal cell carcinoma, synchronous and metachronous metastases, phenotype, clinical outcome

In order to compare synchronous and metachronous metastatic clear cell renal cell carcinoma (ccRCC), we performed a pathological, immunohistochemical and molecular study on primary tumors in a retrospective series of 48 consecutive patients with up to ten years of follow-up. Synchronous metastatic ccRCC had a distinct phenotype that may explain their worse prognosis.

Introduction: Clear cell renal cell carcinomas (ccRCC) are highly metastatic tumors with metastases detected at diagnosis (synchronous) or during follow-up (metachronous). To date, there have been no reports comparing primary ccRCC of synchronous and metachronous metastatic patients, yet different in terms of prognosis. Determining whether there is a phenotypic difference between these two groups could have important clinical implications.

Patients and Methods:

In a retrospective consecutive cohort of 98 patients with ccRCC, 48 patients had metastases including 28 synchronous and 20 metachronous presentations with a follow-up of 10 years. For each primary tumor in these metastatic patients, pathological criteria, expression of VEGF, PAR-3, CAIX and PD-L1 as detected by immunohistochemistry, and complete VHL status were analyzed. Univariate analysis was performed and survival was assessed using Kaplan-Meier curves compared by log-rank test.

Results: Compared to primary ccRCC in metachronous metastatic patients, primary ccRCC in synchronous metastatic patients were significantly associated with a poorer ECOG performance (p=0.045), higher pT status (p=0.038), non-inactivated VHL gene (p=0.01), sarcomatoid component (p=0.007), expression of PAR-3 (p=0.007), and overexpressions of VEGF (>50%) (p=0.017) and PD-L1 (p=0.019). Patients with synchronous metastases had a M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 4 worse cancer-specific survival than patients with metachronous metastases even from metastatic diagnosis (median survival 16 months versus 46 months, respectively, p=0.01).

Conclusion:

This long-term study is the first to support the notion that synchronous m-ccRCC has a distinct phenotype. This is probably linked to the occurrence of oncogenic events that could explain their worse prognosis. These particular metastatic patients could benefit from specific therapy.

INTRODUCTION

Clear cell renal cell carcinoma (ccRCC) is the most common histologic subtype of renal cell carcinoma. [START_REF] Murai | Renal cell carcinoma: etiology, incidence and epidemiology[END_REF] Tumor cells are characterized by inactivation of the VHL gene leading to HIF stabilization, which induces the transcription of genes such as vascular endothelial growth factor (VEGF). [START_REF] Gnarra | Mutations of the VHL tumour suppressor gene in renal carcinoma[END_REF] As a consequence, the tumor microenvironment is highly vascularized.

Another critical component of the tumor microenvironment is the immune system as some tumors have a high density of tumor infiltrating lymphocytes (TIL). [START_REF] Kang | Tumor-infiltrating PD1-Positive Lymphocytes and FoxP3-Positive Regulatory T Cells Predict Distant Metastatic Relapse and Survival of Clear Cell Renal Cell Carcinoma[END_REF] Moreover, tumor cells were shown to express programmed death ligand 1 (PD-L1) to escape the immune system. [START_REF] Callea | Differential Expression of PD-L1 between Primary and Metastatic Sites in Clear-Cell Renal Cell Carcinoma[END_REF][START_REF] Chen | Regulation of PD-L1: a novel role of pro-survival signalling in cancer[END_REF] With approximately 40% of patients dying of metastases, ccRCC are highly aggressive tumors. [START_REF] Jemal | Cancer statistics[END_REF] The most common sites of metastasis are the lungs, distant lymph nodes, liver, bones, brain and adrenal gland. Twenty to 30% of patients are diagnosed with metastatic disease (synchronous presentation) whereas 20% of patients with non-metastatic disease at diagnosis will later develop metastases during follow-up (metachronous presentation). [START_REF] Ljungberg | The epidemiology of renal cell carcinoma[END_REF] Metastases generally arise in the first six years after surgery. [START_REF] Eggener | Renal cell carcinoma recurrence after nephrectomy for localized disease: predicting survival from time of recurrence[END_REF] Synchronous and metachronous ccRCC have a different prognosis. Currently, one of the criteria of MSKCC and Heng risk criteria models for predicting survival in metastatic patients is a time from initial diagnosis (including original localized disease) to treatment of less than one year. [START_REF] Heng | Prognostic factors for overall survival in patients with metastatic renal cell carcinoma treated with vascular endothelial growth factortargeted agents: results from a large, multicenter study[END_REF][START_REF] Motzer | Prognostic factors for survival of patients with stage IV renal cell carcinoma: memorial sloan-kettering cancer center experience[END_REF][START_REF] Mekhail | Validation and extension of the Memorial Sloan-Kettering prognostic factors model for survival in patients with previously untreated metastatic renal cell carcinoma[END_REF] This risk factor includes synchronous metastatic patients. The difference in prognosis between the two groups of patients may be linked to their primary tumors having different phenotypes. To our knowledge, there are no reports comparing primary ccRCC in This is the first study to conduct an in-depth analysis of primary ccRCC in correlation with pathological criteria, VHL status and long-term clinical outcome with a view to seeking differences depending on synchronous or metachronous metastatic status.

MATERIALS AND METHODS

Patients

Between 2002 and 2005, 98 consecutive patients were operated for sporadic ccRCC in the Department of Urology at Rennes University Hospital. Twenty-eight patients had synchronous metastases at initial diagnosis and underwent nephrectomy in accordance with international guidelines. [START_REF] Heng | Cytoreductive nephrectomy in patients with synchronous metastases from renal cell carcinoma: results from the International Metastatic Renal Cell Carcinoma Database Consortium[END_REF] They received no prior therapy. Twenty patients had a nephrectomy before developing metastases which were detected during follow-up (first CT scan 6 months after surgery) and defined as metachronous. Consequently, a total of 48 patients presented metastases between 2002 and 2012 and were included in our study. Patient charts were retrospectively reviewed to assess pretreatment ECOG performance status, methods of detection (incidental or symptomatic), involved metastatic sites with retrieved first staging CT scan, MSKCC score, Heng criteria and therapies [START_REF] Heng | Prognostic factors for overall survival in patients with metastatic renal cell carcinoma treated with vascular endothelial growth factortargeted agents: results from a large, multicenter study[END_REF][START_REF] Mekhail | Validation and extension of the Memorial Sloan-Kettering prognostic factors model for survival in patients with previously untreated metastatic renal cell carcinoma[END_REF] . The outcome was specific death which we assessed with a 10-year follow-up. The study protocol was approved by the local advisory board and informed consent was obtained from each patient for the study. Genomic DNA was extracted from 25 to 35 mg of frozen tissue sections using a QIAamp DNA minikit (Qiagen, Courtaboeuf, France). DNA quantity and quality were estimated by optical density (OD 260/280) measurement and 0.8% agarose gel electrophoresis using standard protocols.

Tissue sample management

Pathological analysis

After fresh tissue sampling, surgical specimens were formalin-fixed. Paraffin sections were stained with hematoxylin and eosin-safran for light microscopy. All slides were reviewed by a dedicated uropathologist (NRL). The macroscopic and histological parameters analyzed were: tumor size, multifocality, nucleolar grade according to the International Society of Urological Pathology (ISUP) grading system, sarcomatoid component, tumor necrosis, granular component, lymphocyte infiltrate and microvessel invasion. [START_REF] Delahunt | The International Society of Urological Pathology (ISUP) grading system for renal cell carcinoma and other prognostic parameters[END_REF] Sarcomatoid component was defined as more than 10% involvement of the tumor. Tumor stage was defined by the latest International Union Against Cancer classification (2009). [START_REF] Printz | New AJCC cancer staging manual reflects changes in cancer knowledge[END_REF]

Immunohistochemistry

For each ccRCC case, a representative slide of the tumor with the highest nucleolar grade and the corresponding paraffin block were selected. VEGF (anti-VEGF antibody, sc-152, dilution 1/100; Santa Cruz Biotechnology, Santa Cruz, CA, USA), CAIX (anti-CAIX antibody,

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 8 ab15086
, dilution 1/1500, Abcam, Cambridge, UK), PAR-3 (anti-PAR-3, HPA0300443, dilution 1/50, Sigma-Aldrich, Saint Louis, USA) and PD-L1 (anti-PD-L1 antibody, clone 130021, dilution 1/200, R&D Systems, Minneapolis, MN, USA) expression was assessed by immunohistochemistry as previously described. [START_REF] Patard | Absence of VHL gene alteration and high VEGF expression are associated with tumour aggressiveness and poor survival of renal-cell carcinoma[END_REF][START_REF] Patard | Low CAIX expression and absence of VHL gene mutation are associated with tumor aggressiveness and poor survival of clear cell renal cell carcinoma. International journal of cancer[END_REF][START_REF] Dagher | Cytoplasmic PAR-3 protein expression is associated with adverse prognostic factors in clear cell renal cell carcinoma and independently impacts survival[END_REF] The cut-off for positive cases was 85% of tumor cells for CAIX as described in a previous study. [START_REF] Patard | Low CAIX expression and absence of VHL gene mutation are associated with tumor aggressiveness and poor survival of clear cell renal cell carcinoma. International journal of cancer[END_REF][START_REF] Bui | Carbonic anhydrase IX is an independent predictor of survival in advanced renal clear cell carcinoma: implications for prognosis and therapy[END_REF] The percentage of tumor cells for VEGF was reported. Only cytoplasmic PAR-3 expression in tumor cells was considered positive. [START_REF] Dagher | Cytoplasmic PAR-3 protein expression is associated with adverse prognostic factors in clear cell renal cell carcinoma and independently impacts survival[END_REF] PD-L1 was overexpressed when intensity of membranous or cytoplasmic staining in tumor cells was moderate to strong as previously described. [START_REF] Choueiri | Correlation of PD-L1 Tumor Expression and Treatment Outcomes in Patients with Renal Cell Carcinoma Receiving Sunitinib or Pazopanib: Results from COMPARZ, a Randomized Controlled Trial[END_REF] Regarding tumor infiltrating lymphocytes, CD3 (anti-CD3 antibody, clone SP7, dilution 1/100; Thermo Scientific, Waltham, MA, USA) and CD20 (anti-CD20 antibody, clone L26, dilution 1/25; Dako, Glostrup, Denmark) expression was assessed. The inflammatory extent was coded as one (few sparse lymphocytes in the tumor) or two (marked dense lymphocytes or lymphoid nodules).

IHC scoring was independently assessed by two pathologists (SFKJ and NRL) blinded to the clinical grouping of the specimens. Discordant cases were reevaluated collegially to reach a consensus score.

VHL gene analysis

We determined the complete VHL status for each tumor by analyzing VHL gene mutation, deletion and promoter methylation. VHL mutations were detected by sequencing using denaturing high-performance liquid chromatography (DHPLC). All mutations were confirmed in a second round of PCR and sequencing reactions. VHL gene deletions and promoter methylation were detected by Multiplex Ligation-dependent Probe Amplification (MLPA) analysis using the SALSA MLPA P016B VHL probe kit and the SALSA MS-MLPA kit, respectively. [START_REF] Jeuken | MS-MLPA: an attractive alternative laboratory assay for robust, reliable, and semiquantitative detection of MGMT promoter hypermethylation in gliomas[END_REF] As VHL is a tumor-suppressor gene, VHL gene impairments necessarily involve biallelic alterations in tumor cells as two hits are required to be inactivated. Tumors

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT

9 with two alterations of the VHL gene were defined as inactivated for that gene (inVHL). Those with no or only one alteration were defined as non-inactivated VHL tumors (niVHL).

Statistical analysis

2, Fisher's exact and Mann-Whitney tests were performed to compare qualitative and quantitative parameters, respectively between the synchronous and metachronous metastatic patient groups. Cancer-specific survival (CSS) was calculated from metastasis diagnosis to death from cancer. The Kaplan-Meier method was used to represent CSS, and the resulting curves were compared using log-rank tests. All p-values were 2-sided, and p-values less than 0.05 were considered statistically significant. All statistical analyses were performed using Stata 11.1 (College Station, TX, USA) software.

RESULTS

Patient and tumor characteristics

The population characteristics and pathological parameters are summarized in Table 1. The median age at diagnosis was 61 years (42-80). Twenty-three patients (47.9%) had an ECOG performance status of 0. The mean tumor size was 9cm with tumors ranging from 2 to 18 cm. In 6 cases (12.5%), nodal invasion was present. Twenty-eight patients (58.3%) had synchronous metastases at diagnosis whereas 20 patients (41.7%) developed metastases after initial diagnosis with a 32-month mean (6-78 months). The most common metastatic sites were the lungs in 32 cases (66.7%), bones in 27 cases (56.3%), distant lymph nodes in 16 cases (33.3%) and liver in 7 cases (14.6%). Less common sites were the brain in three cases (6.3%), soft tissue in four cases (8.3%), adrenal gland in five cases (10.4%), pancreas in two cases (4.1%), peritoneum in one case (2.1%), contralateral kidney in two cases (4.1%) and

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT

10 digestive organs in three cases (6.3%). Several sites were frequently involved in 32 cases (66.6%) whereas only one site was involved in 16 cases (33.3%). Before systemic treatment, most patients had intermediate or high risk according to MSKCC or Heng models. When eligible for systemic treatment, the patients received various therapies such as cytokines in 14 cases, sunitinib (first-line or second-line therapy) in 13 cases, sorafenib in two cases, hormone therapy in three cases and standard chemotherapy in two cases. Eleven patients were referred to supportive care specialists and eight patients were not treated in our center. With a followup of 10 years, 43 patients (89.6%) from our study died of their cancer.

Synchronous and metachronous m-ccRCC: a distinct phenotype

The synchronous and metachronous metastatic patient and tumor characteristics are summarized and compared in Tables 2 and3. Synchronous and metachronous m-ccRCC shared the following features: aggressive tumors with symptomatic detection (75%), a median tumor size of 9cm (2-18 cm), a high nucleolar grade (grade 3-4 in 87.5%), tumor necrosis (70.8%), granular component (72.9%), microvascular invasion (60.4%) and overexpression of VEGF (79.2%). However, compared to metachronous m-ccRCC, synchronous m-ccRCC had a poorer ECOG score (p=0.045), a worse risk group in both MSKCC and Heng models (p=0.007 and p=0.010 respectively) and corresponded to even more aggressive tumors with higher pT status (p=0.038) and sarcomatoid component (p=0.007). In the immunohistochemistry analysis, they were associated with overexpression of VEGF with a 50% cut-off (p=0.017) as defined by a receiver operating characteristic (ROC) curve, expression of PAR-3 (p=0.007) and overexpression of PD-L1 (p=0.019). At the molecular level, they were associated with a non-inactivated VHL gene (p=0.01). The pathological and immunohistochemical phenotype of synchronous m-ccRCC is shown in Figure 1.

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT

Worse clinical outcome with synchronous m-ccRCC

Patients with synchronous metastases had a worse CSS from the date of metastasis diagnosis compared to patients with metachronous metastases without statistical differences between the two groups in care management. They had a median survival of 16 months versus 46 months for the patients with metachronous metastases (p=0.01), Figure 3. Five-year survival was 3.6% for patients with synchronous metastases and 20% for patients with metachronous metastases.

DISCUSSION

To our knowledge, this is the first study to compare synchronous and metachronous primary m-ccRCC. While our results may be seen to be limited in terms of sample size, we present a highly detailed study at the pathological, immunohistochemical and molecular levels in a retrospective series of consecutive ccRCC patients with a long-term clinical follow-up of up to ten years. Due to the period of our study, treatments were quite heterogeneous without statistical difference between the two groups. Besides, patients received no prior systemic therapy before nephrectomy.

Primary ccRCC in both synchronous and metachronous metastatic patients is characterized by aggressive and disseminating tumors. However, we report a distinct pathological and molecular phenotype of the primary tumors. In the m-ccRCC group, synchronous m-ccRCC was particularly associated with a higher pT status, sarcomatoid component, cytoplasmic expression of PAR-3, overexpression of VEGF and PD-L1 and a niVHL gene.

Sarcomatoid ccRCC may represent a completed epithelial-mesenchymal transition (EMT). [START_REF] Bostrom | Sarcomatoid conversion of clear cell renal cell carcinoma in relation to epithelial-to-mesenchymal transition[END_REF] Invasion of the basement membrane and extracellular matrix is an essential event in tumor

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 12
progression and considered a critical step during metastasis. [START_REF] Mikami | Invasion and metastasis of renal cell carcinoma[END_REF] Partitioning-defective 3 (PAR-3), a crucial component of partitioning-defective complex proteins, was recently described as an independent prognostic factor in ccRCC. [START_REF] Dagher | Cytoplasmic PAR-3 protein expression is associated with adverse prognostic factors in clear cell renal cell carcinoma and independently impacts survival[END_REF][START_REF] Dugay | Overexpression of the polarity protein PAR-3 in clear cell renal cell carcinoma is associated with poor prognosis. International journal of cancer[END_REF] It is implicated in the development and maintenance of cell polarity. As it modulates cell-cell communication and promotes collective cell migration, PAR-3 may be involved in cancer cell EMT and metastasis formation in ccRCC. [START_REF] Dugay | Overexpression of the polarity protein PAR-3 in clear cell renal cell carcinoma is associated with poor prognosis. International journal of cancer[END_REF] Angiogenesis is also considered a crucial step in the progression of cancer. VEGF expression is a key molecular mechanism underlying the initiation and maintenance of the entire tumor process. [START_REF] Ferrara | The biology of VEGF and its receptors[END_REF][START_REF] Olsson | VEGF receptor signalling -in control of vascular function[END_REF] Tumor progression is often paralleled by higher levels of VEGF expression as cancer cells gradually acquire their malignant potential. [START_REF] Olsson | VEGF receptor signalling -in control of vascular function[END_REF] Interestingly, PD-L1 and the non-inactivated VHL gene were associated with the primary ccRCC in synchronous metastatic patients. The interaction between PD-1, an inducible inhibitory receptor expressed on lymphocytes and dendritic cells, and PD-L1 ligand, expressed by tumor cells, results in down-regulation of the T-cell response. [START_REF] Jilaveanu | PD-L1 Expression in Clear Cell Renal Cell Carcinoma: An Analysis of Nephrectomy and Sites of Metastases[END_REF] PD-L1 overexpression may reflect the ability of metastatic tumor cells to evade immune surveillance during migration. VHL gene inactivation is likely considered as an initial event in the ccRCC carcinogenic process. [START_REF] Gnarra | Mutations of the VHL tumour suppressor gene in renal carcinoma[END_REF] Non-inactivation of the VHL gene more commonly seen in synchronous m-ccRCC may suggest early involvement of non-dependent VHL pathways such as MAP kinase and PI3K-AKT-mTOR pathways that are also implicated in PD-L1 expression. [START_REF] Gallego | Crossing Paths in Human Renal Cell Carcinoma (hRCC)[END_REF][START_REF] Atefi | Effects of MAPK and PI3K pathways on PD-L1 expression in melanoma[END_REF][START_REF] Ritprajak | Intrinsic and extrinsic control of expression of the immunoregulatory molecule PD-L1 in epithelial cells and squamous cell carcinoma[END_REF] Compared to primary tumors of synchronous m-ccRCC, metachronous m-ccRCC probably acquire less oncogenic events. During the period between nephrectomy and symptomatic metastasis, initially dormant tumor cells acquire oncogenic events that are not predictable in primary tumors. [START_REF] Wuttig | Gene signatures of pulmonary metastases of renal cell carcinoma reflect the disease-free interval and the number of metastases per patient. International journal of cancer[END_REF] This explains the heterogeneity between primary tumors and metastasis M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 13 already described. [START_REF] Callea | Differential Expression of PD-L1 between Primary and Metastatic Sites in Clear-Cell Renal Cell Carcinoma[END_REF] Our study emphasizes the need to assess metastatic sites rather than the primary tumor to identify predictive biomarkers for targeted therapies.

In our study, we confirmed the poor prognosis of synchronous metastatic patients. [START_REF] Heng | Prognostic factors for overall survival in patients with metastatic renal cell carcinoma treated with vascular endothelial growth factortargeted agents: results from a large, multicenter study[END_REF] As expected, most of them had time from initial diagnosis to treatment of less than one year that is a risk factor in both MSKCC and Heng models In a recent publication by Beuselinck et al., synchronous m-ccRCC were particularly linked to poor sunitinib response. [START_REF] Beuselinck | Molecular Subtypes of Clear Cell Renal Cell Carcinoma Are Associated with Sunitinib Response in the Metastatic Setting[END_REF] This poor prognosis may rely on advantageous oncogenic events acquired in their primary ccRCC such as PD-L1 expression. If confirmed by further studies, these patients may be good candidates for PD1/PDL1 immunotherapy.

CONCLUSION

In this long-term study of metastatic patients, we revealed that synchronous m-ccRCC had a distinct phenotype which is likely explained by the occurrence of oncogenic events. Patients with synchronous m-ccRCC have a worse prognosis and could benefit from specific therapy.

Clinical practice points:

• Patients with synchronous metastatic clear cell renal cell carcinoma (ccRCC) are already known to have a worse prognosis than patients with metachronous metastatic ccRCC.

• For the first time, we compared synchronous and metachronous primary tumors at pathological, immunohistochemical and molecular levels and found a different phenotype of synchronous primary ccRCC.

  metastatic patients. Determining whether there is a phenotypic difference between these two groups could have important clinical implications.

  obtained from the processing of biological samples by the Rennes Biological Resources Center-Health (CRB-Health) (BB-0033-00056). The research protocol was conducted under French legal guidelines and fulfilled the requirements of the local institutional ethics committee. All consecutive ccRCC and paired renal cortex samples were analyzed. Immediately after macroscopic examination, small samples were collected from surgical specimens, frozen in liquid nitrogen and stored at -80°C until DNA extraction.

14 •

 14 Patients with synchronous metastasis overexpressed PD-L1 and may be good candidates for PD1/PDL1 immunotherapy.M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT

1 TABLES Table 1 . Summary of the clinical and histopathological characteristics of the 48 metastatic patients.

 TABLES1

	Variables	Number of patients (%)
	Sex
	M	27 (56.3%)
	F	21 (44.7%)
	ECOG
	0	23 (47.9%)
	1	25 (52.1%)
	Age (years)	61 (42-80)
	Mode of detection
	Incidental	12 (25%)
	Symptomatic	36 (75%)
	Tumour size (cm)	9 (2-18)
	ISUP grade
	2	6 (12.5%)
	3	18 (37.5%)
	4	24 (50%)
	Tumour stage (pT)
	1	5 (10.4%)
	2	11 (22.9%)
	3	28 (58.3%)
	4	4 (8.3%)
	Lymph node status (pN)
	0	42 (87.5%)
	1-2	6 (12.5%)
	Metastasis status (pM)
	0	20 (41.7%)
	1	28 (58.3%)
	MSKCC score
	Favorable prognosis	8 (16.7%)
	Intermediate prognosis	27 (56.3%)
	Poor prognosis	13 (27.0%)
	Heng criteria
	Favorable prognosis	9 (18.7%)
	Intermediate prognosis	19 (39.6%)
	Poor prognosis	20 (41.7%)

ACKNOWLEDGMENTS

The authors would like to acknowledge the Ligue Contre le Cancer, the CORECT, Rennes Hospital and the French Institute of Cancer (INCa) for their financial aid.

The authors would also like to thank the Center of Biological Resources of Rennes Hospital (BB-0033-00056, http://www.crbsante-rennes.com/) for managing patient samples as well as Pascale Bellaud and Roselyne Viel from the Histopathology platform H2P2-BIOSIT, Faculty of Medicine of Rennes for their technical support.

DISCLOSURE

The authors of this article have no relevant financial relationships with commercial interests to disclose and no funding to declare.

M A N U S C R I P T A C C E P T E D