Dafnis Krasniqi
email: dafnis.krasniqi@etu.univ-paris1.fr

Jean-Marc Bardet

Joseph Rynkiewicz

Parametric and XGBoost Hurdle Model for estimating accident frequency

Keywords: Insurance pricing, XGBoost algorithm, GLM, Hurdle Model

The Poisson model is a commonly used method for modeling count data with exogenous variables, but it can be limiting when dealing with data that has a high proportion of zeros. To address this issue, we propose the use of the Hurdle model as an alternative approach. We discuss the properties of the Hurdle model and demonstrate how it can be implemented using both parametric and nonparametric estimates, including the XGBoost method. To evaluate the effectiveness of our proposed XGBoost Hurdle model, we apply it to a car insurance dataset from a French insurance company. This dataset includes a significant number of drivers with zero accidents per year. Our results show that the XGBoost Hurdle model outperforms several other models when applied to this data type.

Introduction

The Poisson distribution is commonly used to model count data in relation to exogenous variables, but it becomes inappropriate when the response variable y has mostly zero values and exhibits overdispersion (variance exceeding the mean). In such cases, an alternative regression model is necessary. Some options for handling overdispersion in data with mostly zero-valued response variables include the negative Binomial distribution (as described by [START_REF] Cameron | Regression-based tests for overdispersion in the Poisson model[END_REF], [START_REF] Cameron | Microeconometrics: Methods and Applications[END_REF], Hilbe (2011)), the Zero-Inflated model [START_REF] Lambert | Zero-inflated poisson regression, with an application to defects in manufacturing[END_REF]), and the Hurdle model [START_REF] Mullahy | Specification and testing of some modified count data models[END_REF], [START_REF] Heilbron | Zero-altered and other regression models for count data with added zeros[END_REF]). The negative Binomial distribution is a generalization of the Poisson distribution that allows for overdispersion, often used in count data models with a high number of zeros and a long tail of positive values. It can be written as an extension of the Poisson model with an additional parameter for overdispersion. The Zero-Inflated model is used for data with an excess of zero values and consists of two parts: a binomial model for the probability of a zero value occurring and a Poisson model for the count data, given that the value is not zero. The Hurdle model is similar, with a binary model for the probability of a non-zero value occurring and a count model for the count data given that the value is non-zero. The Zero-Inflated and Hurdle models can be useful for data with a high proportion of zero values and overdispersion. This paper proposes a novel approach to modeling zero excess count data using a Hurdle model with statistical learning models, specifically the XGBoost method [START_REF] Chen | XGBoost: A Scalable Tree Boosting System[END_REF]). Traditional Hurdle models often rely on parametric or semi-parametric estimates, which can have limitations such as assumptions of equal error variances, a default distribution for the response variables, and a linear relationship between the dependent and independent variables. These assumptions may not always hold in real data, and parametric approaches may not be able to adequately model complex, sophisticated, non-linear relationships, and high-degree interactions. Using nonparametric models, such as boosting models or the XGBoost method, we can model complex relationships and interactions without requiring stringent assumptions.

Hurdle count data models were first introduced by [START_REF] Mullahy | Specification and testing of some modified count data models[END_REF]. Since then, they have garnered significant attention in various fields, with many studies exploring their applications and properties. [START_REF] Gurmu | Generalized hurdle count data regression models[END_REF] proposed a generalization of hurdle models for analyzing overdispersed or underdispersed count data and applied their research to the analysis of Medicaid utilization. In a separate study, [START_REF] Greene | Functional Form and Heterogeneity in Models for Count Data[END_REF] compared Hurdle and zero-inflated models, providing insight into these approaches' relative strengths and limitations. [START_REF] Rudra | Models for analyzing over-dispersed hurdle negative binomial regression model: an application to manufactured cigarette use[END_REF] used the Hurdle model to analyze the use of manufactured cigarettes. In their study, many smokers did not purchase manufactured cigarettes, resulting in many zero values in the target variable. Hurdle models constructed with statistical learning models have also been discussed in the literature. [START_REF] Povak | Machine learning and hurdle models for improving regional predictions of stream water acid neutralizing capacity[END_REF] proposed a Hurdle model for detecting pollution in rivers. They used a random forests model based on the large number of zero values in the target variable for the non-polluted rivers. [START_REF] Kong | Deep Hurdle Networks for Zero-Inflated Multi-Target Regression: Application to Multiple Species Abundance Estimation[END_REF] proposed a Hurdle model for detecting species in nature using a dataset with unbalanced classification. Their model was constructed from two neural networks. Hurdle models have been applied in the insurance industry to analyze the number of claims made by insured drivers. [START_REF] Boucher | Modelling of Insurance Claim Count with Hurdle Distribution for Panel In[END_REF] demonstrated that the Hurdle model is a useful alternative to classical Poisson or negative binomial models. [START_REF] Zhang | A new multivariate zero-inflated hurdle model with applications in automobile insurance[END_REF] also developed a multivariate Hurdle model using the expectation-maximization (EM) algorithm. Previous research has primarily relied on parametric models for model construction. To our knowledge, no studies have applied nonparametric estimation in Hurdle models for insurance data.

The paper is structured as follows: Section 2 introduces the classical Hurdle model and describes its parametric estimation. We also demonstrate that the Hurdle model can be decomposed into two independent models without loss of information. In Section 3, we explore nonparametric estimation using statistical learning techniques and develop prediction models. In Section 4, we apply the proposed models to a real-world data set, a car insurance portfolio of a French insurance company. Finally, in Section 5, we discuss the results and implications of our analysis.

The Poisson Hurdle model

The Poisson Hurdle model, also known as the "two-part model," is a well-known parametric model for predicting count data. It consists of two distinct processes: one that generates zeros and one that generates positive integers. The Hurdle model posits that a Bernoulli random variable, which depends on the exogenous variables, determines whether the count variable takes on a zero or positive value. The distribution of positive values, also dependent on the exogenous variables, is governed by a positive integer-valued distribution. These two processes are assumed to be independent of each other. This section introduces the classical parametric Hurdle model and discusses methods for estimating its parameters and selecting the most appropriate model. This widely used and well-established approach serves as the foundation for our analysis.

Statistical models

Problem Statement Let D be a data set defined:

D = {(x i , y i ) 1≤i≤n , with x i ∈ R p , y i ∈ {0, 1, 2, ..} for 1 ≤ i ≤ n} .
The exogenous variables are represented by a R p -vector x and are supposed to explain the response variable y. Our goal is to propose a model to predict this variable y from x.

Definition 1 The Poisson Hurdle regression model is defined for all i = 1...n as:

P [Y i = y i | x i , π i , λ i ] =    π i if y i = 0 (1 -π i ) λ yi i (e λi -1)y i ! if y i > 0
where π i represents the probability of the binary part of the model and λ yi i (e λi -1)y i ! represents the probability of a positive count as determined by the Poisson zero-truncated distribution.

A critical issue in modeling count data is selecting appropriate link functions for the probability distribution. A common parametric choice is to use a logit link for the parameter π i and a log link for the parameter λ i .

i ) = x T i β 1 log(λ i ) = x T i β 2 π i = e x T i β 1 1+e x T i β 1 λ i = e x T i β 2
In Table 1, β 1 represents the vector of regression coefficients for the covariates x i in the Bernoulli model. In contrast, β 2 represents the vector of regression coefficients for the covariates x i in the Poisson zerotruncated model. In both models, we can select the variables x i ∈ R p that we wish to include in the analysis.

The expectation of the hurdle model can be calculated as the sum of the Bernoulli component's expected value and the count component's expected value. Specifically, let Y be the random variable representing the count data, and let π i be the probability of observing a zero value and λ i be the expected count for non-zero values. Then, the expectation of the hurdle model can be written as follows:

E(Y i |x i , π i , λ i ) = π i • 0 + (1 -π i ) • E(Y i | Y i > 0) = (1 -π i ) • λ i e λi e λi -1
The expected value of the Bernoulli component is simply zero since it can only take on values of 0 or 1. The expected value of the count component, on the other hand, is equal to the expected count for non-zero values, E(Y i | Y i > 0), multiplied by the probability of observing a non-zero value, (1 -π).

The variance of the hurdle model can be calculated as the sum of the Bernoulli component's variance and the count component's variance.

V ar(Y

i |x i , π i , λ i ) = (1 -π i ) • V ar(Y i | Y i > 0) + π i • (1 -π i ) • [E(Y i | Y i > 0)] 2 = (1 -π i ) • λ i 1 -e -λi + π i -e -λi • λ i 1 -e -λi 2
where π i is the probability of observing a zero value and λ i is the expected count for non-zero values. Note that the Hurdle models, therefore, take into account the overdispersion since the variance is greater than the expectation (E

[Y ] ≤ V ar[Y ]).

Estimation of the parameters

There are several ways to estimate the regression parameters (β 1 and β 2 ) for the hurdle model. One common approach is to use maximum likelihood estimation (MLE). Assuming that the observations Y 1 , . . . , Y n are independent, the log-likelihood function is given by:

log(L(β 1 , β 2 , y)) = i∈Ω0 log e x T i β1 1 + e x T i β1 + i∈Ω1 log 1 - e x T i β1 1 + e x T i β1 + i∈Ω1 log   e x T i β2 yi y i !(e e x T i β 2 -1)   (1)
with Ω 0 = {i|y i = 0}, Ω 1 = {i|y i ̸ = 0}, β 1 and β 2 are the vectors of regression coefficients for the Bernoulli and count components, respectively. The log-likelihood function of the Hurdle model ( 1) can be decomposed into two independent loglikelihood functions, which allows us to study each component separately and potentially improve the overall performance.

log(L(β 1 , β 2 , y i )) = log(L 1 (β 1 , y)) + log(L 2 (β 2 , y))
We obtain maximum likelihood estimates by separately maximizing log(L 1 (β 1 , y)) and log(L 2 (β 2 , y)).

log(L 1 (β 1 , y)) = i∈Ω0 log e x T i β1 1 + e x T i β1 + i∈Ω1 log 1 - e x T i β1 1 + e x T i β1 log(L 2 (β 2 , y)) = i∈Ω1 log   e x T i β2 yi y i !(e e x T i β 2 -1)   = i∈Ω1 y i log e x T i β2 - i∈Ω1 log (y i !) + i∈Ω1 log e e x T i β 2 -1 (2)
As a result, β1 = argmax

β1∈R p+1 log(L 1 (β 1 , y)) β2 = argmax β2∈R p+1 log(L 2 (β 2 , y))
However, such an estimator does not have a closed-form expression. Therefore, we need to use iterative algorithms, such as the Newton-Raphson or gradient algorithms, to estimate the value of β. To select the best exogenous variables, several techniques can be used, such as the AIC (see [START_REF] Akaike | Information Theory and an Extension of the Maximum Likelihood Principle[END_REF]) or BIC (see [START_REF] Schwarz | Estimating the Dimension of a Model[END_REF]) criteria. These model selection methods can help us identify the combination of variables that best explains the data while avoiding overfitting.

AIC = -2 • log(L( β 1 , β 2 , y)) + 2 • k BIC = -2 • log(L( β 1 , β 2 , y)) + log(n) • k with log(L( β1 , β2 , y))
is the maximized value of the likelihood function, k the number of estimated parameters and n the sample size.

Therefore, increasing the value of k will generally result in a higher AIC and BIC, indicating a lowerquality model. When comparing multiple models, selecting the model with the lowest AIC or BIC is usually recommended, as this indicates the best balance between model fit and parsimony.

3 The XGBoost Hurdle model

The traditional Hurdle model consists of two components: a binary variable that determines whether the response variable is zero or positive and a positive integer-valued variable that models the positive data using a zero truncated Poisson regression. However, these parametric or semi-parametric approaches may not be able to model complex, non-linear relationships and high-degree interactions. Using XGBoost, a nonparametric statistical learning method, we can model these complex relationships without the need for stringent assumptions. We optimize a logistic loss function and a Poisson zero truncated loss function using XGBoost to construct our new Hurdle model, representing a novel contribution to the literature, particularly in actuarial science.

Incorporating nonparametric models, such as those optimized using the XGBoost algorithm, can significantly improve model performance when parametric models are insufficiently flexible to capture the complexity of the data. This is particularly useful in actuarial science, where accurate modeling of insurance risks is crucial for making informed business decisions. Our approach offers a flexible and effective solution for modeling count data in the insurance industry by using nonparametric models to optimize the logistic and Poisson zero truncated loss functions. One key advantage of our approach is the ability to directly optimize the models on the data rather than relying on predetermined distributional assumptions. This allows the models to capture the underlying patterns and relationships in the data, leading to improved predictive accuracy. Additionally, using the XGBoost algorithm allows for efficient optimization of the models, making it a practical and scalable approach for analyzing large datasets.

XGBoost

XGBoost is a tree-based machine learning method introduced by Chen and Guestrin (2016). It builds multiple trees, each learning from and improving upon the previous one. These weak models are then combined to create a more robust model. Compared to other tree-based methods, XGBoost has two main advantages. First, it is computationally efficient because it processes data in compressed blocks, allowing it to be quickly ordered and processed in parallel. Second, it uses second-order Taylor expansion to find the minimum of the objective function. This means that it takes into account the curvature of the objective function, allowing it to converge to a better minimum than other methods.

As an example, consider the objective function of the XGBoost model at the t-th iteration:

L (t) = n i=1 ℓ(y i , ŷ(t-1) + f (x, α t )) + Ω(f (x, α t ))
where ℓ is the loss function, f (x, α t ) the output of the t-th tree, α t the parameters concerning the t tree, and Ω is the regularization. This regularization can take the form of the L1 or L2 regularization or a linear combination of both.

One of the (many) critical steps for a fast computation is the approximation:

L (t) ≈ n i=1 ℓ(y i , ŷ(t-1) ) + g i f (x i , α t ) + 1 2 h i f 2 (x i , α t ) + Ω(f (x i , α t ))
with g i = d dŷ ℓ(y i , ŷ) and h i = d 2 dŷ 2 ℓ(y i , ŷ) . The second-order Taylor approximation is easy to compute because most terms are the same as in a given iteration. For a given iteration, the expression can be computed once and reused as a constant for all splits:

L (t) ≈ n i=1 ℓ(y i , ŷ(t-1) ) constant + g i constant f (x i , α t ) + 1 2 h i constant f 2 (x i , α t ) + Ω(f (x i , α t ))
So, the only thing left to calculate is f (x i , α t ) and Ω(f (x i , α t )).

Part 1: Binary XGBoost

The first model we consider is a statistical learning model with a loss function based on the log-likelihood of logistic regression. This loss function is provided in the Xgboost library called the binary loss function.

Modifying the initial data slightly is necessary to build such a binary model. In particular, it is required to define a new target variable, which is defined as follows:

y * i → 1 if y i ∈ Ω 1 , 0 if y i ∈ Ω 0 .
where Ω 0 = {i|y i = 0} and Ω 1 = {i|y i ̸ = 0} To learn about binary data, let's define the following model: the loss function ℓ 1 associated with this problem can be written as follows:

ℓ 1 (y * i ; f (x i , α t )) = - 1 n n i y * i × log(f (x i , α t )) + (1 -y * i ) × log(1 -f (x i , α t ))
Where ℓ 1 is the loss function and f (x i , α t ) is the output of the t-th tree. To compute a probability, the model will sum all the trees and put them into a sigmoid function:

π M L (x i ) = e M m=1 ηf (xi,αm)
1 + e M m=1 ηf (xi,αm)

.

(3) with π M L (x i ) the final output, M the total number of trees in the XGBoost model, and η the learning rate. The parameters M and η are obtained by cross-validation. The result of (3), we obtain values between 0 and 1 (π M L i ∈ [0, 1]) that can be interpreted as a probability: π M L i and 1 -π M L i are respectively is the probability of belonging to class 0 and 1.

Part 2: XGBoost of zero-truncated Poisson distribution

The second part of the XGBoost Hurdle model is a zero-truncated Poisson model devoted to predicting the positive number of accidents. Here again, we consider a change of the response variable y. The original target variable y will be truncated by 0, and only the positive values will be kept. The new target variable for this model is defined as follows:

ỹi = y i , for i ∈ Ω 1 with Ω 1 = {i|y i ̸ = 0} .
Concerning the database X, we will also remove some rows. For example, x ⧸0 is the database without the rows with a value for the target variable equal to 0. To learn this data, a second loss function will be defined.

ℓ 2 (ỹ i ; f (x i , α t )) = - 1 n n i=1 (ỹ i × log(f (x i , α t )) -f (x i , α t )) (4)
Where ℓ 2 is the loss function, f (x i , α t ) is the output of the t-th tree. This loss function is available by default in the XGBoost model when a target variable y follows a Poisson distribution. Unfortunately, this loss function (4) does not work in our case because a loss function corresponding to the log-likelihood of the zero-truncated Poisson distribution is needed. To this end, a new loss function will be coded in XGBoost. This loss function is one of the contributions of this paper.

By coding a new loss function specifically tailored to our research needs, we have made a significant contribution to the field of modeling count data. Our novel loss function is based on the Poisson zero truncated distribution, which is commonly used for modeling count data with excess zeros. It allows for the modeling of overdispersion and zero-inflation, standard features of count data in many applications. Implementing this loss function in XGBoost enables the efficient optimization of nonparametric models for count data using the XGBoost algorithm.

New loss function

To construct the new loss function, we start from the maximum likelihood of the zero-truncated Poisson distribution (2). In the XGBoost, we have to minimize functions, so the negative of the function (2) will be used. In addition, the parametric part will be replaced (λ i = e xiβ2 ) by a nonparametric part (λ i = f (x i , α m )) and create a loss function dedicated to the boosting model.

ℓ 2 (ỹ i ; f (x i , α t )) = - 1 |Ω 1 | i∈Ω1 (ỹ i × log(f (x i , α t )) -log(e f (xi,αt) -1)) (5)
where ℓ 2 is the loss function, f (x i , α t ) is the output of the t-th tree and Ω 1 = {i|y i ̸ = 0}.

Three ingredients are needed to implement a new loss function in XGBoost: the first derivative of the equation ( 5), the second derivative of the equation ( 5) and a function to evaluate. For the first derivative of ( 5) is equal to :

∂ℓ 2 (ỹ i ; f (x i , α t )) ∂f (x i , α t ) = - ỹi f (x i , α t ) + e f (xi,αt) e f (xi,αt) -1
and the second derivative of ( 5) is :

∂ 2 ℓ 2 (ỹ i ; f (x i , α t )) ∂f (x i , α t ) 2 = ỹi f (x i , α t ) 2 - e f (xi,αt) (e f (xi,αt) -1) 2
The equation ( 5) will be directly used for the evaluation function. The implementation of these functions in Python is available in the appendix. As for the output result, it is the expectation of the zero-truncated Poisson distribution: E[Y i ] = λie λ i e λ i -1 . The output value of our boosting model will be as follows:

λ M L (x i ) = M m=1 β m f (x i , α m ) × e M m=1 βmf (xi,αm) e M m=1 βmf (xi,αm) -1 (6)
with the result of ( 6), values greater than 1 are obtained.

The prediction strategy

Once the two sub-models have been defined, we can specify the prediction strategy for our new XGBoost Hurdle model.

Definition 2

The prediction ŷn+1 for a new vector x n+1 is given by:

ŷn+1 = ⌊μ n+1 ⌉ with μn+1 = (1 -π M L (x n+1 )) × λ M L (x n+1 )
where π M L (x n+1 ) is the predicted probability of the XGBoost binary model (see (3)), λ M L (x n+1 ) is the predicted number of the XGBoost zero-truncated Poisson (see (6)) and ⌊x⌉ represents the nearest integer at x.

μn+1 is the predicted mean of the XGBoost Hurdle model. It is a method used to analyze the count data and improves the parametric hurdle model because it uses statistical learning models instead of parametric models to make predictions.

Application on insurance data

This section will first provide a brief overview of the data and explain why it is highly relevant to our situation. Next, we will describe the various measures used to evaluate the quality of our results. Finally, we will apply our custom-designed algorithms to specific insurance data and analyze the resulting output carefully.

The data

To make a numerical illustration of our different models, the FreMTPL2freq database will be used. It contains 678,013 automobile liability contracts (see Christophe Dutang (n.d.)). This dataset is very frequently used in automobile insurance and has been the subject of many studies [START_REF] Noll | Case Study: French Motor Third-Party Liability Claims[END_REF], [START_REF] Denuit | Autocalibration and Tweediedominance for Insurance Pricing with Machine Learning[END_REF], [START_REF] Pocuca | Modeling Frequency and Severity of Claims with the Zero-Inflated Generalized Cluster-Weighted Models[END_REF], [START_REF] Miljkovic | On Two Mixture-Based Clustering Approaches Used in Modeling an Insurance Portfolio[END_REF]). Here is the list of features and their descriptions in the table below. The variable y i corresponds to the number of accidents (ClaimN B) suffered by the driver i during the year. This variable is a discrete variable {0, 1, 2, ...}. The Exposure variable e i corresponds to the period of the insured's underwriting and his exposure to the insurance risk. This variable is essential in the models as it plays the role of a weighting coefficient. Two accidents in 6 months (e i = 0.5) and two accidents in 1 year (e i = 1) are not the same. This exposure variable will therefore aim to harmonize contracts with unequal subscription periods.

Experiment parameters and optimization

To improve the performance of our models, we apply a series of restatements to the variables in our data. This helps to clean the database and eliminate errors and extreme values. This process involves two key steps:

1. To ensure the accuracy and reliability of our predictions, we will cap the number of claims at three in the target variable. This is because accidents become extremely rare and unpredictable after the third claim. Therefore, we will treat drivers with more than three accidents as if they had only three accidents;

2. The different algorithms do not accept categorical variables, so we will transform them into continuous variables.

Our data only contains nominal qualitative variables, so we will use numerical encoding to represent these variables without considering any inherent hierarchy. This will result in a matrix with column names representing the different modalities for each qualitative variable. Before performing this transformation, we will group some modalities to avoid creating excessively sparse matrices with many zeros. For quantitative variables, no transformation is necessary, and we can directly input them into the algorithms.

Five algorithms are applied to the freMTPL2freq data: GLM Poisson, Hurdle, Decision Tree, Poisson Boosting Machine (PBM) with deep 3 and 5, and the XGBoost Hurdle. The decision tree and the Poisson Boosting Machine are deduced from the article by [START_REF] Noll | Case Study: French Motor Third-Party Liability Claims[END_REF] 1 . Therefore, there are two parametric models and three nonparametric models. Regarding the database, it is divided into two parts: a training set (80% of the database) and a testing set (20% of the database). When the nonparametric methods are applied, the training set is further divided into a validation set to adjust the hyperparameters. A Monte Carlo Cross-Validation with 30 different bootstrapped samples is performed for each method. This method allows us to obtain robust results. For more information, the various types of cross-validation are discussed in detail in the thesis of [START_REF] Cornec | Probability bounds for the cross-validation estimate in the context of the statistical learning theory and statistical models applied to economics and finance[END_REF].

The stepwise approach is used to optimize the parametric Poisson GLM model to minimize the BIC with the library glm on R (see [START_REF] Marschner | glm2: Fitting generalized linear models with convergence problems[END_REF]. Then, the Hurdle is optimized with the same approach with the pscl library on R (see [START_REF] Jackman | pscl: Classes and Methods for R Developed in the Political Science Computational Laboratory[END_REF]. To optimize the Decision Tree and the Poisson Boosting Machine (PBM) [START_REF] Noll | Case Study: French Motor Third-Party Liability Claims[END_REF], use the library rpart on R.

For the XGBoost Hurdle model, XGBoost and Optuna library (see [START_REF] Akiba | Optuna: A Next-generation Hyperparameter Optimization Framework[END_REF] will be used to perform our modeling. XGBoost methods require a lot of parameters that need to be perfectly tuned.

To help us in this task, the library Optuna will be used. This library aims to find the best parameters randomly for each model. The user only has to give a range of parameters for the model to be tested, and the library will do the necessary by performing random tests. An early stop will be implemented to provide a model that generalizes well over all the data and not just the training data. If the score on the training set continues to decrease during the construction phase while the score on the validation set increases.

Comparisons of the methods

To compare our models, we will use three different measures:

Poisson deviance:

The Poisson deviance is closely related to the negative log-likelihood function, which is often used as a loss function in Poisson regression. The log-likelihood function for the Poisson distribution is given by:

L(y, μ) = n i=1
(y i log μi -μi -log y i !)

1 the codes of these two models are available in the following GitHub see here where y i is the observed count for the ith observation, and μi is the predicted count for the ith observation.

The Poisson deviance can be obtained by subtracting the log-likelihood from its saturated value:

d(y i ; μi ) = 2L saturated -2L(y, μ) d(y i ; μi ) = 2 n i=1 y i log y i μi -(y i -μi )
where L saturated is the negative log-likelihood function for a saturated model. A saturated model is a model that perfectly fits the data. The Poisson deviance can therefore be interpreted as the deviation of the model from a saturated model, with a smaller deviance indicating a better fit of the model to the data.

Comparing the model predictions with the observed value:

In this analysis, we compare the sum of the predicted values to the sum of the observed values to measure the difference between the two. A smaller difference indicates a better fit of the model to the data.

d(y i ; μi ) = n i μi - n i y i n i y i × 100
The confusion matrix (C-M):

We constructed a confusion matrix because the first two metrics were insufficient for understanding the model's performance. These metrics only provided a broad overview of the model's accuracy without giving insight into its ability to predict rare events. The confusion matrix allowed us to examine the model's predictions in more detail and evaluate its performance in extreme cases. To build this matrix, we used the predicted values ŷi .

Results

In this section, the results of the different algorithms will be analyzed and compared on a test and training basis. Three models of [START_REF] Noll | Case Study: French Motor Third-Party Liability Claims[END_REF] are also included. The authors of this paper seek only to optimize the Poisson deviance but not the confusion matrices. We used their open-source code to compute the missing criteria.

Conclusion

In this study, we examined the effectiveness of a Hurdle model for modeling zero excess count data in the insurance industry. We found that the Hurdle model can be decomposed into a binomial regression and a zero-truncated Poisson regression. To improve upon traditional parametric Hurdle models, we introduced a novel approach using the XGBoost library and a custom loss function for zero-truncated Poisson regression, which we refer to as the XGBoost Hurdle model. This contribution is valuable to the existing literature on count data modeling in insurance, offering a flexible and effective alternative to traditional methods.

To compare the performance of our novel model, we also evaluated four other methods (two parametric and two nonparametric models). Our results showed that the XGBoost Hurdle model outperformed all other methods across all criteria. However, a limitation of the XGBoost Hurdle model (as well as the other methods evaluated) is that it cannot accurately identify drivers with multiple claims. To address this issue, we suggest further improvements, such as the use of double Hurdles or a cascade structure to address data imbalance. Overall, our results demonstrate the efficacy of the XGBoost Hurdle model in predicting driver behavior and distinguishing between good and risky individuals in the insurance industry.

  Figure 1: Poisson deviance on the training and test set in both boxplots.

Table 1 :

 1 The link functions for the two models

	Bernoulli Model Poisson zero truncated Model
	Logit(π

Table 2 :

 2 The features in FreMTPL2freq

	Features	Description	Type
	ClaimNb	the number of accidents (between 0 and 5) continuous
	Exposure	Exposure (in a fraction of years)	continuous
	BonusMalus Bonus/malus, between 50 and 350	continuous
	Power	the power of the car (from 4 to 48)	continuous
	CarAge	the age of the vehicle (in years)	continuous
	DriverAge	the age of the driver (in years)	continuous
	Brand	the brand of the car	categorical
	Gas	Gas of the car: "Diesel" or "Regular"	categorical
	Region	the regions (in France)	categorical
	Density	number of inhabitants per km2	continuous
	Area	The density value of the community	categorical

Table 3 :

 3 The distribution of the variables ClaimNb and Exposure is shown. In this dataset, there are 643,953 drivers with no claims, with exposure to the risk of 336,616. Then there are 32,178 drivers with one claim, with exposure to the risk of 20,670. And so on.

	ClaimNb	0	1	2	3	4	5	6
	Nombre	643,953	32,178	1,784	82	7	2	1
	Sum Exposure 336,616.1 20,670.8 1,153.4 52.8 3.1 1.1 0.3

Table 4

 4

			: C-M
		ŷi = 0 ŷi = 1 ... ŷi = n
	y i = 0	..	..	..	..
	y i = 1	..	..	..	..
	...	..	..	..	..
	y i = n	..	..	..	..

Table 5 :

 5 Results: The different models are in the first column. The results of the Poisson deviance for the training and test sets are described in the second and third columns. In the fourth and fifth columns, the comparison score of the sum of predictions and the sum of observations are described for the training and test sets.

		P. deviance train P. deviance test ̸ = accidents train ̸ = accidents test
	GLM Poisson	31.33	31.53	0.00%	-0.71%
	Hurdle	30.99	31.15	-1.18%	-1.85%
	Decision Tree	30.79	31.04	-1.68e-02%	-0.68%
	PBM3	30.20	30.66	1.22e-04%	-0.61%
	PBM5	29.51	30.74	2.67e-05%	-0.40%
	XGBoost Hurdle	26.71	29.88	0.39%	-0.38%

In terms of Poisson deviance, both the Hurdle model with parametric models (average scores of 30.99 on the training set and 31.15 on the test set) and the GLM Poisson (average scores of 31.33 on the training

Acknowledgements

The authors would like to thank the entire staff of Allianz France for their help and funding.

According to the two graphs 1a and 1b, it is clear that the XGBoost Hurdle model performs the best. These graphs contain box plots for each method, with 30 values plotted. The box plot for the XGBoost Hurdle model is significantly farther from the others for the training set, indicating that it may have slightly overfitted the data. However, it also performs well on the test set.

Table 6: C-M GLM Poisson; train ŷi = 0 ŷi = 1 ŷi = 2 ŷi = 3 y i = 0 547056 311 1 0 y i = 1 27240 86 1 0

Looking at the confusion matrices for the test data, we should remember that we are faced with a complicated problem: working on a counting model with extremely unbalanced data.

• First, the two parametric models are found to perform poorly. They fail to detect potentially dangerous drivers accurately. The GLM can only detect 16 dangerous drivers on average. The worst-performing model is the Hurdle model with the parametric models. The latter classifies all drivers as ŷ = 0. This model fails to detect drivers with very high risks.

• The two models of Noll, Salzmann, and Wuthrich (2018) (PMB3 and decision trees) are much better than the parametric models. They manage to detect more potentially dangerous drivers. Among these two models, the best is the Poisson Boosting Machine (see ( 13)). On the test set, it detects 68 potential drivers with one claim.

• The XGBoost Hurdle model showed the best performance on the test set, accurately identifying 117 potentially risky drivers with a single claim (see ( 15)). While there were some misclassifications for this model, the results were still favorable compared to the other models. For example, the PMB3 model identified 68 good potential drivers with one claim but also had 180 misclassifications (180/68 = 2.65), while the XGBoost Hurdle model had a lower ratio of 168/117 = 1.44 misclassifications per good prediction. Overall, the XGBoost Hurdle model demonstrated a solid ability to distinguish between good and risky drivers. Some speculation for the misclassifications: the algorithm classifies these drivers in this way for two reasons. These drivers have all the characteristics of drivers who are likely to have accidents but have not had any. They may have been involved in accidents during the year but did not report them to their insurance company because they preferred to pay them in cash (see [START_REF] Lemaire | Automobile Insurance: Actuarial Models[END_REF]).

The XGBoost Hurdle model outperformed the other methods in analyzing automobile insurance actuarial data. It successfully identified many potentially risky drivers with only one claim and had a lower misclassification rate than the other models. This demonstrates the model's ability to accurately predict driver behavior and distinguish between good and risky individuals. Its strong performance on the test set further confirms its effectiveness in analyzing actuarial data.

Appendices

Zero-truncated poisson loss function with python def f i r s t g r a d ( predt , d t r a i n ) : ' ' ' Compute t h e f i r s t d e r i v a t i v e . ' ' ' y = d t r a i n . g e t l a b e l ( )


Fajar Haifani
email: fajar.haifani@mpi-inf.mpg.de

Patrick Koopmann
email: patrick.koopmann@tu-dresden.de

Sophie Tourret
email: sophie.tourret@inria.fr

Christoph Weidenbach
email: christoph.weidenbach@mpi-inf.mpg.de

Connection-Minimal Abduction in ℰℒ via Translation to FOL Extended Abstract

Abduction is about finding extensions to a knowledge base that are sufficient to imply a given entailment. Specifically, in the context of description logics (DPs), we consider an ontology 𝒪 of background knowledge, an axiom 𝛼 s.t. 𝒪 ̸ |= 𝛼 (called the observation), and we want to compute a suitable set ℋ of axioms called hypothesis such that 𝒪 ∪ ℋ |= 𝛼 [START_REF] Elsenbroich | A case for abductive reasoning over ontologies[END_REF]. Abduction as a reasoning problem has a long history [START_REF] Peirce | Deduction, induction, and hypothesis[END_REF] and has several applications: for example, it can be used to explain missing entailments of an ontology [START_REF] Koopmann | Two ways of explaining negative entailments in description logics using abduction[END_REF][START_REF] Ceylan | Explanations for negative query answers under existential rules[END_REF][START_REF] Calvanese | Reasoning about explanations for negative query answers in DL-Lite[END_REF], to repair incomplete knowledge bases [START_REF] Wei-Kleiner | Abduction framework for repairing incomplete ℰℒ ontologies: Complexity results and algorithms[END_REF], and to provide possible explanations for unexpected observations [START_REF] Obeid | Using description logic and Abox abduction to capture medical diagnosis[END_REF]. We consider the special case of TBox abduction in the lightweight description logic ℰℒ, where the observation is an ℰℒ concept inclusion 𝐶 1 ⊑ 𝐶 2 and the background knowledge is an ℰℒ TBox 𝒯 . To avoid useless answers, abduction problems usually come with further restrictions on the solution space and/or minimality criteria that help sort the chaff from the grain. We argue that existing minimality notions suffer from certain limitations, and introduce connection minimality as a new notion that overcomes the limitations of earlier notions. Furthermore, we developed and evaluated a method to compute connection-minimal solutions in practice.

Our criterion follows Occam's razor by rejecting hypotheses that use concept inclusions unrelated to the problem at hand. To illustrate, consider the following TBox about relationships in academia:

𝒯 a = {
ℋ 1 = {PhD ⊑ Diploma, Chair ⊑ ResearchPosition} ℋ 2 = {Doctor ⊑ FundsProvider, GrantApplication ⊑ ResearchPaper}.
We argue that ℋ 1 is preferable over ℋ 2 , because ℋ 2 is less linked to the observation. Indeed, for any two concepts 𝐷 1 and 𝐷 2 such that 𝒯 a entails 𝐷 1 ⊑ ∃writes.𝐷 2 , a hypothesis {Doctor ⊑ 𝐷 1 , 𝐷 2 ⊑ ResearchPaper} could be created (consider for instance that 𝒯 a additionally entails Poet ⊑ ∃writes.Poetry), since neither 𝐷 1 nor 𝐷 2 need to be connected to either Professor or Researcher. Our new minimality notion discards ℋ 2 , and favors ℋ 1 as connection-minimal.

Intuitively, for an observation 𝐶 1 ⊑ 𝐶 2 , connection minimality only accepts those hypotheses which ensure that every CI in the hypothesis is connected to both 𝐶 1 and 𝐶 2 in 𝒯 . The definition of connection minimality is based on the following ideas:

1 Hypotheses for the abduction problem have to create a connection between 𝐶 1 and 𝐶 2 , in the form of a concept 𝐷 that satisfies

𝒯 ∪ ℋ |= 𝐶 1 ⊑ 𝐷, 𝐷 ⊑ 𝐶 2 .
2 To ensure that Occam's razor is followed, we want this connection to be based on concepts 𝐷 1 and 𝐷 2 for which we already have

𝒯 |= 𝐶 1 ⊑ 𝐷 1 , 𝐷 2 ⊑ 𝐶 2 .
3 We additionally want to make sure that the connecting concepts are not more complex than necessary, and that ℋ only contains CIs that directly connect parts of 𝐷 2 to parts of 𝐷 1 by closely following their structure.

We call 𝐷 a connecting concept: from which we construct the hypothesis ℋ 1 by linking the fillers of the existential restrictions.

A concept 𝐷 connects 𝐶 1 to 𝐶 2 in 𝒯 if
In the full paper, we define the connection between 𝐷 1 and 𝐷 2 formally using a relaxed notion of homomorphism between the description trees of 𝐷 2 and 𝐷 1 . We show that this notion of minimality is deeply connected with the generation of prime-implicates in first-order logic [START_REF] Marquis | Extending abduction from propositional to first-order logic[END_REF]. That is, using a translation scheme from abduction problems to first-order clauses, we are able to reconstruct the connection-minimal hypotheses using the prime implicates of the translation. In addition to soundness and completeness, we show a quadratic bound on the depth of the terms occurring in the prime implicates, which gives us a termination condition for our method, and ensures completeness for hypotheses that are both connection-minimal and subset-minimal. We implemented a prototype of our method consisting of two components: a Java component that takes care of preprocessing, translation into first-order logic, and construction of the hypotheses from prime implicates, and a first-order reasoning component that uses a modified version of the theorem prover SPASS [START_REF] Weidenbach | Topic, System description: Spass Version 3[END_REF] for the prime implicate generation. The prototype was evaluated on a set of ontologies from the medical domain for which we generated abduction problems in different ways, show-casing the practicality of our approach. The full paper has been accepted at IJCAR 2022 [START_REF] Haifani | Connection-minimal abduction in ℰℒ via translation to fol[END_REF].

  ∃employment.ResearchPosition ⊓ ∃qualification.Diploma ⊑ Researcher, ∃writes.ResearchPaper ⊑ Researcher, Doctor ⊑ ∃qualification.PhD, Professor ≡ Doctor ⊓ ∃employment.Chair, FundsProvider ⊑ ∃writes.GrantApplication }. The observation 𝛼 a = Professor ⊑ Researcher does not follow from 𝒯 a although it should. If we are only interested in subset minimal solutions, possible hypotheses to create the desired entailment include the following

  and only if 𝒯 |= 𝐶 1 ⊑ 𝐷 and 𝒯 |= 𝐷 ⊑ 𝐶 2 . Note that if 𝒯 |= 𝐶 1 ⊑ 𝐶 2 then both 𝐶 1 and 𝐶 2 are connecting concepts from 𝐶 1 to 𝐶 2 , and if 𝒯 ̸ |= 𝐶 1 ⊑ 𝐶 2 , it means no concept connects them. In the above example, we would use 𝒯 a |= 𝐶 1 ⊑ 𝐷 1 and 𝒯 a |= 𝐷 2 ⊑ 𝐶 2 where 𝐷

1 = ∃qualification.PhD ⊓ ∃employment.Chair 𝐷 2 = ∃qualification.Diploma ⊓ ∃employment.ResearchPosition ⊓ Researcher

Acknowledgments: This work was supported by the Deutsche Forschungsgemeinschaft (DFG), Grant 389792660 within TRR 248.