Hervé Pingaud

Keshav Dahal

Professeure Haiqing Zhang

Tewfik Ziadi

Tao Wang

Devant le jury composé de

Keywords: Lifestyle-related diseases, Prediction, Machine Learning, Missing values, Stacking ensemble

Contrat de diffusion

Ce document est diffusé sous le contrat Creative Commons « Paternité -pas d'utilisation commerciale -pas de modification » : vous êtes libre de le reproduire, de le distribuer et de le communiquer au public à condition d'en mentionner le nom de l'auteur et de ne pas le modifier, le transformer, l'adapter ni l'utiliser à des fins commerciales.

R ÉSUM É

Les pays industrialisés sont arrivés à la conclusion que de nombreuses maladies chroniques non transmissibles sont causées par des facteurs liés au mode de vie après avoir réalisé de nombreuses études épidémiologiques sur ces conditions, et peuvent être appelées maladies liées au mode de vie (MRD). L'obésité, l'hypertension artérielle, Le module de l'imputation de valeur manquante, le module de sélection des caractéristiques et le module de prédiction des maladies sont les trois principaux éléments de l'architecture que nous proposons pour la prédiction des LRD. Pour un grand nombre de valeurs manquantes, la méthode combinant la suppression et l'imputation est sélectionnée comme principale stratégie de traitement des valeurs manquantes. Étant donné que différentes maladies liées au mode de vie ont des caractéristiques différentes, le module de sélection de caractéristiques utilise une méthode basée sur l'apprentissage automatique pour trouver des caractéristiques clés. Enfin, v nous utilisons un scénario chinois pour expérimenter le cadre de prédiction suggéré.

Selon les résultats expérimentaux, le cadre de prédiction proposé peut également améliorer les performances d'évaluation des risques de LRD.

Mots clés: Maladies liées au mode de vie, Prédiction, Apprentissage automatique, Valeurs manquantes, Ensemble d'empilement.

vi vii always weighed on me. I seemed to be floating in an endless sea, but this did not extinguish my determination to continue chasing my dream. When I had the opportunity to apply to study in France, I submitted the application without hesitation.

It's a bit marvelous to say that my love for France originally came from a picture of a sea of lavender flowers in Provence, France that I saw when I was very young. I am grateful for the sheer desire that France brought to me and made me believe that I could make my dreams come true in this place.

If France was the fertile ground for my studies, then my director Abdelaziz BOURAS provided wings for my doctoral studies. I remember that in the first interview, the network was very poor, I was very nervous, and my performance was not satisfactory. But he always said to me, "it's fine", "it's good", "it's ok". I know he was encouraging me and guiding me into a better state. Later, his encouragement has been with me from the beginning to the end and has never stopped, helping me from the hesitation at the beginning to the firmness at the end. I am really grateful for his guidance and help during my doctoral period. I can always improve when I communicate with him. At the same time, he also taught me many skills besides scientific research, including the way of thinking about problems, the ability to logical analysis, and a healthy lifestyle.

Moreover, I would like to thank my three co-supervisors, Aïcha SEKHARI SEKLOULI, Tao WANG, and Haiqiang ZHANG. First of all, I would like to thank Prof. SEKHARI, for her in-depth sharing of French culture and language learning which enables me to enjoy fully In a foreign environment. At the same time, she viii can always provide me with novel research ideas and constantly promote the progress of my research. I also want to thank Prof. WANG for their endless help during these years. He has a wealth of professional knowledge and always provides me with great ideas when I encounter complex research problems. In addition, I am also very grateful to Prof. ZHANG, she is the enlightener of my doctoral study, and she not only provided me with the professional guidance in my study but also shared a lot of work and life experience with me.

I would like to thank Bilgesu BAYIR and Boubou Thiam NIANG, who are not only my colleagues but also my friends. We entered the lab at the same time.

They provided a lot of help when I first arrived in France and made me adapt to the study and life in France quickly. The friendships grow stronger with the days and it would be cherished for a lifetime. Thanks to all my friends in the laboratory, Jiao Zhao, Qing Li, Badreddine TANANE, and Mengji Yang, who have brought me enormous joy. I also thank the professors in the laboratory, Vincent CHEUTET, Yacine OUZROUT, Néjib MOALLA, for giving me a lot of help and support. I also want to thank Huiru Ren, for cooking me delicious Chinese food and traveling with me. Thanks to Jianan Xu for their help when I first come to Lyon. Thanks to Yuxin

Zong, who helped me a lot during these years. I am also very grateful to He Feng for his help in professional learning.

I want to thank my parents and my family for their unconditional love and support. Of course, I also want to thank myself for giving myself more opportunities to see the beautiful world. At last, I would like to thank the China Scholarship Council (CSC) for providing me with a scholarship to help me study in France smoothly.

ix

3.3

The bayesian optimization process for the ALWRF. . . . . . . . .

3.4

The distribution of accuracy in the ALWRF classification experiment 3.5 The AUC-ROC curve in the ALWRF classification experiment . .

3.6

The average of MSE in the ALWRF regression experiment . . . .

3.7

The distribution of R 2 in the ALWRF regression experiment . . .

3.10

The experiment results of for the total missing rates. . . . . . . .

3.11

The experiment results of three classifiers vales. . . . . . . . . . .

3.12

The missing matrix of Hepatitis Dataset . . . . . . . . . . . . . .

3.13

The missing matrix of Cleveland Dataset . . . . . . . . . . . . . .

3.14

The missing matrix of Primary Tumor Dataset . . . . . . . . . . .

3.15

The missing matrix of Chronic Kidney Dataset . . . . . . . . . .

3.16

The missing matrix of Thyroid Dataset . . . . . . . . . . . . . . .

3.17

The missing matrix of Framingham Dataset . . . . . . . . . . . .

3.18

The experiment results of datasets with real missing values. . . . Traditional medical services used to be kept and recorded on paper [1,2],

which is difficult to serve people effectively and easily as society has developed [3]. The medical industry has seen revolutionary changes as a result of the digitization of medical information [START_REF] Kanti | Internet of things, smart sensors, and pervasive systems: Enabling connected and pervasive healthcare[END_REF]. Through the systematization, standardization, and intelligence of big data, the digital medical system effectively integrates various patient information data, offers intelligent services for patients, and intelligent management based on electronic files for hospitals [START_REF] Ahmed | Telemedicine takes centre stage during covid-19 pandemic[END_REF]. The construction of a digital hospital management system is essential in order to improve the operational efficiency of modern hospitals. The database of the hospital information system includes a variety of medical data, including administrative data, laboratory data, treatment data, and prescription data [START_REF] Yang | A multicriteria framework for addressing digitalization solutions of medical system under interval-valued t-spherical fuzzy information[END_REF]. The amount of data keeps growing over time, and the gathered knowledge about medical practices can serve as a guide for the conduct of the clinical medical staff as well as a wealth of useful information for hospital administrators [START_REF] Asmat | Major applications of data mining in medical[END_REF]. Additionally, examining and mining this beneficial data can yield important references for making medical decisions [START_REF] Zhang | A machine learning framework for identifying influenza pneumonia from bacterial pneumonia for medical decision making[END_REF].

However, the exponential rise of medical data as a result of the quick development of medical information technology has made its hidden value an urgently needed treasure. Especially, with the improvement of people's living standards and health awareness, more and more health check data are collected. For example, the "China Health Statistical Yearbook" showed that 444 million health examinations were performed in China in 2019 compared to 406 million in 2017.

These health examinations produce enormous amounts of medical data with hidden value. In order to offer people intelligent and individualized medical services, it is urgently necessary to mine the valuable information concealed in massive amounts of medical data [START_REF] José A Castellanos-Garzón | An evolutionary framework for machine learning applied to medical data[END_REF]. In particular, the use of information-based methods to screen data allows administrators and healthcare professionals to thoroughly research patient medical histories and deliver more effective care [START_REF] Azmi | A systematic review on machine learning approaches for cardiovascular disease prediction using medical big data[END_REF]. Accurate and individualized health care services can be provided by utilizing big data analysis techniques in the medical and health fields, as well as data mining and analysis technology to examine medical data [START_REF] Yang | A multicriteria framework for addressing digitalization solutions of medical system under interval-valued t-spherical fuzzy information[END_REF]. In this context, it is crucial to employ big data and artificial intelligence to discover valuable information hidden in massive data sets held in medical information systems and to equip local hospitals with smart medical systems to boost the effectiveness of healthcare services.

Research Significance

As we know, there is a global lack of medical resources, including general practitioners and medical supplies. For example, only 800 doctors were practicing medicine in the French department of Seine-et-Marne as of December 31, 2020, or less than 6 doctors for every 10,000 people [START_REF] Dalmat | Brève : La seine-et-marne, un désert médical ?[END_REF]. Therefore, more and more researchers use information technology to assist doctors in their work to improve service efficiency. For example, Mohamed Elhoseny et al. [START_REF] Elhoseny | Intelligent diagnostic prediction and classification system for chronic kidney disease[END_REF] proposed a classification system of chronic kidney disease to help doctors distinguish different groups and achieved a prediction accuracy of 95%. Although it is difficult for these methods to predict all cases perfectly, they can be used as additional tools to provide information to doctors. On the other hand, some studies focus on preventing or delaying the progression of the disease. For example, Shuqiong However, disease prevention approaches have some limitations, and they are more suitable for diseases where risk factors are readily available and disease progression is improvable. Lifestyle-related diseases (LRDs) have natural advantages to building disease risk prediction models. LRDs refer to diseases whose psychophysiology is significantly affected by lifestyle factors, and changes in these etiological factors can significantly improve disease prevention and treatment [START_REF] Sagner | Lifestyle medicine potential for reversing a world of chronic disease epidemics: from cell to community[END_REF][START_REF] Yeh | The advent of lifestyle medicine[END_REF]. From the definition of LRDs, they are extremely related to people's lifestyles or behaviors, their risk factors are easily obtained, and many studies [START_REF] Earl S Ford | Healthy living is the best revenge: findings from the european prospective investigation into cancer and nutrition-potsdam study[END_REF][START_REF] Sanchez | Effectiveness of physical activity promotion interventions in primary care: a review of reviews[END_REF] have shown that LRDs can be improved by healthy lifestyles.

On the other hand, as countries become more industrialized and wealthier, the prevalence of LRDs increases due to changes in people's behavior. Generally, most chronic diseases, including cardiovascular disease, metabolic syndrome, obesity, type 2 diabetes, and some cancers, are lifestyle-related diseases and closely related to people's lifestyles [START_REF] Yeh | The advent of lifestyle medicine[END_REF]. Studies have found that lifestyle-related diseases are the absolute and relative most common diseases in the world today, and the death toll exceeds that of AIDS, malaria, and tuberculosis combined [START_REF] Abdallah S Daar | Grand challenges in chronic non-communicable diseases[END_REF]. Cardiovascular disease, obesity, type 2 diabetes, hypertension, and some particular malignancies have all grown to be significant problems in the twenty-first century.

In the Republic of Ireland, 61% of adults are overweight or obese, and over 40% of adults report having at least one lifestyle-related disease, the most prevalent of which is high blood pressure and high cholesterol [START_REF] O'donoghue | Assessment and management of risk factors for the prevention of lifestyle-related disease: a cross-sectional survey of current activities, barriers and perceived training needs of primary care physiotherapists in the republic of ireland[END_REF]. Additionally, 17.8 million individuals globally passed away from cardiovascular disease (CVD) in 2017, according to the Global Burden of Disease report published in 2018 and the estimated overall number of tumor-related fatalities (mostly cancer) is 9.56 million [START_REF] Wilds | Global prevalence of diabetes: estimates for the year 2000 and projections for 2030[END_REF]. The WHO predicts that by 2030, there will be 366 million individuals worldwide who have diabetes, up from the present estimate of 175 million [START_REF] Gregory A Roth | Global, regional, and national agesex-specific mortality for 282 causes of death in 195 countries and territories, 1980-2017: a systematic analysis for the global burden of disease study 2017[END_REF]. Despite the availability of a wide range of medicines, the frequency of lifestyle illnesses is not controlled due to the safety concerns connected with these medicines [START_REF] Kakkar | The rising status of edible seeds in lifestyle related diseases: A review[END_REF].

To sum up, there is a crisis in the global healthcare system as a result of the prevalence of these lifestyle-related disorders.

Smoking, poor diet, excessive alcohol use, and a sedentary lifestyle are all clear contributors to various lifestyles related diseases [START_REF] Alwan | Global status report on noncommunicable diseases 2010[END_REF][START_REF] Morgan | alcohol use in ireland: a profile of drinking patterns and alcohol-related harm from slán[END_REF]. According to research, even tiny adjustments to one's behavior can have a significant impact.

Ford et al. [START_REF] Earl S Ford | Healthy living is the best revenge: findings from the european prospective investigation into cancer and nutrition-potsdam study[END_REF] found that those who did not smoke, had a body mass index of less than 30 kg/m 2 , engaged in 3.5 hours of physical activity per week, and consumed a nutritious diet had a 78% decreased risk of getting a chronic illness throughout the course of the 8-year trial. The risks of myocardial infarction, stroke, cancer, and type 2 diabetes all decreased by 93%, 81%, 50%, and 36% respectively. A change in physical activity level alone would result in an increase in life expectancy of between 2.8 and 7.8 years for men and between 4.6 and 7.3 years for women, depending on the degree of the increase in activity, according to actual disease and death rates of physically active and inactive people in Denmark aged 30 to 80 years [START_REF] Sanchez | Effectiveness of physical activity promotion interventions in primary care: a review of reviews[END_REF].

Despite this convincing evidence, neither general medical treatment nor modern physiotherapy practice is dominated by lifestyle-related diseases or methods for avoiding, reversing, and managing them [START_REF] Britt | General practice series no 19 aihw cat no gep 19. canberra: Australian institute of health and welfare[END_REF]. The idea of health is drastically altering in response to these modern health trends and goals [START_REF] O'donoghue | Assessment and management of risk factors for the prevention of lifestyle-related disease: a cross-sectional survey of current activities, barriers and perceived training needs of primary care physiotherapists in the republic of ireland[END_REF]. The focus of healthcare is shifting from disease models to health models on a global scale. Contrarily, lifestyle-related diseases are multi-factorial illnesses that are influenced by both environmental and genetic variables and are brought on by the interaction of numerous risk factors [START_REF] Sagner | Lifestyle medicine potential for reversing a world of chronic disease epidemics: from cell to community[END_REF]. These illnesses have sneaky onsets, a protracted incubation period, and a quick progression. Identifying and treating large numbers of patients in a timely manner is challenging. Additionally, as the majority of lifestyle-related diseases still have unclear etiologies and pathogens and poor therapeutic outcomes, it is important from a practical standpoint to prevent the development of lifestyle-related diseases.

In terms of the characteristics of lifestyle-related diseases and contemporary health trends, early disease prediction has significant research ramifications.

It is one of the key steps in preventing and treating diseases that are caused by a person's lifestyle because identifying population risks prior to the onset of diseases can help people change their lifestyles as soon as possible, especially the life behaviors of high-risk groups, lowering the risk of disease [START_REF] Shahadat Uddin | Comparing different supervised machine learning algorithms for disease prediction[END_REF]. The primary tool for assessing and preventing lifestyle-related diseases is the disease prediction model [START_REF] Hou | Models for predicting risk of dementia: a systematic review[END_REF]. Disease prediction models specifically establish an intelligent model to predict the probability of a specific disease at a specific point in the future, classify high-risk groups in accordance with the probability cut-off point, and conducts behavior, diet, and other interventions to prevent future disease. It can fall under the heading of illness prevention. In other words, the disease prediction model may show assessment subjects about the likelihood that they will become ill in the future and anticipate this likelihood, as well as advise them on how to manage their own health.

Research Status of LRDs Prediction

The original disease prediction model is a disease prediction model of coronary heart disease, which was established by the United States based on the Framingham cohort study [START_REF] Truett | A multivariate analysis of the risk of coronary heart disease in framingham[END_REF], and other cardiovascular disease risk assessment models with various markers [START_REF] Thomas | Multiple biomarkers for the prediction of first major cardiovascular events and death[END_REF][START_REF] Paul M Ridker | Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the reynolds risk score[END_REF]. The disease prediction models have grad-ually expanded from cardiovascular disease to include a variety of diseases. For instance, the United States has developed a model for predicting stroke based on the Framing cohort [START_REF] Philip A Wolf | Probability of stroke: a risk profile from the framingham study[END_REF]. Tool [START_REF] Colditz | Harvard report on cancer prevention volume 4: Harvard cancer risk index. risk index working group, harvard center for cancer prevention[END_REF], the breast cancer disease prediction-Gail model [START_REF] Mitchell H Gail | Projecting individualized probabilities of developing breast cancer for white females who are being examined annually[END_REF], and a prediction model for lung cancer proposed by the Cancer Research Center of University of Texas Anderson [START_REF] Margaret R Spitz | A risk model for prediction of lung cancer[END_REF].

Machine learning (ML) techniques, a subset of artificial intelligence techniques, employ computer systems to predict diseases using statistical models and algorithms, opening up a wide range of opportunities for illness prevention [START_REF] Shahadat Uddin | Comparing different supervised machine learning algorithms for disease prediction[END_REF].

Researchers have utilized a number of ML algorithms to predict various diseases in the field of disease prediction. For instance, the use of ensemble techniques for the early diagnosis of coronary heart disease [START_REF] Yu | Application of support vector machine modeling for prediction of common diseases: the case of diabetes and pre-diabetes[END_REF]; the use of support vector machines to detect pre-diabetes and diabetes [START_REF] Shorewala | Early detection of coronary heart disease using ensemble techniques[END_REF]; the use of random forest algorithms to predict the risk of diabetes in the population examined physically [START_REF] Zhanlin | Predictive value of random forest algorithms for diabetic risk in people underwent physical examination[END_REF];

To predict hypertension, a combination of sub type (the least absolute shrinkage and selection operator, LASSO) and support vector machine recursive feature elimination (SVMRFE) was used [START_REF] Islam | Machine learning algorithm for characterizing risks of hypertension, at an early stage in bangladesh[END_REF]. A new ensemble learning-based framework for the early detection of type 2 diabetes utilizing lifestyle markers was also developed [START_REF] Mohammad | An ensemble machine learning approach for predicting type-ii diabetes mellitus based on lifestyle indicators[END_REF].

Our values. On further analysis, the quantitative relationship between models and diseases in the studied papers was as follows.

• One to one: almost all studied papers only focused on predicting a single disease.

• one-to-many: three studied papers used the same model and different datasets to predict multiple diseases.

• many-to-many: only one studied paper employed different models to predict different diseases in different datasets.

Based on the above analysis, existing studies are unable to intelligently identify key features of diseases while building prediction models with different structures and robustness for different LRDs. Therefore, our objective is to design an intelligent risk prediction framework for LRDs that can smartly identify key features of different LRDs for dirty real medical data, accurately predict the risk of LRDs and visualize prediction results.

Organization of Thesis

The present thesis is organized in 6 chapters as shown in Figure 1.1. Following the introduction in Chapter 1, the rest of the thesis chapters are as follows:

Research Status of Related Technologies

Based on the analysis in Chapter 1 and the studied data characteristics, three research issues need to be considered in the proposed risk prediction framework for LRDs:

1) Most common prediction techniques are challenging for people to use in accordance with standard processes because medical data that have been collected are dirty and contain a lot of missing values.

2) Effective and precise risk factor identification is essential because removing redundant variables can decrease model complexity and makes it easier to analyze and comprehend model predictions.

3) Since data noise may lower the model's convergence rate and accuracy, it is crucial to research robust models. Enhancing model robustness can reduce sensitivity to noisy data, make models more accurate and offer more reliable auxiliary services.

In conclusion, the proposed risk prediction framework for LRDs must take into account the mentioned three issues: 1) analysis and processing of missing values; 2) identification of key features; and 3) accurate disease prediction. To specifically handle these three issues, the proposed framework must take into account three important techniques.

Research Status of Missing Value Processing Methods

As analyzed above, with the construction of modern health information systems, healthcare organizations are experiencing explosive growth in medical data. These medical data contain an abundance of hidden but potentially valuable information, i.e., unknown correlations between diseases and features, and links between diseases with their complications [START_REF] Shafenoor Amin | Identification of significant features and data mining techniques in predicting heart disease[END_REF]. Such information is useful for medical diagnosis, therapy, and decision-making [START_REF] Xu | A cluster-based oversampling algorithm combining smote and k-means for imbalanced medical data[END_REF]. However, some unavoidable reasons, such as the early withdrawal of participants from medical research studies and the refusal of participants to attend certain items in medical examinations, can easily result in missing values in research data [START_REF] Suthar | A Survey: Classification of Imputation Methods in Data Mining[END_REF][START_REF] Houari | Handling missing data problems with sampling methods[END_REF][START_REF] Emmanuel | A Survey On Missing Data in Machine Learning[END_REF]. Since the existence of missing values makes it more challenging for people to mine relevant information, many methods for dealing with missing values have been proposed, which can be mainly divided into three categories, namely, deleting missing values, tolerating missing values, and imputing missing values.

Missing value deletion, also known as disregarding missing values, is the process of explicitly deleting instances or variables that contain missing data items to solve the problem of missing data [START_REF] Emmanuel | A Survey On Missing Data in Machine Learning[END_REF]. Although a test pattern with missing values cannot be classified since the deletion procedure would ignore it, deletion methods have the advantage of allowing the normal pattern classification methods to be used directly for complete data [START_REF] Jadhav | Comparison of Performance of Data Imputation Methods for Numeric Dataset[END_REF]. For ignoring missing data, there are two general strategies [START_REF] Roderick | Statistical analysis with missing data[END_REF][START_REF] Jadhav | Comparison of Performance of Data Imputation Methods for Numeric Dataset[END_REF][START_REF] Emmanuel | A Survey On Missing Data in Machine Learning[END_REF]. First, Listwise Deletion (LD), also known as casewise deletion, or case removal, is a technique for removing instances (rows, cases) with missing data. This technique is also known as complete case analysis because it only keeps complete cases for analysis (CCA). The analysis is then restricted to those observations for which all values are observed, which frequently leads to biased estimates and loss of precision [START_REF] O'donoghue | Assessment and management of risk factors for the prevention of lifestyle-related disease: a cross-sectional survey of current activities, barriers and perceived training needs of primary care physiotherapists in the republic of ireland[END_REF] because this method excludes all cases with missing values for any variable of interest. The second technique is known as Pairwise Deletion (PD) or Available Case Analysis (ACA), also referred to as variable deletion, and it is used to delete variables (columns) with missing data [START_REF] Joseph | Missing data: Our view of the state of the art[END_REF]. This method analyzes all situations in which the variables of interest are present, using as much data from each case as is feasible rather than excluding the entire case. Even though some of its variables have missing values, it can nevertheless maintain the most amount of data possible for analysis since it uses distinct sample sizes for each variable [START_REF] Joseph | Missing data: Our view of the state of the art[END_REF]. As a result, the ACA approach has a larger sample size than the CCA method.

In the second type of missing value processing approach, the model is built with some strategies to tolerate missing values. For instance, XGBoost, Light-GBM, and Catboost ensemble tree models and decision trees both process missing data during training. These models specifically attempt, during the decision tree construction process, to allocate samples with missing values in the features selected as split points to the left sub-tree or the right sub-tree, and then analyze which side will reduce the loss. This method preserves all data while also assisting in the discovery of hidden information in missing data. Nevertheless, these techniques only work with certain model architectures, which makes the model more complex.

In the third type of missing value processing method, the value estimated by the model is used to replace the missing value. Early approaches for imputing missing data were specifically motivated by traditional statistical models and estimate processes, which are referred to as imputation methods based on statistics. These techniques are designed to model the information included in the non-missing parts of the data set in order to as correctly estimate the missing values as possible [START_REF] Aittokallio | Dealing with missing values in large-scale studies: microarray data imputation and beyond[END_REF]. Researchers initially substituted missing values with the mean, median, mode, and zero values. The disadvantage is that when there are numerous missing data, a significant portion of the data is replaced by the same value (i.e., mean, median, mode, zero), which can easily lead to serious deviation.

The mean imputation approach should not be used, according to certain recent research that has demonstrated its shortcomings [START_REF] Van Hulse | A comprehensive empirical evaluation of missing value imputation in noisy software measurement data[END_REF][START_REF] Eekhout | Missing data in a multi-item instrument were best handled by multiple imputation at the item score level[END_REF]. The in-depth study on missing values has been accompanied by the proposal of a number of innovative techniques. For instance, the Least Squares (LS) imputation approach is based on the least squares principle to estimate missing values, whereas the hot-deck imputation method predicts missing values by seeking for the nearest neighbor using non-missing information [START_REF] Hellem Bø | Lsimpute: accurate estimation of missing values in microarray data with least squares methods[END_REF].

Further, the researchers used machine learning models to impute missing values. Machine learning-based imputation approaches are complex processes that often include building a predictive model to estimate values that will substitute those missing [START_REF] García-Laencina | Pattern classification with missing data: A review[END_REF]. The machine learning-based imputation method often involves building a predictive model to predict the values for missing data. Many machine learning-based imputation methods have been proposed recently, and these methods frequently produce good imputation results. Examples of these methods include imputation methods based on decision trees (DT) [START_REF] Rockel | Decision Trees for the Imputation of Categorical Data[END_REF][START_REF] Du | A data mining method for structure design with uncertainty in design variables[END_REF],

imputation using multilayer perceptrons [START_REF] Kancherla | Soft computing based imputation and hybrid data and text mining: The case of predicting the severity of phishing alerts[END_REF], imputation using artificial neural networks (ANNs) [START_REF] Fallah | Application of a multi-stage neural network approach for time-series landfill gas modeling with missing data imputation[END_REF], and imputation using self-organizing maps (SOMs) [START_REF] Vatanen | Self-organization and missing values in som and gtm[END_REF].

The three missing value processing methods and their advantages and disadvantages are shown in Table 2 The three missing value processing methods and their advantages and disadvantages.

Research Status of Feature Selection Methods

As it can be challenging for people to distinguish between significant and superfluous features when gathering data, feature selection is an essential component of data reprocessing. Specifically, feature selection refers to choosing a task-related feature subset from the full set of features in order to reduce the amount of data that must be stored, shorten the time needed to train machine learning models, and enhance the predictive skills of machine learning models.

Therefore, feature selection can assist in both the identification of essential features and the elimination of superfluous features. Data mining techniques based on machine learning techniques were used to select the primary characteristics of lifestyle-related diseases. The benefit of this approach is that the outcomes are generated by data analysis without the need for human interaction. This approach is appropriate for those without strong expertise in medicine and uses sophisticated algorithms to guide people in choosing essential factors. Our research belongs to the category of supervised learning because it focuses on the prediction of LRDs disease. We, therefore, concentrate on feature selection for supervised issues in this study. Three categories of feature selection techniques can be distinguished based on the form of the feature selection [START_REF] Khaled | Prediction of chronic kidney disease using different classification algorithms[END_REF]:

• Filter: Determine thresholds or the maximum number of features to be selected, and then rank each feature according to specific statistical indicators.

• Wrapper: When choosing alternative feature subsets for the model's training, consider the impact of cross-validation as the optimization objective.

Then, choose the best combination.

• Embedded: After the model has been trained, many machine learning models allow for the evaluation of the contribution of each feature to the prediction result. The threshold, or the number of thresholds to be selected, can then be set in accordance with the contribution, and the feature can be chosen.

Three feature selection categories and their advantages and disadvantages are shown in Table 2

Technical Challenges

As we have already mentioned, as living standards have increased, people's concern for their personal health has increased. To lower risks or postpone the development of contracting lifestyle-related diseases, people have chosen a variety of strategies, including health screenings, diet, and exercise. Over time, a large amount of health and medical information is recorded and stored in detail by the medical information system. The foundation for research on lifestyle-related diseases has been set in this situation by enough health examination data and some lifestyle-related data. Researchers are now concentrating on applying machine learning techniques to mine valuable information hidden in health test data to assist people in predicting and preventing diseases connected to lifestyle choices.

But when data mining is used, two key aspects in gathered medical examination data-missing values and noise-present technical difficulties for the analysis and prediction of lifestyle-related diseases. Next, we will provide an in-depth analysis and introduction of missing values and noise in the dataset of lifestyle-related diseases.

Missing Values in Imbalanced and Mixed-type Features

A simple and easy-to-operate missing value processing technique is missing value deletion, but this technique is prone to losing valuable information and is unable to be utilized with data that has a lot of missing values. Furthermore, some predictive models develop techniques to deal with missing values, which can help preserve more useful information but makes the predictive model more complex and only works with specific model structures. The missing value imputation method can keep more valuable information, is more flexible, and is not dependent on the prediction model.

Numerous methods are available in the literature to impute missing values in metrically scaled data, such as imputation by mean, hot-deck [START_REF] Donald B Rubin | Multiple imputation for nonresponse in surveys[END_REF], k-Nearest Neighbors (kNN) [START_REF] Troyanskaya | Missing value estimation methods for dna microarrays[END_REF], Decision Tree (DT) [START_REF] Xia | A boosted decision tree approach using bayesian hyper-parameter optimization for credit scoring[END_REF] and Random Forest (RF) [START_REF] Daniel | Missforest-non-parametric missing value imputation for mixed-type data[END_REF].

The two types of mean imputation are conditional and unconditional mean imputation, both of which are quick but may destroy the data distribution [START_REF] Eekhout | Missing data in a multi-item instrument were best handled by multiple imputation at the item score level[END_REF]. The kNN technique finds the k-nearest records to fill in missing values. The kNN strategy has the advantage of simplicity, but it requires searching the entire dataset to locate the k-nearest neighbors. In addition, as kNN ignores the correlation between covariates, Shahla and Gerhard [START_REF] Faisal | Imputation methods for high-dimensional mixed-type datasets by nearest neighbors[END_REF] proposed a sophisticated imputation method for mixed-type data that uses non-parametric nearest-neighbor and takes into account the correlation between covariates. Although it yields smaller imputation errors and higher performance in datasets with significant covariate correlation, it easily encounters disaster in time and space in large-scale datasets since it needs to multiple search datasets and calculates distances between records.

Further, researchers prefer tree-based imputation methods like the decision tree and random forest model because of their high interpretability, quick prediction speed, and adaptability for mixed-type datasets. For example, Rahman and Islam [START_REF] Md | Missing value imputation using decision trees and decision forests by splitting and merging records: Two novel techniques[END_REF] employed decision trees and decision forests to impute missing values by dividing and merging records and achieved outperformed results on nine public datasets. Even though they used tree-based approaches to impute missing data, their methods are computationally complicated and demand a lot of memory when merging records from many trees with various structures. In another tree-based example, Nikfalazar and Yeh et al. [START_REF] Nikfalazar | Missing data imputation using decision trees and fuzzy clustering with iterative learning[END_REF] introduced a new missing value imputation approach that considers mixed-type data by combining decision trees and fuzzy C-means (FCM) [START_REF] James C Bezdek | Fcm: The fuzzy cmeans clustering algorithm[END_REF] with iterative learning. But single decision tree is susceptible to noise, and it is time-consuming to search for the number of clusters and perform clustering.

In the medical field, large-scale datasets with mixed type and imbalance characteristics are widespread [START_REF] Cabeli | Learning clinical networks from medical records based on information estimates in mixed-type data[END_REF][START_REF] Li | A learning method for the class imbalance problem with medical data sets[END_REF] which can adjust the weight of features adaptively when building a random forest and improve prediction accuracy for imbalance features. As far as we know, our work is the first imputation method to consider both adaptive weights and imbalanced problems based on a tree model. We will give a detailed introduction to the proposed missing value imputation method in Chapter 3.

Diverse Noises in Lifestyle Related Disease Context

Data noise is the term for errors or unusual data present in the data. The processing and analysis of data sets can be significantly impacted by these data noises. To discover a suitable approach to deal with data noise, it is required to identify the different types of noise in the data. Two categories of noise-attribute noise and class noise-are generally separated in terms of disease prediction [START_REF] Zhu | Class noise vs. attribute noise: A quantitative study[END_REF].

Class noise happens when examples are incorrectly classified into a class, and attribute noise influences the attribute values of examples in the dataset. Both attribute noise and noise-like noise can affect the classifier's performance [START_REF] José | Analyzing the presence of noise in multi-class problems: alleviating its influence with the one-vs-one decomposition[END_REF].In medical data classification, noise can come from multiple sources:

• Human error. Errors in the labeling process, which are more likely to occur in jobs dealing with complex data, can occur due to fatigue, routine, checking each case quickly, or time pressure. In addition, subjectivity also creates category noise. For example, when there are discrepancies in the labels of multiple experts.

• Machine errors. When machines are responsible for providing automated data, design errors or transient errors can result in incorrect attributes and labels.

• Digitization and filing errors. When creating digital records of inspection cases, categories can be entered incorrectly due to simple mistakes. The same happens when using history.

In particular, noise is a combination of attribute noise and class noise in the medical data of lifestyle-related diseases, where attribute noise is mostly made up of abnormal attribute values, or outliers, which are distinct individual points from the system as a whole. However, these points cannot be ignored because they might potentially have useful information. For instance, a study [START_REF] Gollapalli | A novel stacking ensemble for detecting three types of diabetes mellitus using a saudi arabian dataset: prediabetes, t1dm, and t2dm[END_REF] on diabetes discovered that outliers were indicated by significant disparities between the maximum value of two key characteristics, triglycerides (TG) and low-density lipoprotein (LDL), and the third quartile. Because they belonged to valid patients, these outliers were not excluded. On the other hand, class noise is known as wrong instance labels. In practice, the diagnosis of lifestylerelated diseases is prone to mislabeling. For example, if you speak during blood pressure measurement, blood pressure will increase by 5-19 mmHg; or when blood pressure is measured in a cold environment, blood pressure may increase by 5-23 mmHg; these situations can affect blood pressure measurement and result in a misdiagnosis. Furthermore, it is impossible for medical professionals to ensure the utmost accuracy of their diagnosis results when making medical diagnoses in the face of complicated lifestyle-related diseases, such as coronary heart disease and tumors.

According to the above analysis, medical data of lifestyle-related diseases inevitably have attribute noises and label noises. In general, there have been two

basic strategies for dealing with noisy data in medical data:

• Algorithmic-level methods. These techniques are characterized by being less affected by noisy data. For example, C4.5 [START_REF] Ross | C4. 5: programs for machine learning[END_REF] uses a pruning strategy to reduce the chance of the tree overfitting due to noise [START_REF] Quinlan | Induction of decision trees[END_REF].

• Data-level approach. The most well-known type of method in this group is the noise filter [START_REF] Taghi | Improving software quality prediction by noise filtering techniques[END_REF]. They identify noisy examples, which can be eliminated from the training data.

Using data-level methods to directly delete outliers is easy to lose effective information because there are some in the dataset of lifestyle-related diseases examined that correspond to valid patients. The ensemble approach, an algorithm-level technique, is a great way to lower variation, bias, and noise, and it can combine several individual models as a whole to outperform each individual model. Therefore, in order to improve the accuracy of lifestyle-related disease prediction, we employed an ensemble method in our research to develop a robust disease prediction model, which makes the model less sensitive to noise and improves the prediction accuracy of LRDs. In Chapter 4, a comprehensive introduction to the proposed ensemble method is presented.

The Overview of Prediction Framework

A framework for LRDs prediction is proposed based on the above find-

Missing Value Module

In the missing value module, in order to enable the comprehensive analysis of missing values, missing rates, and missing patterns are visually presented.

Firstly, The missing rate analysis help to rapidly comprehend the missing conditions in the data set. Meanwhile, it can also use this information to help choose the processing strategy for missing values. For clarity of definitions, we assume that data set X includes n instances and k features. Let M represent a missing value matrix, where m ij has a value of 0 if any value

x ij (i ≤ n, j ≤ k) in X is
observed and 1 otherwise. The total missing rate M R can be represented as

M R = n i=1 k j=1 m ij m × n (2.1)
On the other hand, the missing rate of ith row (denoted by r i ) can be calculated by

M R r i = k j=1 m ij m (2.2)
Finally, the missing rate of jth column (denoted by c j ) can be computed by

M R c j = n i=1 m ij m (2.3)
Second, by displaying the distribution of missing values, such as univariate, monotone, and non-monotone [START_REF] Emmanuel | A Survey On Missing Data in Machine Learning[END_REF], the study of missing patterns can assess how complex missing values are. Finally, by examining the relationships between different features with missing values, such as MCAR, MAR, and NMAR [START_REF] Roderick | Statistical analysis with missing data[END_REF], the study of the missing mechanism can investigate the causes of missing values.

In reality, some features or instances will have a disproportionate number of missing values for a variety of reasons; for instance, 99% of the values will be absent. The major features of lifestyle-related diseases are used in our study to build excellent predictive models, so when features or instances have a large number of missing values, this is difficult to apply in our study. Instead, we will prefer to use the deletion method rather than filling in a large number of estimates. We need to describe the criteria for deleting missing values, or the threshold for using it, in more detail. According to the 80% rule [START_REF] Bijlsma | Largescale human metabolomics studies: a strategy for data (pre-) processing and validation[END_REF], which states that a substance should be removed if its non-missing portion is less than 80% of the sample size as a whole, the suggested prediction framework excludes features or instances whose missing rate is more than 80%. At the same time, the framework provides an interface for customizing the threshold, making it simple for knowledgeable specialists to adjust the threshold based on their own expertise.

There are still some missing values in the dataset even though some features and instances are compelled to be removed in accordance with the threshold setting of the missing rate. The reasons and ways of missing are typically dispersed among several features and instances, making it difficult to simply eliminate them using a deletion procedure. Therefore, to appropriately handle missing values, we shall employ more sophisticated techniques. We suggest a missing value imputation technique in Chapter 3 that can be used with datasets that are imbalanced or mixed types. We employed the proposed missing value imputation approach as our default missing value handling method in the missing value imputation step since features with characteristics of unbalanced and mixed types are common in datasets of lifestyle-related diseases. Similarly, we incorporate various well-known and excellent imputation methods for missing values, such as MissForest and KNNI, as alternatives or benchmarks in order to provide people with more options.

Feature Selection Module

Data mining techniques based on machine learning techniques are employed to select the primary characteristics of lifestyle-related diseases. The benefit of this approach is that the outcomes are generated by data analysis without the need for human interaction. This approach is appropriate for those without strong expertise in medicine and uses sophisticated algorithms to guide people in choosing essential factors. In previous studies, we surveyed existing feature selection methods, and each method has its own advantages and disadvantages.

The feature selection of the wrapper has high computation complexity, and the filtering mechanism ignores the connection between the feature and the target variable. As a result, the tree-based strategy in the embedding method is employed for feature selection in the proposed prediction framework. Splitting into tree-based approaches occurs in the classification model due to Gini impurity or information gain/entropy, whereas it occurs in the regression model due to variance. Using techniques like random forests and gradient boosting, features are chosen according to the relevance of each one. Generally, features with high im-portance are more likely to have an impact on the target feature. The proposed prediction framework uses the random forest importance approach as the main algorithm of the feature selection module because the random forest has high generalization capabilities and is appropriate for large-scale datasets.

Specifically, the random forest feature importance evaluation calculates the mean value of each feature's contribution to each tree in the random forest. There are two techniques to obtain the final collection of key features after assessing the importance of each feature: 1) select Top-N features, 2) Select larger than the set threshold. Since the value of N is difficult to determine and in order to keep as many task-related features as possible, the feature selection module selects according to the important threshold of the feature.

Disease Prediction Module

As we previously mentioned, a variety of machine learning algorithms have been utilized by researchers to estimate the risk of various diseases in the field of disease risk prediction. There are various noises in the data set, which threaten the accuracy of the disease prediction model. Therefore, in order to build a robust prediction model for LRDs, we will employ ensemble techniques to reduce the impact of noise. We propose a stacked ensemble method in Chapter 4, a technique that can be used on datasets with diverse noise. We adopted the proposed stacked ensemble method as the default prediction method for the disease prediction On mixed-type data, tree-based models have a natural advantage because their construction is concentrated only on the information gain of features rather than the distance between cases [START_REF] Emmanuel | A Survey On Missing Data in Machine Learning[END_REF]. On the other hand, tree-based models show high interpretability compared to algorithms such as neural networks, because their routes from the root node to the leaf node represent a rule [START_REF] Zhu | Rapid identification of highquality marine shale gas reservoirs based on the oversampling method and random forest algorithm[END_REF]. A decision tree is one of the most representative tree-based models. The decision tree starts from the root node of the tree, continually splits by selecting the optimal attribute, and builds the tree nodes one by one until a stopping condition of tree building is satisfied. There are two typical stopping conditions, including no samples in the child nodes and exhaustion of attributes. As a single decision tree frequently suffers from overfitting, ensemble approaches based on decision trees have been proposed including Boosting [START_REF] Freund | A decision-theoretic generalization of on-line learning and an application to boosting[END_REF] and Bagging [START_REF] Breiman | Bagging predictors[END_REF]. Random forest is an ensemble algorithm based on the bagging approach that has strong anti-noise properties and can perform effectively on large data sets [START_REF] Xia | A boosted decision tree approach using bayesian hyper-parameter optimization for credit scoring[END_REF][START_REF] Feng | Imputation of missing well log data by random forest and its uncertainty analysis[END_REF].

Meanwhile, as it blends the idea of the ensemble with randomization, overfitting is well-controlled. In particular, random forest uses the bootstrap technique to randomly draw samples from original samples to build a single decision tree, and then repeat this process a specific number of times (the number of trees) [START_REF] Zhu | Rapid identification of highquality marine shale gas reservoirs based on the oversampling method and random forest algorithm[END_REF].

Finally, the final prediction result is obtained by combining these decision trees.

In random forests, features with high quality are not fully used because features are selected consistently and randomly to construct a feature subspace. As a result, the random forest's performance may be limited, because all features, including those with little or no information, have the same probability [START_REF] Guyon | An introduction to variable and feature selection[END_REF].

From the standpoint of feature subspace selection, some better random forest methods have been developed. Amaratunga and Cabrera et al. [START_REF] Amaratunga | Enriched random forests[END_REF] proposed enriched random forests: choose the eligible subsets at each node by weighted random sampling instead of simple random sampling, with the weights tilted in favor of the informative features. Then, stratified Random Forests [START_REF] Ye | Stratified sampling for feature subspace selection in random forests for high dimensional data[END_REF] utilized the weights that obtained by Fisher discriminant projection to divide the features into two parts, namely strong and weak features. However, it needs to determine the segmentation threshold of strong and weak features, as well as the amount of strong and weak characteristics. Further, Liang and Huang et al. [START_REF] Liang | Laplacian-weighted random forest for high-dimensional data classification[END_REF] took advantage of the Laplacian score [START_REF] He | Laplacian score for feature selection[END_REF] to quantify the importance of different features by considering their locality preserving power and then generated a set of diverse subspaces by weighted random sampling. To sum up, these studies are mostly concerned with estimating features and raising the weight of excellent features. However, the diversity of random forests is easily reduced by utilizing fixed weights. To improve this situation, we proposed an adaptive Laplacian weight random forest (ALWRF) by dynamically adjusting the weight when constructing trees.

As the decision tree is the basic model of random forest, common decision tree algorithms are introduced first, ie., ID3 [START_REF] Platt | Simplifying decision trees[END_REF], C4.5 [START_REF] Steven | C4. 5: Programs for machine learning by j. ross quinlan[END_REF], Classification and Regression Tree (CART) [START_REF] Breiman | Cart. Classification and Regression Trees[END_REF]. The ID3 algorithm iterates through every unused attribute and calculates the entropy or the information gain of that attribute and it then selects the attribute which has the smallest entropy (or largest information gain) value. ID3 is harder to use on continuous data than on factored data (factored data has a discrete number of possible values, thus reducing the possible branch points) [START_REF] Steven | C4. 5: Programs for machine learning by j. ross quinlan[END_REF]. The C4.5 algorithm is an extension of the earlier ID3 algorithm and can be used for classification. ID3 and C4.5 are time-consuming because of logarithmic operations in entropy models. In the CART algorithm, each node has less than or equal to two children. The bisection method can simplify the scale of decision trees and improve the efficiency of generating decision trees. On the other hand, the CART algorithm can be used to create both classification trees and regression trees, which is suitable for categorical missing values and numerical missing values [START_REF] Nikfalazar | Missing data imputation using decision trees and fuzzy clustering with iterative learning[END_REF]. Therefore, the CART algorithm is employed as a basic model in ALWRF. In the CART algorithm, the outputs for the classification tree and regression tree are discrete value and continuous value respectively. In detail, the output in the classification tree is the majority class of the leaf node, while the regression tree uses the mean value of the leaf node as the output. In addition, the CART algorithm uses the Gini coefficient as the impurity of variables, which can reduce the complexity of logarithmic operations compared with ID3 and C4.5. The smaller Gini coefficient shows the feature is better. The equation for the Gini coefficient is:

Gini(D) = n i=1 p(x i ) * (1 -p(x i )) = 1 - n i=1 p(x 2 i ) (3.1)
where p(x i ) is the probability of occurrence of category x i and n is the number of categories. Gini(D) reflects the probability of two randomly drawn samples from dataset D whose class labels are inconsistent. Therefore, the smaller Gini(D)

represents the higher purity of the dataset D.

In our work, in order to evaluate the feature importance for enhancing the performance of random forest, we resort to the adaptive feature selection technique termed adaptive Laplacian score. The Laplace score is a classical and popular feature selection algorithm in filter style, which aims to find the most discriminative features [START_REF] He | Laplacian score for feature selection[END_REF]. To avoid confusion, we assume that training data with n samples and d dimension. its data matrix can be represented as X ∈ R n×d .

Each row in X = (x 1 , x 2 , ..., x n ) T corresponds to a sample, while each column corresponds to a feature. x i ∈ R d is the i-th sample. Thereby, the data matrix can also be denoted as X = (f 1 , f 2 , ..., f d ), where f j ∈ R n is the j-th feature.

Particularly, a k-nearest neighbor graph is first constructed, which is used to calculate the Laplacian scores of different features by considering their localitypreserving power. We denote this graph as G = {V, E}, where

V = {x 1 , x 2 , ..., x n }
is the set of the training samples and E ∈ R nn is the adjacent matrix. Here, we use the 5-nearest neighbor and the cosine similarity to compute the edge weights.

That is

E = {x ij } n×n (3.
2)

e ij =        ψ(x i , x j ) ψ(x i , x i ). ψ(x j , x j ) , x i ∈ kN N (x j ) or x j ∈ kN N (x i ), 0 , otherwise, (3.3)
where ψ(. ) computes the inner product of two vectors and kN N (x i ) denotes the set of k-nearest neighbors of x i . Let D ∈ R nn be the degree matrix, which is a diagonal matrix with its (i, i)-th element being the sum of the i-th row in E. Let the graph Laplacian of G be denoted as L = D -E. Then, the Laplacian score of the i-th feature f i can be computed as

s i = f ⊤ i L f i f ⊤ i D f i (3.4) f i = f i - f ⊤ i De e ⊤ De e (3.5)
where e = (1, ..., 1) ⊤ . According to equation (3.4), all Laplacian scores of d features can be denoted as S = (s 1 , s 2 , ..., s d ) ⊤ . As a smaller Laplacian score indicates that this feature can better preserve the locality information and therefore can be viewed as a feature of greater importance, the feature weight of the feature f i can be defined as ℓ i = 1s i . Then normalized feature weights can be denote as L = ( ℓ 1 , ℓ 2 , ..., ℓ d ). The ℓ i is computed by

ℓ i = ℓ i d j=1 ℓ j (3.6)
The computed weights L serve as an initial indicator of the importance of each feature and then diversified random subspaces are generated using the weighted random sampling. With the construction of the Laplacian-weighted random forest, the weights of features are adjusted according to the importance of features on prediction. In detail, the importance of features on prediction can be estimated by the accuracy of out-of-bag (OOB) data after adding random noise in the process of constructing a random forest. Generally, the higher importance of a feature on prediction means that changing its value makes predictions more prone to errors. Specifically, the importance of features on prediction in random forests is the sum of importance in all decision trees. We assume that the number of trees in the random forest is m, and the already established set of decision trees is T = {t 1 , t 2 , ..., t m }. The importance of i-th feature on prediction is calculated by

ι i = m j=1 e OOB1 i,j -e OOB2 i,j m (3.7)
where e OOB1 i,j

is the error of corresponding out-of-bag data in j-th decision tree and e OOB2 i,j

is the error of out-of-bag data with randomly added noise. Similarly, the importance of features on prediction can be normalized and denoted as

I = { ι 1 , ι 2 , ..., ι d }
With the increase of the decision tree, the weights of features are adjusted by I. The adaptive weights of features are computed by

w i = (1 -µ) × ℓ i + µ × ι i 2 (3.8)
where ℓ i is the normalized Laplacian weight for the i-th feature and ι i is the normalized importance of the i-th features on prediction. µ is an adjusted parameter which is the ratio of the number of trees that have been constructed

to the number of trees that needs to be generated. The interval for updating weights is γ which means features' weights are updated in specific iterative times.

Additionally, a random operator ϵ is employed to increase the diversity of trees.

Specifically, the weights of features are updated according to a frequency that they are selected when building decision trees. Therefore, the selection probabilities of features with lower weight are increased, which helps construct various trees.

Assuming that the number of selected times for features is N = (ν 1 , ν 2 , ..., ν d ) in decision trees that have been built, and the selected probability of i-th feature is defined as

ρ i = ν i d j=1 ν j (3.9)
As the smaller number of selected times shows the higher locality and lower importance for the feature, ϵ is set to 0.9. When the random number is larger than ϵ, the weight of i-th feature can be updated by Step 2: In ALWRF n 1 number of random records are taken from the training data set having d number of records.

w ′ i = w i + ρ i 2 (3.
Step 3: Individual decision trees are constructed for each sample based on adaptive Laplacian weights.

Step 4: Each decision tree will generate an output.

Step 5: Final output is considered based on majority Voting or averaging for classification and regression respectively using the testing data.

The training and testing process for adaptive Laplacian weighted random forest is shown in Figure 3.1.

Oversampling Technique: SMOTE-NC

Imbalanced classifications pose a challenge for missing value imputation algorithms used for classification were designed around the assumption of an equal number of samples for each class. This results in algorithms that have poor predictive performance, specifically for the minority class [START_REF] Douzas | Improving imbalanced learning through a heuristic oversampling method based on k-means and smote[END_REF]. Many nominal features with missing values have an imbalanced class distribution in medical data. For example, when diabetes is a feature to predict hypertension, the class of diabetes is the majority and the class of health is a minority. Therefore, imputation algorithms have to pay more attention to incomplete and imbalanced focus on the algorithm level [START_REF] Zhao | A cost sensitive decision tree algorithm based on weighted class distribution with batch deleting attribute mechanism[END_REF] and the data level [START_REF] Douzas | Improving imbalanced learning through a heuristic oversampling method based on k-means and smote[END_REF]. At the algorithm level, mainly combined with the characteristics of imbalanced data, to improve the accuracy of minority samples [START_REF] Xu | A cluster-based oversampling algorithm combining smote and k-means for imbalanced medical data[END_REF]. Although this method retains the original data distribution, its usual range is relatively limited [START_REF] Haixiang | Learning from class-imbalanced data: Review of methods and applications[END_REF]. At the data level, the imbalanced level of data is reduced or eliminated mainly by changing the sample distribution of data. Common approaches at the data level contain oversampling of the minority class or undersampling of the majority class. Undersampling technologies have the risk of losing important concepts because they remove a part of the data from the majority classes. At the same time, when the number of observed data is small, undersampling produces smaller data sets, which may limit the performance of models.

Although the random forest method uses the ensemble idea to preserve the original data distribution and improve the performance of a single decision tree in imbalanced data, its application is limited in highly unbalanced data [START_REF] Haixiang | Learning from class-imbalanced data: Review of methods and applications[END_REF]. In this work, an oversampling technique for mixed-type data is employed to overcome the imbalanced problem. Random oversampling and Synthetic Minority Over-sampling TEchnique (SMOTE) [START_REF] He | Imbalanced learning: foundations, algorithms, and applications[END_REF] are two popular oversampling methods. Random oversampling reduces data imbalance by randomly copying minority samples, but blind copying may lead to overfitting [START_REF] Han | Borderline-smote: a new over-sampling method in imbalanced data sets learning[END_REF]. The SMOTE algorithm uses linear interpolation to synthesize a new minority sample between some minority samples, which effectively alleviates the risk of overfitting. Although more improved SMOTE methods have been proposed [START_REF] Hien M Nguyen | Borderline oversampling for imbalanced data classification[END_REF][START_REF] Xu | A cluster-based oversampling algorithm combining smote and k-means for imbalanced medical data[END_REF][START_REF] Douzas | Improving imbalanced learning through a heuristic oversampling method based on k-means and smote[END_REF], they introduce more computations and parameters. For example, Last and Douzas et al. [START_REF] Douzas | Improving imbalanced learning through a heuristic oversampling method based on k-means and smote[END_REF] proposed an advanced oversampling method combining K-Means [START_REF] Macqueen | Some methods for classification and analysis of multivariate observations[END_REF] and SMOTE, which avoids the generation of noise and effectively overcoming the imbalance between classes and within classes. However, this method introduces additional clustering calculations and additional parameters (i.e., the number of clusters k and the density de) compared to the naive SMOTE method. Therefore, we resort Synthetic Minority Over-sampling Technique for Nominal and Continuous features (SMOTE-NC) [START_REF] Nitesh | Smote: synthetic minority over-sampling technique[END_REF] to improve imputation performance when facing incomplete and imbalanced features, which creates synthetic data for categorical as well as quantitative features in the data set. The steps of the SMOTE-NC algorithm are described below and an example of nearest neighbor computation for SMOTE-NC is demonstrated in Table 3.1. Here, M ed 2 is the median of the standard deviations of continuous features of the minority class.

Step 1: Median calculation. Calculate the median of the standard deviations of all continuous features of the minority class. If the nominal features differ between a sample and its potential nearest neighbors, then this median is included in the Euclidean distance computation. The median is used to penalize the variance of nominal features, the amount of which is related to the typical variance of continuous feature values.

Step 2. Nearest neighbor calculation. Calculate the Euclidean distance between the feature vector that is identifying the k-nearest neighbors (minority class samples) and other feature vectors (minority class samples) using a continuous feature space. For each distinct nominal feature between the considered feature vector and its potential nearest neighbor, including the median of the standard deviations previously computed, in the Euclidean distance computation.

Step 3. Populate the synthetic sample. The continuous features of the new synthetic minority class sample are created using the same approach of SMOTE [START_REF] He | Imbalanced learning: foundations, algorithms, and applications[END_REF] as described earlier. The nominal feature is given the value occurring in the majority of the k-nearest neighbors (mode).

Two Cases

F1 = {1 2 3 A B C}, F2 = {4 6 5 A D E}

Median Calculation

It includes twice for the 5th feature: B→D and the 6th: C→E, which differ for the two feature vectors.

Nearest Neighbor

Calculation

Euclidean Distance: sqrt[(4 -1) 2 + (6 -2) 2 + (5 -3) 2 + M ed 2 + M ed 2 ]
Table 3.1: Example of nearest neighbor computation for SMOTE-NC.

The Proposed Imputation Method

The random forest method is suitable for imputing incomplete and mixedtype data as it works for classification and regression tasks [START_REF] Daniel | Missforest-non-parametric missing value imputation for mixed-type data[END_REF]. We apply the proposed adaptive Laplacian weight random forest and the SMOTE-NC method to impute incomplete data with the characteristics of imbalance and mixed type, called SMOTE-NC and ALWRF Imputation (SncALWRFI). Specifically, its procedure is iterative, in which it uses mean and mode values to replace missing data and then it updates missing values on each successive iteration. Consider a given dataset D, where The feature set is F . The features can be either numerical or categorical. The SncALWRFI method includes 6 steps as follows:

Step 1. Calculate the missing rate of all features F with missing values, and sort the features in descending order. The sorted feature set is donated as

F ( F ⊆ F ).
Step 2. Calculate an indicator matrix (donate as M ) to record the location of missing values, where observed values are 1 and missing values are 0. Then the average of the numerical features and the mode of the categorical features are used to initially impute missing values, donate as D ′ .

Step 3. For each feature f i ∈ F that has a missing value for some of the records, the full dataset D ′ is divided into two subsets D i I and D i C according to the indicator matrix M , where D i I contains all records with missing values at the feature f i and D i C contains records with no missing value at the feature f i . Step 4. Some available values (value = 1) in the data matrix are set to missing (value = 0) and then these values will be used for estimating the tuning parameters. According to the location of simulated missing values, D i C is dived into D training , D testing . Cross-validation is used to automatically select the values of the tuning parameters yielding the smallest imputation error. Meanwhile, the SMOTE-NC method is applied to imbalanced and categorical features. At last, an ALWRF model (donate as F f i ) is built so that the feature f i is the targeted variable and the rest of the features without missing values are predictive features.

If the targeted variable is a numerical variable, the built forest is a regression forest. If the targeted variable is a categorical variable, a classification forest is built. To compute the optimal values of the tuning parameters, the optimization procedure is described in section 3.2.

Step 5. Use the optimal values of the tuning parameters to build an ALWRF model (donate as F optimal

Hyperparameter Optimization

The proposed imputation method SncALWRFI requires the tuning parameters to be specified including the parameters in random forest and the parameters in SMOTE-NC. Bayesian optimization(BO) [START_REF] Snoek | Practical bayesian optimization of machine learning algorithms[END_REF] is a state-of-the-art optimization framework for the global optimization of expensive black-box functions [START_REF] Snoek | Practical bayesian optimization of machine learning algorithms[END_REF][START_REF] Xia | A boosted decision tree approach using bayesian hyper-parameter optimization for credit scoring[END_REF], which can find the optimal value through only a small number of

M SE(D) = 1 n n i=1 (f (x i ) -y i ) 2 (3.12)
where f (x i ) is the prediction value and y i is the real value. Due to MSE reflecting the overall deviation of the predicted and true values, the smaller M SE is better.

As the low MSE values mean better performance, negative MSE is applied as the optimization function for regression tasks. Step 4. Repeat the above steps until reaching the limit of iterations.

Bayesian Optimization: ALWRF

In addition, in order to optimize the hyper-parameters of the ALWRF, the dataset is divided into training data, validation data, and testing data. Training data is applied to train the ALWRF model. Validation data is used to tune hyperparameters. The performance of ALWRF is evaluated using testing data.

The flow chart of the Bayesian optimization process for the ALWRF is shown in

Bayesian Optimization: SncALWRFI

Similarly, Bayesian optimization is also employed for hyperparameters of the proposed SncALWRFI method. However, the bayesian optimization process for the SncALWRFI has three differences from ALWRF. Firstly, as SncALWRFI pays attention to missing values, the optimization function is different and it needs to consider both categorical and numerical features. Therefore, the Proportion of Falsely Imputed Categories (P F C cat ) is employed as a performance measure for categorical variables, while the Mean of Squared Imputation Errors for numerical values (M SIE num ) is used as a performance measure for continuous variables.

P F C cat = 1 N I(x ij ̸ = x ′ ij ) (3.13)
where I(.) is an indicator function, which is 1 when the predicted value and the true value are the same. In addition, the M SIE num can be calculate by

M SIE num = 1 N (x ij -x ′ ij ) (3.14)
where N is the number of numerical missing values, x ij is the true value in the complete data matrix, and x ′ ij is the corresponding imputed value. Then the optimization function of the SncALWRFI is the sum of P F C cat and M SIE num .

Secondly, missing values should be randomly introduced in validation data. In detail, we temporarily set as missing some of the available values in the full data matrix, and these missing records make up the validation data for estimating hyperparameters. The third difference is that more parameters should be considered because of the SMOTE-NC method. The additional parameters include knn and irt which are the number of neighbors in SMOTE-NC and the threshold of imbalance rate respectively. The ranges of value for hyperparameters are knn ∈ {3, 5, 20} and irt ∈ {2, 5, 10}.

Experiments for Adaptive Laplacian Weight Random Forest

At first, two experiments are conducted to evaluate the performance of the proposed adaptive Laplacian weight random forest. As two category tasks including classification and regression tasks can be applied in the random forest model, we used 4 public medical datasets and 4 public datasets to evaluate the classification and regression performance of the AILWRF method, respec-tively. In this experiment, feature scaling is not required since the proposed and compared methods are tree-based models. All models were implemented using Python Language and the configuration of the experimental environment is Intel(R) Core(TM) i5-8265U CPU @ 1.60GHz 1.80 GHz, 8 GB RAM.

Classification Task

As we introduced in section 3.1, the classification task is that learn how to assign a class label to samples. Therefore, prediction accuracy and AUC are employed as performance measurements for the classification task. The AUC-ROC curve is a common performance measurement for classification problems at various threshold settings. ROC is a probability curve and AUC represents the degree or measure of separability. It tells how much the model is capable of distinguishing between classes. A higher AUC means that the model has a higher capability to predict class 0 as 0 and class 1 as 1. By analogy, a higher AUC in disease prediction shows the model has a better ability at distinguishing between patients with the disease and no disease.

In the classification task, the information of 4 public medical datasets is shown in Table 3.2. Specifically, three datasets focus on hypertension prediction including Men's dataset, Women's dataset, and the NHANES dataset. Men's dataset and Women's dataset are freely available in a web repository for reproducible purposes [START_REF] Golino | Women's dataset from the'predicting increased blood pressure using machine learning[END_REF][START_REF] Golino | Men's dataset from the'predicting increased blood pressure using machine learning[END_REF]. The predictive variables included in these datasets were Body Mass Index (BMI), WC (Waist Circumference), HC (Hip Circumference), and WHR (Waist-to-Height Ratio). NHANES dataset [START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF] is a subset of National Health and Nutrition Examination Survey (NHANES) from 2007 to 2017. This dataset can be used to predict the occurrence of hypertension using 7 features that associate with hypertension, such as gender, race, age, smoking, BMI, diabetes, and kidney conditions. The fourth datasets is called Pima dataset [START_REF] Smith | Using the adap learning algorithm to forecast the 159 onset of diabetes mellitus[END_REF], which is originally from the National Institute of Diabetes and Digestive and Kidney Diseases. The objective of this dataset is to diagnostically predict whether or not a patient has diabetes, based on certain diagnostic measurements included in the dataset. In order to evaluate the performance of the ALWRF after Bayesian optimization (BO-ALWRF), random forest(using default parameters) and random forest after Bayesian optimization (BO-RF) methods are employed for comparison. First, the three methods were performed 20 times, and then four boxplots were used to present their accuracy values across the four datasets, as shown in

R 2 = 1 - n i=1 (y i -f (x i )) 2 n i=1 (y i -y) 2 (3.15)
where f (x i ) is the prediction value, y i is the real value and y is the mean of y.

The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). As such variance is dataset dependent, R 2 may not be meaningfully compared across different datasets. The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant and then improves the imputed data by random forest model on each successive iteration. Further, the wN N Sel mix approach makes practical and effective use of the information on the association among the variables to improve imputation accuracy. In this experiment, three missing percentages of 10%, 20%, and 30% are simulated. For each missing percentage, we repeat each configuration 200 times to reduce noise from simulating missing values. In order to compare the performance of different imputation procedures, P F C cat and M SIE num are used as performance measures for categorical and continuous variables, respectively.

Specifically, five public datasets are used including German Breast Cancer Study Group 2 (GBSG2) data, Hepatitis dataset (Hepatitis), Body Mass Index dataset (BMI), Cars dataset(Cars), and Automobile dataset (Automobile). The Hepatitis dataset is from UCI Machine Learning Repository [START_REF] Frank | Uci machine learning repository[END_REF] and the other four datasets can be found in the R package. In order to compare performance with wN N Sel mix (denoted as wN N ), the same experimental datasets [START_REF] Faisal | Imputation methods for high-dimensional mixed-type datasets by nearest neighbors[END_REF] are used and their information is shown in Table 3 According to Table 3.4, it is easy to observe that some of the existing features in the four datasets are unbalanced, especially in the Automobile dataset.

The experiment results are shown in Table 3 Table 3.5: The experiment results of imputation errors.

In Table 3.5, the total error is listed on the left and the error for categorical variables is on the right. The error for numerical variables can be calculated using the total error minus the error for nominal variables. From Table 3.5, the imputation quality is affected by the percentage of missing values. Especially in the BMI and Cars datasets, imputation accuracies of all approaches deteriorated rapidly with increasing missing values. While compared with other methods, our proposed SncALWRFI method has the best total error regardless of the percentage of missing data. The KNNI method always provides poor imputation because the other three methods seem to use the correlation among covariates for imputation to provide better imputation results. However, the imputation performance of the proposed method for categorical variables is lower than that of the wN N Sel mix method in the BMI and Cars datasets at the 30% missing rate. The main reason is that the features in the BMI dataset are nearly balanced, and only a few samples are available to build the random forest model for the Cars dataset. As expected, our proposed SncALWRFI method outperforms the other three models overall, which is attributed to adaptive Lapland weights and oversampling techniques.

Imputation Effectiveness in Classification Tasks

The imputation error describes how accurately the imputation of missing values is done by the imputation techniques. However, it does not guarantee that a good imputation always improves data quality for a data mining task such as classification [START_REF] Md | Data quality improvement by imputation of missing values[END_REF]. Therefore, the main objective of this section is to evaluate the effectiveness of the imputation techniques for data mining by applying several classifiers on the original data set, imputed data set and the data sets have missing values. As the prediction accuracy of a classifier can be used to evaluate the impact of the imputation of missing values [START_REF] Md | Data quality improvement by imputation of missing values[END_REF], an evaluation model is built in order to find the prediction accuracy as the effectiveness of an imputation technique. In addition, since the true value of missing data is unknown in the real world, imputation effectiveness in classification tasks is more important than the evaluation of imputation error. Therefore, we paid more attention to this experiment and used two types of dataset terms complete data and incomplete data. different missing rates can be easily simulated in complete data, while it can not replace real missing values. The overall block diagram of the experiment flow is shown in Figure 3.9.

In this experiment, a dataset is firstly divided into two sub data sets namely Specifically, five missing rates are adopted including 10%, 15%, 20%, 25%, and 30% in this experiment. For each missing rate, each configuration is also repeated 200 times. In addition, in order to compare our model with more models that are suitable for mixed-type data, we implemented four imputation methods using a similar strategy with [START_REF] Daniel | Missforest-non-parametric missing value imputation for mixed-type data[END_REF] based on the CART tree [START_REF] Breiman | Cart. Classification and Regression Trees[END_REF], the Adaptive Boosting Decision Tree (ABDT) [START_REF] Freund | A decision-theoretic generalization of on-line learning and an application to boosting[END_REF], the Gradient Boosting Decision Tree (GBDT) [START_REF] Hastie | Boosting and additive trees[END_REF] and Multi-Layer Perceptron (MLP) [START_REF] Kubat | Neural networks: a comprehensive foundation by simon haykin[END_REF]. Therefore, seven imputation methods including k-nearest neighbors imputation (kNNI), the random forest imputation (RFI), the decision tree imputation (DTI), the AdaBoost decision ent structures are adopted as evaluation classifications, namely Linear Regression (LR) [START_REF] Mark | Multiple regression and correlation[END_REF], Naive Bayes Network (NB) [START_REF] Webb | Naïve bayes[END_REF] and Support Vector Machine (SVM) [START_REF] Cortes | Support-vector networks[END_REF].

Firstly, we used six complete medical datasets to evaluate imputation effectiveness including Statlog heart data (Statlog), heart failure by cardiovascular diseases (Heart Failure), early-stage diabetes risk prediction dataset (Diabetes Risk), contraceptive method choice dataset (CMC), the dataset for estimating obesity levels based on eating habits and physical condition (Obesity) [START_REF] Chicco | The coefficient of determination r-squared is more informative than smape, mae, mape, mse and rmse in regression analysis evaluation[END_REF], and cardiovascular disease dataset (Cardiovascular). The Obesity and Cardiovascular datasets are from the Kaggle platform and the other four datasets are from UCI Machine Learning Repository [START_REF] Frank | Uci machine learning repository[END_REF]. The information of these six experiment datasets is shown in Table 3 Table 3.6: The information of six public medical datasets.

In order to compare the overall performance of imputation methods, we calculated the average accuracy of five levels of missing rate in three classifiers for each dataset, as shown in Figure 3.10.

models in all missing rates. While our proposed method performs best under different missing rates across all datasets, it shows that our proposed imputation method is robust in different missing rates.

Further, to verify the performance of our proposed imputation method in datasets with true missing values, six public medical datasets from UCI [START_REF] Frank | Uci machine learning repository[END_REF] with real missing values are used, including the Cleveland heart disease dataset (Cleveland), Hepatitis, primary tumor dataset (Primary Tumor), chronic kidney disease dataset (Chronic Kidney), Thyroid dataset and Framingham heart study cohort dataset (Framingham). Their information is shown in Table 3 In order to show the distribution of missing values in the datasets, we use the missing matrix to identify where missing values occur in real cases. The missing matrix of these six datasets is shown from Figure 3.12 to Figure 3.17.

dataset, even for two deletion methods. The reasons are the missing rate is low and missing features play less impact on the predicted outcome. In this case, PD may be a suitable choice. However, for the other four datasets with high instance missing rates, our proposed method consistently shows the best performance, except for the performance of the LD method for the NB classifier on the Primary Tumor Dataset. Although the LD method outperforms our proposed method on the Primary Tumor dataset for the NB classifiers, its performance is highly weaker for LR and SVM classifiers. Generally, when missing values are concentrated in a few features, the PD can be used, but if these features need to be preserved, our proposed method has great competitiveness. To sum up, although our proposed imputation method performs similarly to other methods in datasets with low missing rates, it shows the best imputation effectiveness in datasets with high missing rates. But where the missing rate of the dataset is low, the prediction accuracy of our method is not always better than the deletion method in classification tasks. Therefore, our method is not necessarily the best choice for studies that have low missing rates and focus only on classification accuracy. But for studies with a high missing rate or need to retain more samples, our method can significantly improve data quality.

Summary

As features uniformly and randomly are selected to form a feature subspace in a random forest, features with high quality are not fully utilized. We proposed an improved random forest model, called adaptive Laplacian weight ran-dom forest (ALWRF), in which features' weights adaptively adjust when building a random forest. Meanwhile, cross-validation and Bayesian optimization are employed to search hyper-parameters. Then eight pubic datasets are used to verify the prediction ability of the ALWRF on the classification and regression tasks.

The experiment results show that the ALWRF outperforms random forest and Bayesian optimized random forest.

Missing values is an inevitable problem when mining useful information from medical data. In order to improve the quality of incomplete medical data with the characteristics of imbalance and mixed type, an imputation method (SncALWRF) is proposed based on the ALWRF and the oversampling technology SMOTE-NC. In the experiment for missing values, we first compared the imputation errors of the proposed method with three advanced imputation methods using five small complete data subsets. Experiment results show that the proposed method provides excellent imputation estimates for missing values in categorical and numerical variables.

We then focus on the imputation effectiveness of the proposed imputation method in the classification tasks. We first used six complete datasets with the characteristics of imbalance and mixed type to evaluate the prediction accuracy of the proposed imputation model at different missing rates. Experiment results show although with the increasing of missing ratio, the imputation performance for all imputation methods deteriorates, the decrease is more gradual for the proposed method. At the same time, our method outperforms other imputation methods in the same missing values. We then adopted six public medical datasets with real missing values and compare them to evaluate the effectiveness of our proposed method in classification tasks and compared them with other 6 imputation methods and 2 deletion methods. Experiment results show when datasets with low missing rates (5%), our model can not always perform well than deletion methods, but it outperforms other imputation methods in the real case study. Therefore, our imputation method can significantly improve data quality for studies with high missing rates or the need to retain more samples. to employ a single model to generate more accurate forecasts and attain higher levels of performance due to the noise from attributes and classes. In machine learning, an ensemble is a sort of model that is built by merging the predictions of various individual models. Typically, ensembles increase performance by reducing the mistakes created by each individual model that contributes to the ensemble.

Generally, there are two challenges in the ensemble framework in terms of model selection and model fusion.

Model Selection

There are many types of research devoted to the selection of meta-leaners.

The paper [START_REF] Shunmugapriya | Optimization of stacking ensemble configurations through artificial bee colony algorithm[END_REF] adopted prediction accuracy as an objective, optimized by an artificial bee colony algorithm to collect meta-learners. In [START_REF] Chen | Applying ant colony optimization to configuring stacking ensembles for data mining[END_REF], the ant colony algorithm was applied to optimize local information, which represented the precisions of the meta-level classifiers to configure stacking ensembles. But the single-objective optimization algorithms usually adopt a greedy search strategy that easily leads to a local minimum. It doesn't take much accuracy improvement but excess meta-learners. The paper [START_REF] Furtuna | Multi-objective optimization of a stacked neural network using an evolutionary hyper-heuristic[END_REF] adopted a multi-objective optimization algorithm named non-dominated sorting genetic algorithms-II (NSGA-II) to evolve an ensemble and the result is averaged by each individual. It maximizes the generalization capacity of the ensemble and minimizes its structural complexity simultaneously to get a better ensemble. While the papers [START_REF] Mao | Maximizing diversity by transformed ensemble learning[END_REF] and [START_REF] Chen | Multiobjective neural network ensembles based on regularized negative correlation learning[END_REF] describe that the ideal ensemble is constructed using learners of small error and good diversity. However, rich diversity may cause the predicted value of meta-learners to deviate from the true values, and the improvement of individual accuracy often reduces the diversity of meta-learners, that is, accuracy and diversity are usually conflicting with each other. Further, the selection of meta-learners in the paper [START_REF] Li | An optimal stacking ensemble for remaining useful life estimation of systems under multi-operating conditions[END_REF] followed the NSGA II algorithm to balance the two conflicting objectives in terms of accuracy and diversity. As the NSGA II algorithm randomly initializes the population, optimal individuals are changeable and it requires more meta-learners when generating the offspring in the NSGA II algorithm.

To sum up, accuracy and diversity are two crucial factors that decide the success of stacking. In order to maximize the diversity and the accuracy of ensemble models simultaneously, we proposed a Multi-objective Iterative Model Selection (MoItMS) algorithm. Specifically, accuracy measures the difference between the predicted values and actual values while diversity measures the differences between meta-learners. Suppose there are k individual models which are selected by MoItMS, for the cost function C m i is defined as:

C m i = E m i + λD m i (4.1)
where E m i represents the accuracy, and D m i represents the diversity. λ is the weight and λ = 1. Here E m i can be computed by:

E m i = 1 N N j=1 (p j m i -y j ) (4.2)
where y j is the actual values of the j-th training sample, and the p j m i is the predicted values obtained by the i-th meta-learner for the j-th training sample.

Here, the predicted probabilities are applied to the predicted values instead of class labels. N is the number of samples. According to the paper [START_REF] Li | An optimal stacking ensemble for remaining useful life estimation of systems under multi-operating conditions[END_REF], the correlation D m i is defined as:

D m i = 1 N N j=1 ((p j m i -p j avg ) k l̸ =i p j m l -p j avg ) (4.3)
where p j m i and p j m l represent the predicted values of the i-th and l-th metalearners for the j-th training instance, respectively. p j avg is the average predicted value of the models in the ensemble. Reference [START_REF] Chen | Multiobjective neural network ensembles based on regularized negative correlation learning[END_REF] proves that good diversity can be achieved (if there is no bias) when the individual models are negatively correlated, which means the lower the D m i is, the larger the diversity is. Further, since the two objective functions have different magnitudes, normalization is required so that the algorithm does not favor a larger magnitude. Therefore, the objective functions f , f ∈ {E, D} can be normalize by

f (X) = f (X) Z f (4.4)
where X represents candidate models. Z f is the normalization factor of each objective function which is the maximum function value in the candidate models. Therefore, the cost function for the ensemble can be the average of these individual model's costs:

C = 1 k k i ( E m i + D m i ) (4.5)
where E m i , D m i are the accuracy and diversity of i-th meta-learner after normalization, respectively. The small value of C means that the ensemble model combines meta-models with high accuracy and diversity. In order to maximize the accuracy and diversity of the ensemble model, an iterative process is employed to search for the best cost. In detail, the proposed MoItMS algorithm mainly includes six steps:

(1)Firstly, five-fold cross-validation is used to generate a predicted set of a dataset X, which will be applied to assess the accuracy and diversity of each individual model.

(

Y ′ = {Y ′ 1 , Y ′ 2 , ..., Y ′ s } ←

Model Fusion

In general, ensemble models can be categorized into the homogeneous ensemble and heterogeneous ensembles according to the structure of the component model. Homogeneous ensemble mainly ensemble decision trees in terms of bagging and boosting technologies. Bagging technology [START_REF] Breiman | Bagging predictors[END_REF] often considers homogeneous learners, learns them independently from each other in parallel, and combines them following some kind of deterministic averaging process. Random Forest [START_REF] Breiman | Random forests[END_REF] is the representative model in bagging technology. While boosting technology [START_REF] Mason | Boosting algorithms as gradient descent[END_REF] learns learners sequentially in an adaptative way (a model depends on the previous ones) and combines them following a deterministic strategy, such as Adaptive Boosting (AdaBoost) [START_REF] Robert | Improved boosting algorithms using confidence-rated predictions[END_REF], Extreme gradient boosting (XGBoost) [START_REF] Chen | Xgboost: A scalable tree boosting system[END_REF] and Light gradient boosting machine (LightGBM) [START_REF] Ke | Lightgbm: A highly efficient gradient boosting decision tree[END_REF]. Further, stacking technology [START_REF] David | Stacked generalization[END_REF] generally considers heterogeneous learners, learns them in parallel, and combines them by training a meta-model to output a prediction based on the different model predictions. Even though different models may have similar error rates, stacking ensembles tend to make different mistakes, since they get different professions. In order to search best leaners for staking ensemble, ACO (Ant Colony Optimization) [START_REF] Chen | An ant colony optimization approach for stacking ensemble[END_REF], GA (Genetic Algorithms) [START_REF] Francisco | Genetic approach for optimizing ensembles of classifiers[END_REF] and NSGA II (non-dominated sorting genetic algorithms-II) [START_REF] Li | An optimal stacking ensemble for remaining useful life estimation of systems under multi-operating conditions[END_REF] have been resorted. The categories of ensemble models and their representative models are shown in Figure 4.1. According to some studies [START_REF] López-Martínez | Machine learning classification analysis for a hypertensive population as a function of several risk factors[END_REF][START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF] related to high blood pressure, they all exclude people under the age of 20. The main reason is that the occurrence of hypertension at the age of 20 is mainly related to genetic factors. According to the American Heart Association's definition of hypertension, which uses blood pressure as the dichotomous dependent variable in this study, hypertension is defined as having a systolic blood pressure that is more than or equal to 140 mmHg [START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF]. Following the cleaning of the data, we used only the records that included values that were not null. Table 4.1 presents the distribution of samples based on the type of hypertensive people, as well as the people's gender and race.

Ensemble Approach

93

We conducted literature research that have used machine learning techniques to predict the occurrence of hypertension among different populations to identify several risk factors, including demographic variables in terms of age [START_REF] Wmaw Ahmad | Association of hypertension with risk factors using logistic regression[END_REF][START_REF] Ye | Prediction of incident hypertension within the next year: prospective study using statewide electronic health records and machine learning[END_REF][START_REF] Latifa A Alkaabi | Predicting hypertension using machine learning: Findings from qatar biobank study[END_REF][START_REF] Daniel H Katz | Phenomapping for the identification of hypertensive patients with the myocardial substrate for heart failure with preserved ejection fraction[END_REF], gender [START_REF] Ye | Prediction of incident hypertension within the next year: prospective study using statewide electronic health records and machine learning[END_REF][START_REF] Latifa A Alkaabi | Predicting hypertension using machine learning: Findings from qatar biobank study[END_REF][START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF], race [START_REF] López-Martínez | Machine learning classification analysis for a hypertensive population as a function of several risk factors[END_REF][START_REF] Huang | Evaluating the risk of hypertension using an artificial neural network method in rural residents over the age of 35 years in a chinese area[END_REF][START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF], education [START_REF] Huang | Evaluating the risk of hypertension using an artificial neural network method in rural residents over the age of 35 years in a chinese area[END_REF][START_REF] Latifa A Alkaabi | Predicting hypertension using machine learning: Findings from qatar biobank study[END_REF],

examination data like body measures [START_REF] Wmaw Ahmad | Association of hypertension with risk factors using logistic regression[END_REF][START_REF] Ye | Prediction of incident hypertension within the next year: prospective study using statewide electronic health records and machine learning[END_REF][START_REF] Huang | Evaluating the risk of hypertension using an artificial neural network method in rural residents over the age of 35 years in a chinese area[END_REF][START_REF] Latifa A Alkaabi | Predicting hypertension using machine learning: Findings from qatar biobank study[END_REF][START_REF] Fernandes Golino | Predicting increased blood pressure using machine learning[END_REF] and waist [START_REF] Daniel H Katz | Phenomapping for the identification of hypertensive patients with the myocardial substrate for heart failure with preserved ejection fraction[END_REF][START_REF] Fernandes Golino | Predicting increased blood pressure using machine learning[END_REF], chronic diseases in terms of diabetes and kidney conditions [START_REF] Ye | Prediction of incident hypertension within the next year: prospective study using statewide electronic health records and machine learning[END_REF][START_REF] López-Martínez | Machine learning classification analysis for a hypertensive population as a function of several risk factors[END_REF][START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF] and lifestyle factors such as smoking cigarette use [START_REF] Latifa A Alkaabi | Predicting hypertension using machine learning: Findings from qatar biobank study[END_REF][START_REF] López-Martínez | Machine learning classification analysis for a hypertensive population as a function of several risk factors[END_REF][START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF], alcohol use [START_REF] Huang | Evaluating the risk of hypertension using an artificial neural network method in rural residents over the age of 35 years in a chinese area[END_REF][START_REF] Wai | A prediction model of blood pressure for telemedicine[END_REF], exercise [START_REF] Huang | Evaluating the risk of hypertension using an artificial neural network method in rural residents over the age of 35 years in a chinese area[END_REF][START_REF] Latifa A Alkaabi | Predicting hypertension using machine learning: Findings from qatar biobank study[END_REF], diet [START_REF] Huang | Evaluating the risk of hypertension using an artificial neural network method in rural residents over the age of 35 years in a chinese area[END_REF][START_REF] Latifa A Alkaabi | Predicting hypertension using machine learning: Findings from qatar biobank study[END_REF] and sleeping [START_REF] Wmaw Ahmad | Association of hypertension with risk factors using logistic regression[END_REF][START_REF] Yen | Deep learning algorithm evaluation of hypertension classification in less photoplethysmography signals conditions[END_REF]. We used hist charts to simply analyze the correlation between these features with hypertension as shown in The diagnostic ability of a classifier system to distinguish between non-hypertension and hypertensive people.

Experimental Setup

This research mainly focuses on improving the AUC of the ensemble classification approach because it tells how much the model is capable of distinguishing between classes. For performance evaluation, firstly the proposed ensemble learning approach is compared with various individual learners such as multilayer perceptron (MLP) [START_REF] Patnaik | Predicting the occurrence of essential hypertension using annual health records[END_REF], k-nearest neighbors (KNN) [START_REF] Singh Rajput | Hypertension diagnosis index for discrimination of high-risk hypertension ecg signals using optimal orthogonal wavelet filter bank[END_REF], Decision Tree (DT) [START_REF] Fiarni | Analysis and prediction of diabetes complication disease using data mining algorithm[END_REF], support vector machine (SVM) [START_REF] Amaratunga | Uses and opportunities for machine learning in hypertension research[END_REF], Gaussian Naive Bayes (Gaussian NB) [START_REF] Kraisangka | Derivation of a bayesian network model from an existing risk score calculator for pulmonary arterial hypertension[END_REF] and Logistic Regression Model (LRM) [START_REF] Nusinovici | Logistic regression was as good as machine learning for predicting major chronic diseases[END_REF], which are mostly utilized in the existing research on the diagnosis of hypertension. Secondly, the proposed method is compared with six well-known ensemble learning methodologies namely bagging, boosting, and stacking. Specifically, random forest (RF) uses a bagging ensemble technic based on multiple decision trees, and Adaptive Boosting (AdaBoost), Extreme gradient boosting (XGBoost), and Light gradient boosting machine (LightGBM) are based on residual iterative tree. Further, two state-of-the-art staking ensemble models are used. The paper [START_REF] Singh | Stacking-based multi-objective evolutionary ensemble framework for prediction of diabetes mellitus[END_REF] developed a stacking-based evolutionary ensemble learning system 'NSGA-II-Stacking' for predicting the onset of Type-2 diabetes mellitus based on SVM and DT. Then, the paper [START_REF] Li | An optimal stacking ensemble for remaining useful life estimation of systems under multi-operating conditions[END_REF] proposes an optimal stacking ensemble approach combining different learning algorithms, which selects meta-learners following a multi-objective evolutionary algorithm named non-dominated sorting genetic algorithms-II. We

Hyperparameter Optimization

The parameter adjustment range of all models is set to a commonly used range and the final setting of parameters is carried out by using Bayesian optimization [START_REF] Snoek | Practical bayesian optimization of machine learning algorithms[END_REF]. Specifically, the study population [START_REF] Dalmat | Brève : La seine-et-marne, un désert médical ?[END_REF]341) was split into a training dataset and a testing dataset. The training dataset was derived from a random sampling of 70% (7,939) of the extracted study population and the testing sampling of the remaining 30% (3,402) to evaluate the model on data sets with known labels (ground truth) that were never used for training. Therefore, we employed Bayesian optimization and five cross-validations to search parameters using the training dataset, which is implemented by the hyperopt package [START_REF] Cox D Bergstra | Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures[END_REF] in Python.

The maximum iterative time is set as 50. The hyperparameter space for models is shown in Table 4 We can see in Figure 4.6 that KNN has the best performance, whereas MLP and LRM have similar E and D values. Then, when we applied the proposed the proposed method in accordance with the MoItMS methodology. Additionally, stacking is used in this paper for better fusion, and a neural network model with a hidden layer is used for the meta-classifier. This is due to the fact that the neural network model has the potential to produce, and that having one hidden layer can shorten the time that is consumed.

Model Evaluation

In this section, a comparative analysis of the suggested approach and thir- Here, two sophisticated stacking ensemble models are used as benchmarks in this study. [START_REF] Singh | Stacking-based multi-objective evolutionary ensemble framework for prediction of diabetes mellitus[END_REF] SVMs and DTs were used as the base learner, and the NSGA-II algorithm was used to combine models that were trained on different sub-datasets. In the paper [START_REF] Li | An optimal stacking ensemble for remaining useful life estimation of systems under multi-operating conditions[END_REF], the NSGA-II algorithm was used to choose a model from a set of individual and tree-based ensemble models. In addition, voting is usually beneficial when aggregating a large number of base learners that attain comparable performance for similar work. As a result, an ensemble model based on Majority Voting is used as a benchmark against which the proposed stacking framework is measured. demonstrates that it is superior to an inference drawn at random.

teen

Extensive Approach Evaluation

The paper has shown that the classification capability of the model improved (AUC=0.8420) when applied to the input features19 features. In previous research, the results of artificial neural networks (AUC=0.77) were utilized when applied to the input features of gender, race, BMI, age, smoking, kidney conditions, and diabetes. In order to further explore the performance of our proposed approach, we conduced an experiment on the same dataset [START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF] with the previous research. According to the proposed approach for model selection, MLP, LRM, and Gaussian NB models are employed as base models in level-0. Six machine learning algorithms in the paper [START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF] were identified and compared, including de- utilized in the process of training the meta-learner. After that, the meta-learner is used to make predictions regarding the testing samples. The effectiveness of the proposed ensemble technique is evaluated with reference to both individual and ensemble models, which serve as baseline models. The comparative findings reveal that the proposed model performs better than the baseline individual and ensemble models according to five specified evaluation measures, such as accuracy, precision, recall, F1-measure, and AUC value. These metrics include accuracy, precision, and recall. In addition, we assessed the suggested stacking structure by employing hypertension datasets that included gender, race, BMI, age, smoking, kidney problems, and diabetes. According to the findings of the experiment, the proposed method performs better than the previous studies on all five of the evaluation measures that were used. Finally, we evaluated the effect of lifestyle factors on the classification performance for hypertension, and we found that lifestyle factors can help the model discriminate hypertensive samples from normal samples. In future studies, a more in-depth examination and screening of features will be considered. On the other hand, in order to verify the proposed framework, hypertension can be predicted using a variety of data sets, including those with various features and risk factors. The maximum prediction result of 75.98 is obtained in the LGBM model, according to experimental results, while removing features with missing values yields the lowest prediction results. However, our proposed approach performs at its best, achieving an average ideal value of 84.83 in the LGBM model. Therefore, as the output data for the missing value module, we will ultimately select the data set without missing values that was processed using the suggested SncALWRFI approach.

Feature Selection Based on Feature Importance

The highly accurate and robust random forest-based feature selection (RF FS) method was introduced in 5.1.2. In the feature selection module, specifically, the data without missing values preprocessed by the missing value module will be The top-N important features or all features with importance greater than 0 can be chosen once the calculation of feature importance is complete. In order to keep as many features as possible, the feature selection module selects according to the important threshold of the feature, that is, the features with importance of more than 0 are picked, and 16 features are then discarded. The final experimental dataset will have 32,784 instances and 45 features. We use the same three prediction models and conduct 20 runs to confirm the impact of feature selection strategies on LRDs' prediction outcomes. The Table 5 The experimental results demonstrate that feature selection increased the performance of the three prediction models, demonstrating that the feature selection method based on random forest can increase the accuracy of LRDs prediction after removing some features with low importance.

The Construction of LRDs Ensemble Prediction Model

After analysis based on key features, the dataset with key features will Finally, in order to manage and prevent LRDs, we will also plan to create a website that predicts LRDs. We present various modules in the thesis that can assist our website to provide effective medical services.

Summary

This chapter primarily serves to demonstrate the three modules-missing value, feature selection, and disease prediction-that make up the proposed prediction framework. It begins by thoroughly introducing each module before analyzing a case from Nanjing, China, and using hypertension as an example for this case study. The integrated prediction model is built using a data set that only contains the important features after the missing values in the case have been processed, analyzed, and evaluated in terms of importance. Finally, the constructed model is evaluated using a range of indicators to examine its applicability to the scenario.

Chapter 6. Conclusions and Future Work Finally, we use a case from China to apply the proposed prediction framework. Two significant models-missing value imputation models and disease prediction models-are produced following processing by the three primary modules of missing value, feature selection, and disease prediction. The proposed prediction framework can also enhance LRDs' predicting performance for better public health prevention, according to the experimental results.

6.2 Future Work

Designing of LRDs Risk Prediction Website

In order to demonstrate the generalizability of the proposed approach, the study also lacks a long-term perspective on various use cases (chronic diseases

• Health information. The website will regularly update health information and provide knowledge and advice on health.

The technical implementation of the LRDs risk prediction website includes four important parts.

• The front end of the website will be implemented using HTML, CSS, JavaScript, and other technologies, and adopts a responsive design to adapt to different devices and screen sizes.

• The back-end of the website will be implemented with Python language and Django framework, including user management, data management, prediction model, and other functions.

• The data of the website will be stored in a MySQL database, including user information, prediction results, health advice, etc.

• The Prediction model of the website will be implemented using the proposed forecasting framework, which can be trained according to different disease types and data provided by users to improve the accuracy of forecasting.

In order to ensure the security of user information, the website uses SSL certificates for encrypted transmission, and at the same time backs up and encrypts user data. This website aims to help users better understand their physical conditions and risks, and provide corresponding health advice and information, but it cannot replace the doctor's diagnosis and treatment. Users should treat it with caution when using it, and consult a professional in time if they have any questions doctor.

Considering Medical Data with Multiple Structures

The long-term objective of this study is to take data from multiple structures into account as this can provide more comprehensive feature information, such as fundamental knowledge, clinical examination, physiological indicators, imaging data, etc., that can be used to predict LRDs disease. The data can more accurately reflect both the disease's progression and the patient's physical state. In addition, by combining deep learning and traditional machine learning techniques, collecting feature information from various levels, and improving the model's accuracy and reliability, more sophisticated prediction models can be created using data from various structures.

Medical Data. 2022 14th 14th International Conference on Software, Knowledge, Information Management and Applications (SKIMA). IEEE, 2022.

[Publishing]

  les maladies coronariennes et autres maladies cardiovasculaires, les accidents vasculaires cérébraux et autres maladies cérébrovasculaires, le diabète et plusieurs tumeurs malignes font partie de ces maladies. Toutes ces conditions constituent une menace majeure pour la vie et la santé des personnes et sont difficiles à traiter avec la technologie médicale actuelle. Dans ce contexte, la prévention des maladies liées au mode de vie est extrêmement importante. La prévention des maladies facilite la détection précoce pour améliorer les chances de résultats positifs pour la santé. Par conséquent, cette étude vise à proposer un cadre de prédiction des maladies liées au mode de vie basé surl'imputation des valeurs manquantes et l'ensemble la méthode ensembliste. Plus précisément, l'application des technologies de l'information dans le domaine médical produit une grande quantité de données médicales. Cependant, à cause de certaines situations de la collecte de données, comme le retrait précoce et le refus des participants, il y a beaucoup de valeurs manquantes dans les données médicales. Nous avons proposé une méthode d'imputation basée sur la technologie de suréchantillonnage SMOTE-NC et la méthode ALWRF pour les données déséquilibrées et de type mixte, appelée SncALWRFI. Pendant ce temps, l'optimisation bayésienne et la validation croisée sont utilisées pour rechercher les paramètres optimaux. Dans l'imputation des valeurs manquantes, le SncALWRFI présente une meilleure précision d'imputation et réalise iv une efficacité d'imputation élevée pour l'ensemble des bases de données publiques avec des caractéristiques de déséquilibre et de type mixe. Étant donné que les performances de prédiction peuvent être facilement impactées par la présence de bruit dans les données, nous devons rechercher une bonne stratégie pour améliorer cette situation. Le bruit peut provenir de vrais patients et il ne peut être supprimé directement.Les approches d'ensemble sont un excellent moyen de réduire la variation, le biais et le bruit. Par conséquent, afin d'augmenter les performances de prédiction des maladies liées au mode de vie, nous utilisons la technologie d'approche ensembliste dans notre étude pour confronter au bruit des données. Plus précisément, afin de maximiser simultanément la diversité et la précision des modèles d'ensemble, nous avons proposé un algorithme multi-objectif de sélection itérative de modèles (MoItMS). Les données ont été obtenues à partir de l'enquête nationale sur la santé et la nutrition de 2007 à 2018. Notre étude a utilisé un ensemble de données déséquilibrées de 11 341 personnes avec (67,16%) personnes non hypertendues et (32,84%) patients hypertendus. Les résultats indiquent une sensibilité de 51,41 %, une spécificité de 70,48 %, une précision de 76,62 % et une AUC mesurée à 0,84, ce qui a surpassé 12 modèles individuels et d'ensemble. Ce modèle peut être mis en oeuvre dans des applications pour aider les programmes de santé publique à identifier les patients présentant un risque élevé de développer une hypertension.

3. 8

 8 The experiment flow of simulation missing values for imputation error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.9 The overall block diagram of the imputation effectiveness experiment flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xv 4 . 1 133 xviiof Tables 1 . 1 4 . 9

 411331149 The categories of ensemble models and representative models . . . 4.2 The framework of the proposed ensemble model . . . . . . . . . . 4.3 Distribution of hypertension for features . . . . . . . . . . . . . . 4.4 Comparison between default and optimized parameters . . . . . . 4.5 The procedure of model selection based on the accuracy (E) and diversity (D) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.6 The objective values for models . . . . . . . . . . . . . . . . . . . 4.7 The ensemble model's AUC values for each weight . . . . . . . . . 4.8 The ensemble model's AUC values for each iteration . . . . . . . . 4.9 Boxplot of percentage precision . . . . . . . . . . . . . . . . . . . 4.10 Boxplot of percentage recall . . . . . . . . . . . . . . . . . . . . . 4.11 Boxplot of percentage accuracy . . . . . . . . . . . . . . . . . . . 4.12 Boxplot of percentage F1-measure . . . . . . . . . . . . . . . . . .4.13 Boxplot of percentage AUCs . . . . . . . . . . . . . . . . . . . . . 5.1 Missing rate of features in the case study . . . . . . . . . . . . . . 5.2 Segmented statistics of the missing rate of instances in the case study118 5.3 Distribution of missing values in the case study . . . . . . . . . . 5.4 Hot map of missing values in the case study . . . . . . . . . . . . 5.5 Imbalance rate analysis of categorical features in the case study . 5.6 Ranking of feature importance in the case study . . . . . . . . . . 5.7 The generation process of the ensemble model . . . . . . . . . . . 5.8 Ensemble model evaluation . . . . . . . . . . . . . . . . . . . . . . 5.9 The data flow diagram of the proposed prediction framework . . . 5.10 The data for 10 people in the simple application scenario . . . . . 5.11 The execution result of feature selection module in the simple application scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi 5.12 The result of missing value module in the simple application scenario131 5.13 The imputed data in the simple application scenario . . . . . . . . 132 5.14 The prediction results in the simple application scenario . . . . . 132 5.15 The features' contribution for hypertension of two persons . . . . List Studied papers about LRDs prediction. . . . . . . . . . . . . . . . 2.1 The three missing value processing methods and their advantages and disadvantages. . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Three feature selection categories and their advantages and disadvantages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Two types of disease prediction methods and their advantages and disadvantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Example of nearest neighbor computation for SMOTE-NC. . . . 3.2 The data information for the ALWRF classification experiment . . 3.3 The data information for the ALWRF regression experiment . . . 3.4 The information of five public datasets. . . . . . . . . . . . . . . 3.5 The experiment results of imputation errors. . . . . . . . . . . . . 3.6 The information of six public medical datasets. . . . . . . . . . . 3.7 The information of datasets with real missing values. . . . . . . . 4.1 Number of people by hypertension category, gender and ethnicity. 4.2 Selected categorical variables in the NHANES dataset . . . . . . . 4.3 Selected numerical variables in the NHANES dataset . . . . . . . 4.4 The introduction of six performance indicators . . . . . . . . . . . 4.5 Hyperparameter space for models . . . . . . . . . . . . . . . . . . 4.6 Performance comparison with individual classifiers . . . . . . . . . 4.7 Performance comparison with other ensembles . . . . . . . . . . . 4.8 Classification Report . . . . . . . . . . . . . . . . . . . . . . . . . xviii Classification methods comparison . . . . . . . . . . . . . . . . . . 4.10 Comparing the impact of lifestyle factors in hypertension prediction 5.1 Prediction results of different processing methods for missing values in the case study . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Prediction results of feature selection in the case study . . . . . .

Figure 1 . 1 :

 11 Figure 1.1: Chapters organization

  ings. The missing value module, feature selection module, and disease prediction module are the three key components of this framework. The method of combining deletion and imputation is chosen as the primary strategy for missing value processing for the significant number of missing values in the data set gathered from lifestyle-related diseases first. The feature selection module employs machine learning-based feature selection to discover key features for lifestyle-related diseases since different lifestyle-related diseases have distinct important features.In order to create a strong ensemble prediction model for lifestyle-related diseases and achieve a more accurate prediction of lifestyle-related diseases, the data processed by the missing value module and the feature selection module are used as the input of the prediction model.

Figure 2 .

 2 1 is a diagram of the proposed prediction framework for LRDs.

Figure 2 . 1 :

 21 Figure 2.1: LRDs prediction framework

  module. Specifically, the disease prediction module includes visualization of the development of forecasting models, evaluation of the models, and interpretation of forecasting results. Visualization of model development can better explain the prediction process of lifestyle-related diseases. The model's evaluation is also crucial because it defines how usable the final model will be. The evaluation index provides a quantitative index of the quality of the algorithm or parameters and is designed to input the same data into several algorithm models or the same algorithm model with varied parameters. It is frequently important to employ a variety of various indications while evaluating a model. The majority of the numerous evaluation indicators can only indicate a portion of the model's performance. If the evaluation indicators are not used properly, flaws with the model itself cannot be detected, which will result in incorrect inferences. The interpretation of prediction results can provide people with rich information. Chapter 3. A Missing Value Imputation Approach for Imbalance and Mixed-Type Data 3.1 Methodology of the Proposed Imputation Method 3.1.1 Adaptive Laplacian Weight Random Forest (ALWRF) Method

10 )Algorithm 1 Step 1 :

 1011 Based on the previous introduction, the proposed adaptive Laplacian weighted random forest (ALWRF) is shown in algorithm 1. The adaptive Laplacian weighted random forest (ALWRF) Input: D: A data set with n rows and d columns; m: The number of trees; γ: The interval to update the weights Output: ALW RF L = { ℓ 1 , ℓ 2 , ..., ℓ d } ← The normalized Laplacian weighted for i = 0 to m do V = {ν 1 , ν 2 , ...ν d } ← The number of selected times for features DT ← ∅ while True do if DT meets conditions then break end D i ← Sampling m times with replacement from D D i oob ← The Corresponding out-of-bag data W c ← W random ← A random number in the range (0,1) if random>ϵ then W ′ c ← Update weights by equation (3.10) F sub ← Weighted random sampling of feature subsets using W ′ c else F sub ← Weighted random sampling of feature subsets using W c end f j ← Select the optimal splitting feature using D i DT ← Generate branches and update DT ν j ← ν j + 1 end ALW RF ← ALW RF ∪ DT if len(ALW RF ) mod γ equals 0 then I = { ι 1 , ι 2 , ..., ι d } ← Calculate the normalized importance of features using D oob W ← Update weights by equation (3.8) end end On the other hand, the training and testing process for the proposed AL-WRF is similar to a random forest. Five steps involved in the ALWRF: The dataset(n×d) is divided into training data (n 1 ×d) and testing data (n 2 × d), where n 1 + n 2 = n.

Figure 3 . 1 :

 31 Figure 3.1: The training and testing process for adaptive Laplacian weighted random forest

. 52 Algorithm 2 ←

 522 f i ), and then use it to impute missing values at the f i feature in D i I . Step 6. Repeat steps 3 to 5 until all features with missing values are traversed. The proposed SncALWRFI method for missing values is shown in algorithm 2The proposed imputation method: SncALWRFI D: A data set with missing values D: Data set has been imputed M ← Calculate indicator matrix D ′ ← Using mean or mode values as an initial imputation F ← The sorted feature set by missing rate in descending order for f i ∈ F do D i C , D i I ← Divide dataset according to M ; // optimal parameter for Cross-validation do D training , D testing ← Randomly generate missing values in D i C for n,m,s,γ,knn,irt do if f i is categorical and ir > irt then D training ← Use SMOTE-NC to oversample and update D training end F fi ← Build ALWRF for the feature f i loss ← Use D testing to compute the loss value of F fi Using optimal parameters to build the model D i I ← Use F optimal fi to impute and update D i I D ← Update D using D i I end

  samples. Compared with traditional optimization methods, it does not need the explicit expression of the function. Therefore, Bayesian optimization is employed to search best parameters in our work. In our work, the goal is to improve the predictive performance of the proposed model on both classification and regression tasks where the optimization functions are different. In the classification task, the output can be two or more classes. Therefore, a Confusion Matrix with four different combinations of predicted and actual values commonly used to evaluate classifier performance, as shown in Figure 3.2.

Figure 3 . 2 :

 32 Figure 3.2: Confusion Matrix

Firstly,

  Bayesian optimization resorts to tuning hyper-parameters for AL-WRF. The optimization process is similar to a random forest. The hyperparameters include the number of decision trees in the random forest n, the size of the predictor variables subset m, minimum sample split s, and the interval for updating weights γ. The default values of hyperparameters are n = 100, m = √ M (M is the number of predictor variables), s = 2. The ranges of value for hyperparameters are n ∈ Range(50, 500, 50), m ∈ (0.1, 0.999), s ∈ [2, 25] and γ ∈ {10, 20, 30, 40}, respectively. Here, m is a fraction and it means that m percentage features are considered at each split. Based on the analysis of the 3.1 section, the model prediction accuracy and negative MSE on the test set are chosen as optimization functions. Specifically, the Bayesian optimization process for ALWRF works as follows: Step 1. Select five sample points randomly in the hyperparameters space and calculate the prediction accuracy or negative MSE of the ALWRF. The five samples are used as the training set; Step 2. Obtain a new sample point by optimizing the acquisition function and calculating the acquisition function value at the new sample point; Step 3. Add the new sample point into the training set and update the posterior distribution of the function;

Figure 3

 3 Figure 3.3.

Figure 3 . 3 :

 33 Figure 3.3: The bayesian optimization process for the ALWRF.

Figure 3 . 4 .In Figure 3 . 4 ,

 3434 Figure 3.4.In Figure3.4, the distribution of accuracy values for RF, BO-RF, and BO-ALWRF is presented. The results showed that the median of RF was the lowest in all datasets. After hyperparameter optimization, the accuracy values increased, especially in the NHANES dataset. As we expected, BO-ALWRF showed the

Figure 3 . 5 :

 35 Figure 3.5: The AUC-ROC curve in the ALWRF classification experiment

(

  coefficient of determination) are used as performance indicators. r 2 represents the proportion of variance that has been explained by the independent variables in the model and provides an indication of goodness of fit and therefore a measure of how well-unseen samples are likely to be predicted by the model, through the proportion of explained variance.

a

  testing data set and a training data set. As we used two types of experiment data including complete data and incomplete data, we then need to introduce missing values in the complete training dataset. Next, deletion and imputation techniques are employed to deal with missing values in both the training dataset and the testing dataset. Further, some popular classifiers are applied to each complete data set and thereby build prediction models. Finally, for each prediction model, we calculate its prediction accuracy by applying it to the testing data set.

Figure 3 . 9 :

 39 Figure 3.9: The overall block diagram of the imputation effectiveness experiment flow

Chapter 4 . 4 . 1

 441 A Stacking-Based Ensemble Approach for Noise Data Methodology of the Proposed Stacking-Based Approach It is notoriously difficult to utilize an individual model due to its unidirectionality, domain unity, and inherent quality. In addition, it is challenging

) 3 . 3

 33 All candidate models M = {m 1 , m 2 , ..., m s } is an ensemble model, and then the cost function of each candidate model in this ensemble model is calculated according to equations (1)-(4). The model m 1 with smallest cost function is selected and add into M = m 1 and it is removed from the candidate models M = m 1 , m 2 , ..., m s-1 . and G is the generation number. Specifically, the proposed MoItMS is shown in Algorithm Algorithm Multi-objective Iterative Model Selection (MoItMS) algorithm Input: Data set D = (X, Y ), Candidate models M = {m 1 , m 2 , ..., m s }, the threshold β, the weight λ Output: Selected models M Selected models M = {m 1 , m 2 , ..., m k } for the ensemble model Selected models M ← ∅; The improved performance p ← 0 Using 5 cross-validations to train each candidate model

  The predicted probabilities of all candidate models in validation datasets E ← Calculating error of candidate models based on Y ′ and Y by equation (2) m best ← The model with the smallest error value p ← The performance of m best on 5 cross-validation M ← M ∪ m best while p > β do accuracy E ← ∅, diversity D ← ∅ for each m i not in M do E i , D i ← Computing error and diversity of m i when m i and M form an ensemble model E ← E ∪ E i , D ← D ∪ D i end E ′ , D ′ ← Normalized E, D by equation (5) C ← Calculating cost values using E ′ , D ′ by equation (1) m best ← The model with the smallest cost value M ← M ∪ m best , S M ← stacking M p ′ ← Evaluating the ensemble model S M using 5 cross-validation p ← (p ′p) end M remove the last model

  Evaluation on A National Health Dataset 4.2.1 Dataset Introduction In order to provide complete access, we used the National Health and Nutrition Examination Survey (NHANES) datasets that were generated and published by the Centers for Disease Control and Prevention (CDC). The dataset includes information on human population statistics (i.e., age and gender), as well as data from examination (i.e., blood pressure and body measures), and questionnaires in terms of disease condition and healthy habits. From 2007 to 2018, there are six folders containing PDF files with NHANES response rate data and SAS Transport files for each of the investigation measurement factors. Following importing the primitive datasets into Python, data extraction and processing was essential to identify and classified variables. We generated a Github repository including the original NHANES files, and the final dataset applied for constructing and evaluating the model. The prediction model was trained and evaluated using data from the National Health and Nutrition Examination Survey (NHANES), which was gathered between 2007 and 2018. The purpose of developing this model was to evaluate the disease risk of hypertension using relevant risk factors in a representative sample of American adults aged 20 and older (n = 11,341).

Figure 4

 4 Figure 4.3.

Figure 4 . 3 :

 43 Figure 4.3: Distribution of hypertension for features

  Identifies the proportion of hypertension samples that have been correctly classified.Accuracy T P +T N T P +T N +F P +F NCalculates the overall proportion of samples that have been successfully categorized.F1-measure2P recisionRecall P recision+RecallThe harmonic average of the value of recall and precision.

AUC

  utilized the Standard Scaler approach to normalize the dataset first because KNN and SVM are easily affected by feature scale. All experiments were simulated on a machine with Intel(R) Core (TM) i5-8265U CPU @ 1.60GHz 1.80 GHz, 8 GB RAM., Windows 10 64-bit O.S., and Python 3.8.6 environment.

Figure 4 . 4 : 4 . 2 . 5

 44425 Figure 4.4: Comparison between default and optimized parameters

4. 4 Summary

 4 Various categorization algorithms for the early detection of lifestyle-related diseases have been presented in recent years. One of the current study areas is selecting an acceptable methodology that strikes a compromise between efficiency and implementation complexity. According to the reports of the National Health and Nutrition Examination Survey (NHANES), the prevalence of hypertension in the adult population of the United States is high and has been rising over the past few years. We initially devised a Multi-objective Iterative Model Selection (MoItMS) strategy to simultaneously maximize the ensemble model diversity and the accuracy of meta-learners in this work. Subsequently, a stacking-based aggregative method for accurately classifying the data of hypertension patients was created. The proposed model uses three distinct types of learners namely, KNN, SVM, and MLP, as its basic learners. Each of these models is trained using cross-validation to ensure accuracy. The level-1 data is comprised of the predictions made on training samples in addition to the actual labels, both of which are

Chapter 5 .

 5 A Case Study for A Lifestyle-Related Disease 5.1 Data Source This study used real medical data gathered during a hospital health checkup in Nanjing, China. This dataset is from 2012 to 2022. All subjects in the study gave informed consent to the use of the data, and all sensitive information about the subjects was removed from the original dataset. In this real case study, hypertension is an example of a lifestyle-related disease because it is really common in our daily life. First, we removed 23 records who were 20 years of age or younger. The remaining data comprised 32,784 instances and 65 attributes. Specifically, there are 41 features including age, gender, heart rate (HR), height, weight, waist circumference (WC), body massive index (BMI), hemoglobin (HB), white blood cell (WBC), platelets (PL), urinary protein (UP), Urinary sugar (US), Urinary ketones (UK), Urinary occult blood (UOB), blood sugar (BS), alanine aminotransferase (ALT), aspartate transaminase (AT), Total Bilirubin (TB), Creatinine (CR), BU (Blood Urea), Total cholesterol (TC), triglycerides (TG), high density lipoprotein cholesterol (HDL-C), low density lipoprotein cholesterol (LDL-C), right systolic blood pressure (right SBP), right diastolic blood pressure (right DBP), left systolic blood pressure (left SBP), left diastolic blood pressure (left DBP), exercise frequency (L EF), exercise year(L EY), exercise time

Figure 5 . 1 :

 51 Figure 5.1: Missing rate of features in the case study

Figure 5 . 2 :Figure 5 . 2

 5252 Figure 5.2: Segmented statistics of the missing rate of instances in the case study

Figure 5 . 3 :

 53 Figure 5.3: Distribution of missing values in the case study

Figure 5 . 4 :

 54 Figure 5.4: Hot map of missing values in the case study

Figure 5 . 5 :

 55 Figure 5.5: Imbalance rate analysis of categorical features in the case study

  input, followed by the use of RF FS to analyze the importance of features, and finally the selection of the data set containing only key features in accordance with the ranking of feature importance. A predictive model for LRDs was cre-ated using an experimental dataset. Initially, there were 65 features in our case, but since 4 of them (L SQ, L SA, L DQ, and L DA) were 80% absent from the dataset, they were excluded and the remaining 61 features were input into the feature selection module. When calculating feature importance, the result will be rounded to 3 decimal places. The final output of RF FS is shown in Figure5.6.

Figure 5 . 6 :

 56 Figure 5.6: Ranking of feature importance in the case study

  be utilized to create a strong ensemble LRD predictive model. The final LRDs prediction model will be combined from candidate models including multilayer perceptron (MLP), K-Nearest Neighbors (KNN), Decision Tree (DT), support vector machine (SVM), Gaussian Naive Bayes Network (Gaussian NB), Logistic Regression Model (LRM) model, Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM) and Random Forest (RF). Three steps make up the model construction: ensemble model construction, hyperparameter optimization, and model evaluation. The disease prediction module will first model's prediction by a value of 0.06. SHAP can assist doctors to understand the prediction results of the model, rather than providing a black box to doctors.

6. 1

 1 ConclusionsLifestyle-related diseases are the conclusions drawn by developed countries after conducting a large number of epidemiological investigations on chronic non-communicable diseases. One main cause of these chronic non-communicable diseases is people's unhealthy lifestyles. These diseases include obesity, hypertension, coronary heart disease, other cardiovascular diseases, stroke, and other cerebrovascular diseases, diabetes, and some malignant tumors. These diseases are difficult to cure even with modern medicine, and seriously endanger people's lives and health. Now, healthcare has been digitized and generated massive new datasets. These include electronic medical record (EMR) systems, health declaration data, radiology images, and lab results. Health service providers can propose different approaches to predictive analysis of medical diagnosis, predictive modeling of health risks, and even prescription analysis of precision medicine by combining data from different sources. Among them, disease prediction has emerged as a crucial component of any strategy for health analysis. By predicting the occurrence of diseases, it aids medical facilities in improving patient care and lowering expenditures. The development of evidence-based best practices and aiding in the identification of people at risk for lifestyle-related diseases are two areas where disease prediction has enormous potential. This makes it possible for data to assist clinicians in staying one step ahead and offering patients proactive care before their health issues become serious.The significant dataset noise and missing values make it challenging to use conventional machine learning methods when building LRDs prediction models utilizing medical data. Particularly, some inescapable causes, including early subject withdrawal from medical research, might quickly result in missing values in research data. Many approaches to coping with missing values have been put forth since the presence of missing values makes it more difficult to mine pertinent data. Large-scale datasets with mixed types and unbalanced features are common in the medical industry, nevertheless. Only a few approaches may be utilized for data of mixed types and unbalanced features at the same time, despite the fact that existing state-of-the-art methods can decrease imputation errors and increase the quality of missing data. In order to achieve this, we propose a novel missing value interpolation technique based on Adaptive Laplacian Weighted Random Forest (ALWRF) and SMOTE-NC oversampling technology. This method can improve Unbalanced prediction accuracy features by adaptively adjusting feature weights when building random forests.Additionally, the algorithm's robustness will be impacted by the presence of noise. However, as noise is frequently present in medical data, a lot of studies has concentrated on how to handle it. Since some of the datasets of the analyzed lifestyle-related disorders correspond to real patients, it is difficult in practice to directly remove outliers. Combining ensemble methods with algorithm-level techniques is an excellent strategy to minimize variance, bias, and noise. The performance of each model in the ensemble varies depending on the circumstance.In this method, an ensemble model partially addresses these shortcomings and outperforms each individual model on a combined basis. Therefore, an ensemble approach was used in our work to reduce data noise and increase the precision of lifestyle-related illness prediction. We propose a multi-objective iterative model selection (MoItMS) technique to maximize ensemble models' variety and accuracy at the same time. The proposed stacking-based multi-objective integration framework can offer useful data-driven methodologies to categorize patients for population health management, promote disease control, and support the detection of LRDs when applied to large clinical datasets.

Table of Contents

 of ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii R ÉSUM É . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv List of Figures 1.1 Chapters organization . . . . . . . . . . . . . . . . . . . . . . . . 2.1 LRDs prediction framework . . . . . . . . . . . . . . . . . . . . . 3.1 The training and testing process for adaptive Laplacian weighted random forest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . .

  5.3 Prediction results of each module in the case study . . . . . . . .

	MSE Mean Squared Error 55
	NB Naive Bayes Network 9, 72
	NHANES National Health and Nutrition Examination Survey 60
	RF Random Forest 9, 27, 124 Chapter 1. Introduction
	RFI Random Forest Imputation or MissForest 66 1.1 Background
	SHAP SHapley Additive exPlanations 132, 133
	SMOTE Synthetic Minority Over-sampling TEchnique 49
	SMOTE-NC Synthetic Minority Oversampling Technique for Nominal and Con-
	tinuous 29
	SncALWRFI SMOTE-NC and ALWRF Imputation 51
	SVM Support Vector Machine 9, 72
	XGBoost Extreme Gradient Boosting 10, 124
	LRDs Lifestyle-related diseases 3
	LRM Logistic Regression Model 3, 9, 98
	MLP Multi-Layer Perceptron 10, 70
	MoItMS Multi-objective Iterative Model Selection 114
	xx xxi

xix Abbreviations M SIE num Mean of Squared Imputation Errors for numerical values 57 wN N Sel mix Weighted Nearest Neighbor Imputation using Selected Variables 66 P F C cat Proportion of Falsely Imputed Categories 57 ABDT Adaptive Boosting Decision Tree 70 AdaBoost Adaptive Boosting 11 ALWRF Adaptive Laplacian Weight Random Forest 29 AUC Area under the ROC(Receiver operating characteristic) curve 3 BMI Body Mass Index 59 CART Classification and Regression Tree 40 DT Decision Tree 9, 27 Gaussian NB Gaussian Naive Bayes Network 110, 124 GBDT Gradient Boosting Decision Tree 9, 70 kNN k-Nearest Neighbors 9, 27 kNNI k-Nearest Neighbors Imputation 66 LightGBM Light Gradient Boosting Machine 12, 124 LR Linear Regression 9, 72

  The Cox proportional hazards model approach is used in this model to create an individual stroke risk model for American whites. Age,

	systolic blood pressure, hypertension, smoking, atrial fibrillation, left ventricular
	hypertrophy, and other cardiac conditions were among the factors in the model
	(i.e., myocardial infarction, congestive heart failure, coronary insufficiency, and

intermittent cardiac claudication). Additionally, several nations are actively creating and validating disease prediction models for various diseases appropriate for their particular ethnic characteristics because populations in different countries have varied disease spectrums and prevalence risk factors. For instance, the UK Prospective Diabetes Study (UKPDS)

[START_REF] Pm Clarke | A model to estimate the lifetime health outcomes of patients with type 2 diabetes: the united kingdom prospective diabetes study (ukpds) outcomes model (ukpds no. 68)[END_REF]

, Harvard Cancer Risk Assessment

Table 1 . 1 :

 11 study employed Web of Science and Google Scholar as search engines to thoroughly analyze the current research status of LRDs prediction. The search was limited to conference and journal papers published between 2013 and 2022. Studied papers about LRDs prediction.

	It is important to note that lifestyle-related diseases are a disease set including
	those diseases related to lifestyles. Since our aim was to investigate the research
	status on the prediction of LRDs diseases, the most common LRDs diseases (i.e.
	hypertension, diabetes, obesity, overweight, and coronary heart disease) were rep-
	resented for analysis. Searches were conducted with terms including lifestyle dis-
	eases (this expression was more commonly used in earlier papers), lifestyle-related
	diseases, hypertension, diabetes, obesity, coronary heart disease (CHD), and car-
	diovascular disease (CVD).
	Specifically, 45 papers are studied. Data extraction included the author's
	name, year of publication, predicted disease, type of model, and the specific
	model used. The categories of models were mainly divided into statistical models
	(SM) and machine learning models (ML). Statistical models are mainly used to
	discover correlations between variables and thus predict the output, while machine
	learning models build analytical systems by learning from data and do not rely on
	explicit rules of construction [40]. Statistical modeling is more about discovering

  .1.

	Methods	Advantage	Disadvantage	Example
	Deletion	Simple	Ignore valuable information	PD, LD
	Toleration	Learning hidden in-	Increase model complexity; spe-	DT, XGBoost
		formation	cific model structure
	Imputation	Independent of pre-	Additional computing space and	Mean, KNNI
		dictive models	time

Table 2 .

 2

1:

  .2.

	Category	Advantage	Disadvantage	Example
	Filter	High	computa-	Ignore combination effect be-	Pearson correlation coeffi-
		tional efficiency	tween features	cient, chi-square test
	Wrapper	Oriented to algo-	High complexity and easy to	Complete search, random
		rithm optimization	overfit with small samples	search
	Embedded	Automatically se-	Need to select loss functions and	Feature Selection Method
		lects features	adjust parameter	Based on Tree Model

Table 2 . 2 :

 22 Three feature selection categories and their advantages and disadvantages.2.2.3 Research Status of Disease Prediction MethodsNowadays, a lot of academics are researching disease prediction models and have developed a number of useful models. In earlier research, we investigated the state of various prevalent lifestyle diseases prediction methods (hypertension, diabetes, obesity, overweight, and coronary heart disease). There are specifically performance is poor due to the assumed prior model in some situations; the KNN model is challenging people to apply to high-dimensional and sparse data; the SVM model is also simple to manage when the number of sample features and the number of samples are close together. Two different categories of feature selection methods together with their benefits and drawbacks are shown in Table2.3.

	Category	Advantage	Disadvantage	Example
	Traditional Sta-	Strong model inter-	Modeling is based on multiple	Framingham,
	tistical Methods	pretability	assumptions; underperform in	Cox Regression
			complex data
	Machine learn-	High flexibility and	High model complexity; Poor	XGBoost, Neu-
	ing method	learning capability	interpretability	ral Network

Table 2 .

 2

3: Two types of disease prediction methods and their advantages and disadvantages

  . Although advanced methods can reduce imputation errors and improve the quality of missing data, existing methods can-not perform missing values well in data with mixed types and unbalanced characteristics. As a result, we proposed a new missing value imputation method based on the Adaptive Laplacian Weight Random Forest (ALWRF) and the Synthetic Minority Oversampling Technique for Nominal and Continuous (SMOTE-NC),

Table 3 .

 3

	Dataset	Total Samples	Total Variables	Categorical	Numerical
	Men's dataset	175	7	2	5
	Women's dataset	224	7	2	5
	Pima dataset	768	9	1	8
	NHANES dataset	24,434	8	8	0

2:

The data information for the ALWRF classification experiment

  The second dataset is related to life expectancy and it consists of 22 columns and 2938 rows which means 20 predicting variables.Both insurance cost and life expectancy datasets are available on the Kaggle website. The other two datasets are related to red and white variants of the 62 Portuguese "Vinho Verde" wine[START_REF] Cortez | Modeling wine preferences by data mining from physicochemical properties[END_REF]. The quality is the target variable and the other 11 variables are predicting variables. The information of these four datasets is shown in Table3.3.

	Dataset	Total Samples	Total Variables	Categorical	Numerical
	Insurance cost dataset	1,338	7	3	2
	Life expectancy dataset	2,838	22	2	20
	Red Wine dataset	1,599	12	0	12
	White Wine dataset	4,898	12	0	12

3.3.2 Regression Task

Further, four public datasets are used for validating the performance of the proposed ALWRF method in regression tasks. Specifically, insurance cost and dataset include 7 variables in terms of age, sex, BMI, children, smoker, region, and charges, where a charge is the target variable and it represents individual medical costs billed by health insurance.

Table 3 . 3

 33

: The data information for the ALWRF regression experiment Similarly, random forest(using default parameters) and random forest after Bayesian optimization (BO-RF) methods are employed for comparison and each method is performed 20 times. In regression tasks, MSE (equation (3.13)) and R 2

Table 3 . 4 :

 34 The information of five public datasets.

  .5.

	Dataset	MR	Total MSIE		Categorical PFC
			kN N I RF I	wN N	Proposed	kN N I RF I	wN N	Proposed
	GBSG2	10%	1.4156 1.0012 0.8524	0.8141	0.3540 0.2820 0.2140	0.1937
		20%	1.5560 1.0700 0.9492	0.8713	0.4075 0.3005 0.2540	0.2247
		30%	1.5257 1.0753 0.9411	0.9104	0.4097 0.3040 0.2490	0.2301
	Hepatitis	10%	1.3805 1.3098 1.0108	0.917	0.3912 0.3600 0.3169	0.2723
		20%	1.3782 1.3425 1.1111	1.0312	0.3962 0.3653 0.3259	0.2914
		30%	1.4429 1.3622 1.2050	1.112	0.4060 0.3690 0.3467	0.3009
	BMI	10%	0.9780 0.8623 0.7728	0.7646	0.2928 0.3469 0.2543	0.2481
		20%	1.1857 1.0588 1.0740	0.968	0.3902 0.4264 0.3475	0.3348
		30%	1.3215 1.1857 1.1731	1.1118	0.3904 0.4367 0.3575	0.3614
	Cars	10%	0.4860 0.2854 0.1735	0.1677	0.2125 0.1450 0.1250	0.1167
		20%	0.5462 0.3038 0.2058	0.2023	0.2275 0.1550 0.1403	0.1368
		30%	0.6335 0.3821 0.2368	0.2340	0.2308 0.1562 0.1430	0.1456
	Automobile 10%	0.4412 0.1756 0.1824	0.1579	0.2537 0.0981 0.0881	0.0745
		20%	0.4637 0.1872 0.1910	0.1732	0.2819 0.1081 0.0942	0.879
		30%	0.4727 0.1978 0.2060	0.1868	0.2978 0.1185 0.1050	0.941

  .6.

	Dataset	Total	Used	Total	Categorical Continuous	IRs
		Samples	Samples	Variables
	Statlog	270	270	14	7	7	[2.1, 68.5]
	Heart Fail-	299	299	13	6	7	[1.32, 2.11]
	ure
	Diabetes	520	520	17	16	1	[1.02, 4.91]
	Risk
	CMC	1,473	1,473	10	8	2	[1.89,20.43]
	Obesity	2,110	2,110	17	9	8	[1.02, 225.71]
	Cardiovascular	70,000	5,000	12	7	5	[1.0,17.6]

  .7.

		Total	Total	Categorical	Continuous	IRs	% MV
	Dataset	Samples	Vari-
			ables
	Hepatitis	155	19	13	6	[1.03,8.69]	5.39
	Cleveland	303	14	9	5	[2.06,37.75]	0.14
	Primary	339	18	17	1	[1.10,47.43]	3.69
	Tumor
	Chronic	400	25	23	2	[1.91,22.31]	10.09
	Kidney
	Thyroid	2,800	30	8	22	[1.0,199.0]	5.42
	Dataset
	Framingham	4,238	16	7	9	[1.33,168.52]	0.95

Table 3 . 7 :

 37 The information of datasets with real missing values.

Table 4 . 1 :

 41 Number of people by hypertension category, gender and ethnicity.

Table 4 .

 4 From Figure 4.3, individuals with hypertension in different cohorts have increased kidney disease, diabetic issues, and a notable relationship with unhealthy habits throughout follow-up. Although it has been shown that careful management of BMI can reduce the incidence of hypertension (López-Martínez et al., 2020), other factors such as age, race, education level, and lifestyle choices also influence the prevalence of hypertension. Meanwhile, the number of healthy and unhealthy people is imbalanced according to the last subplot. Therefore, based on the previous analysis, 19 features including age, gender, race, education level, BMI, waist, smoking, drinking, physical exercise, sleeping, diabetes, and kidney problems were chosen as input features. Table 4.2 and Table 4.3 show all the selected variables. 4 explains how the confusion matrix is used to calculate six indicators.

	Variable Code	Variable Description	Code	Description
	RIAGENDR	Gender	1	Male
			2	Female
	RIDRETH1	Race/Hispanic origin	1	Mexican American
			2	Other Hispanic
			3	Non-Hispanic White
			4	Non-Hispanic Black
			5	Other Race
	DMDEDUC2	Education level	1	Grade lower than ninth
			2	9-11th grade (Consists of 12th
				grade without a diploma)
			3	High school graduate/GED or
				equivalent
			4	University or AA degree
			5	A university degree or higher
	SMQ040	Do you currently keep smoking?	1	Yes
				Continued on next page

Table 4 . 4 :

 44 The introduction of six performance indicators

  .5.

	Model Name	Hyperparameter	Options/Range	Selected value
	MLP	hidden layer sizes	[(50,50,50),(50,100,50),(100,)]	(50,100,50)
		activation	['tanh','relu']	relu
		solver	['sgd','adam']	sgd
		alpha	['constant','adaptive']	constant
		learning rate	[0.0001,0.01,0.05,0.1]	0.1
		max iter	[*range(100,500,100)]	300
	KNN	n neighbors	['uniform','distance']	distance
		weights	[*range(1,15)]	14
	DT	splitter	['best','random']	best
		criterion	["gini","entropy"]	entropy
		max depth	[*range(1,50,5)]	5
		min samples leaf	[*range(1,15)]	11
		class weight	['balanced',None]	balanced
	SVM	kernel	['linear','poly','rbf','sigmoid']	rbf
		gamma	[0.001,0.01,0.1,1]	0.01
		C	[0.001,0.01,0.1,1,10,100,1000]	1
		class weight	['balanced',None]	None
	LRM	solver	['newton-	liblinear
			cg','lbfgs','liblinear','sag','saga']
		penalty	['l1','l2','elasticnet','none']	l2
		C	[0.001,0.01,0.1,1]	0.1
		class weight	['balanced',None]	None
		max iter	[*range(100,800,100)]	700
	RF	criterion	["gini","entropy"]	entropy
				Continued on next page

Table 4 . 6 :

 46 other methods is carried out. The results of 20 separate simulations are summarized in Table 4.6, which compares the proposed stacking ensemble model against a total of six distinct individual models. Performance comparison with individual classifiers Considering the results of Table 4.6, it is evident that in the context of

	Individual Models	Precision	Recall	Accuracy	F1-	AUC
	Name				measure
	MLP	0.5637	0.4105	0.7002	0.4733	0.7383
	KNN	0.6588	0.4994	0.7495	0.5679	0.8154
	DT	0.4820	0.7522	0.6514	0.5872	0.7254
	SVM	0.0	0.0	0.6702	0.0	0.7968
	Gaussian NB	0.4989	0.3604	0.6697	0.4182	0.6940
	LRM	0.5604	0.3600	0.6957	0.4382	0.7304
	Proposed Staking	0.7113	0.5376	0.7682	0.6105	0.8420
	accuracy, the proposed methodology achieves a maximum AUC value of 0.8425
	succeeded by individual learner KNN (0.8154). In addition, the proposed model
	showed significant improvement in the accuracy indicator (0.7682) compared with

Table 4 . 7 :

 47 Performance comparison with other ensembles

	Models	Base		Ensemble	Precision Recall	Accuracy	F1-	AUC
	Name	Learn-		Technic				measure
		ers
	RF	DT		Bagging	0.6991	0.4922	0.7626	0.5775	0.8306
	Adaboost	DT		Boosting	0.5689	0.3658	0.6993	0.4451	0.7365
	XGBoost	DT		Boosting	0.6564	0.5394	0.7549	0.5920	0.8102
	LightGBM	DT		Boosting	0.6409	0.5181	0.7453	0.5729	0.7895
	[186]	SVM, DT	Stacking	0.5508	0.4421	0.6967	0.4893	0.7335
	[164]	RF,	XG-	Stacking	0.6947	0.5108	0.7637	0.5871	0.8361
		Boost, Light-
		GBM, MLP
	Majority	KNN, SVM,	Majority	0.5582	0.3974	0.6971	0.4630	0.7359
	Voting	MLP		Voting
	Proposed	KNN, SVM,	Stacking	0.7113	0.5376	0.7682	0.6105	0.8420
	Staking	MLP

Table 4 .

 4 [START_REF] Asmat | Major applications of data mining in medical[END_REF] shows that our model (0.8420) had the best performance with AUCs, followed by the paper[START_REF] Li | An optimal stacking ensemble for remaining useful life estimation of systems under multi-operating conditions[END_REF] (0.8361). Surprisingly, Majority Voting's AUCs (0.7359) is dismal, even worse than KNN's, implying that Majority Voting is not suitable as a simple ensemble approach in our study. On the other hand, the stacking architecture that we utilized possesses a substantial benefit in the sense that it is able to learn the values that are produced by each model. In terms of recall, our model achieved the highest value possible, which was 0.5376, followed by XGBoost (0.5394). The proposed technique achieves the highest value in terms of precision, which is 0.7113. This is followed by Random Forest, which achieves 0.6991, and the paper[START_REF] Li | An optimal stacking ensemble for remaining useful life estimation of systems under multi-operating conditions[END_REF] achieves 0.6947. The specificities displayed by the pa-methods. DT and Gaussian Naive Bayes Network (Gaussian NB) are the two approaches with the lowest precision values. When looking at the distributions of recall in Figure4.10, it can be seen that DT achieved the highest recall value, followed by the proposed technique. Despite the fact that DT appeared to have the highest recall, it generated the least amount of precision and accuracy. The accuracy distributions are displayed in Figure4.11 and indicate that the suggested stacking strategy achieves a much greater accuracy when compared to the other classifiers. These algorithms, including DT, SVM, and Gaussian NB, produce lower accuracy values. The ensemble learner[START_REF] Li | An optimal stacking ensemble for remaining useful life estimation of systems under multi-operating conditions[END_REF] achieves the second-lowest accuracy of all the learners shown in this image. It is evident from the distributions of the F1-measure that are presented in Figure4.12 that the strategy that has been proposed produces the highest F1-measure value. The AdaBoost, DT, and LRM techniques, on the other hand, produce solutions with lower F1-measure values. Finally, the area under the curve (AUC) comparisons of the proposed technique and the benchmark method are shown in Figure4.13. As can be seen in this figure, the suggested method performed better than any of the other classifiers when it came to AUC. The paper[START_REF] Li | An optimal stacking ensemble for remaining useful life estimation of systems under multi-operating conditions[END_REF] came in second, which suggests that the proposed stacking model performs better than the complex model. The greater AUC is largely attributable to the aggregation of the decision-making capabilities of the chosen base learners, which are then combined with the suitable meta-learner. Therefore, in terms of predicted precision, accuracy, F1-measure, and AUC, the suggested method fared better than all of the individual and ensemble approaches. The overall positive performance of the suggested methodology may be valuable in assisting doctors in providing diagnoses that are more accurate and trustworthy, and it may have significant promise in the field of clinical hypertension diagnosis. In addition, the classification report generated by our model is included in Table4.8 for the purpose of carrying out analysis in the clinical sense. Additionally, sensitivity and specificity can be determined using 4.4 and are displayed in Table4.8 respectively.

	formance was obtained from our model, followed by the paper [164] (0.7637) and
	Random Forest (0.7626). Additionally, our method achieves better performance
	than the paper [164] with a smaller number of models and has lower complex-
	ity in the process of model selection. The proposed approach's promising and
	competitive performance results demonstrated its superiority to the conventional
	stacking approach. In conclusion, in terms of prediction performance, the sug-
	gested stacking technique surpasses both the six individual and seven ensemble
	approaches.
	Furthermore, a boxplot depicts the distribution of the data and is helpful
	in determining whether or not there are typical observations or outliers present

per

[START_REF] Singh | Stacking-based multi-objective evolutionary ensemble framework for prediction of diabetes mellitus[END_REF]

(0.5508) and Majority Voting was the most problematic (0.5582). The proposed strategy was able to obtain an average F1-measure that was 0.6105, making it the most successful method overall. In accuracy terms, the best per-

Table 4 . 8

 48

: Classification Report

Since its sensitivity is only 53.76%, the model proposed here may be ineffective as a healthcare diagnostic tool for detecting people who are genuinely hypertensive. However, the model's true negative rate (88.72%) suggests that it is successful in detecting those who are not hypertensive. We can also see that our model has a high negative predicted value of 1,990/2,526 (or 78.11%), demonstrating its suitability as a testing instrument. As well as it has provided a reference value for positive prediction in 623 out of 876 (or 71.11%), which

Table 4 . 9 :

 49 cision jungle, logistic regression, support vector machine, boosted decision tree, Bayes point machine, and artificial neural network. Among them, parameters of MLP, LRM, and Gaussian NB from the paper[START_REF] López-Martínez | Machine learning classification analysis for a hypertensive population as a function of several risk factors[END_REF], and parameters of other models are optimized by the Bayesian Optimization algorithm. The experiment results are shown in Table4.9. Classification methods comparisonThe findings of a comparison of six distinct approaches with our suggested method are presented in Table4.9. In terms of predictive precision, recall, accuracy, f1-measure, and area under the curve (AUC), we discovered that the proposed approach outperformed all other methods. Moreover, based on the f1measure, our model scored the highest attainable value, which was 0.536, followed by the Artificial Neural Network achieved 0.474. This is a significant improvement. Furthermore, our research considered lifestyle factors compared with the previous research[START_REF] López-Martínez | Machine learning classification analysis for a hypertensive population as a function of several risk factors[END_REF][START_REF] López-Martínez | An artificial neural network approach for predicting hypertension using nhanes data[END_REF]. So as to explore the effect of lifestyle factors on hypertension prediction, a sub-dataset without lifestyle factors is used. The input features are age, gender, race, education, BMI, waist, diabetes, and kidney. The experiment results are shown in Table4.10.

	Models Name	Precision	Recall	Accuracy	F1-measure	AUC
	SVM	0.59	0.464	0.737	0.464	0.759
	DJ	0.581	0.453	0.734	0.453	0.769
	BDT	0.564	0.462	0.729	0.462	0.765
	BPM	0.583	0.456	0.735	0.456	0.763
	LR	0.589	0.465	0.737	0.465	0.764
	ANN	0.578	0.474	0.732	0.474	0.770
	Proposed Stacking	0.592	0.490	0.745	0.536	0.788

Table 4 .

 4 10: Comparing the impact of lifestyle factors in hypertension prediction

	The experimental results show that after removing lifestyle characteris-
	tics, the prediction performance, including precision, accuracy, and AUC values,
	only slightly dropped, while recall and F1-measure decreased by 4.2% and 2.71%,
	respectively. As demonstrated in 4.3, the four characteristics of gender, age, edu-
	cation level, and obesity have strong discriminatory power for hypertension in our
	study. Furthermore, the model's capacity to correctly hypertensive samples is de-
	grading, as evidenced by the fall in recall terms. In practice, a model with a better
	hypertension discrimination performance is preferable. Despite the slight perfor-
	mance gain, we still suggest integrating lifestyle features in the model because
	they can improve the model's performance while also assisting in the analysis of
	the causes of the patient's condition.

Table 5 .

 5 1 displays the average outcomes.

	Methods	PD	MEAN	KNNI	MissForest	SncALWRFI
	RF	75.02	80.07	81.10	82.72	83.88
	LGBM	75.98	81.46	82.92	83.94	84.83
	LRM	71.08	72.19	71.91	72.20	73.31

Table 5 . 1 :

 51 Prediction results of different processing methods for missing values in the case study

  .2 below 123 displays the average AUC results obtained from 20 runs using various prediction models.

	Methods	RF	LGBM	LRM
	Non -Feature Selection	83.88	84.83	73.31
	Random Forest Feature Selection	84.17	85.28	73.89

Table 5 . 2 :

 52 Prediction results of feature selection in the case study

papers on hypertension prediction, 9 papers on overweight or obesity prediction, 9 papers on cardiovascular disease prediction, and only 4 studies focusing on multiple disease prediction. Specifically, Yaganteeswarudu[START_REF] Yaganteeswarudu | Multi disease prediction model by using machine learning and flask api[END_REF] proposed a system using the Flask API to predict multiple diseases including diabetes, diabetic retinopathy, heart disease, and breast cancer. This system uses different datasets to train different machine-learning models for different diseases. Rezaee M et al.

statistics-based models: Framingham, FINRISK Risk Calculator, and Cox Regression. The Framingham risk score can be used to calculate a person's 10-year cardiovascular risk, even in those without a history of heart disease. Based on the findings of the Framingham Heart Study, this risk score has been developed.Based on risk factor information and incidence tracking from researchers in the five-year FINRISK study, the FINRISK calculator was developed. Each risk factor that was taken into account while developing the risk coefficients was first evaluated for its impact on disease prevalence and mortality using multivariate analysis. For analyzing the relationship between patient survival time and one or more predictor factors, the Cox Regression model is frequently employed in medical research.On the other hand, a wide range of machine learning models, including but not limited to SVM, NB, and Neural Networks, have been employed to predict LRDs. These models use the rules to forecast unknown data after automatically analyzing the data. Different machine learning methods are suitable for different types of data[START_REF] Suthar | A Survey: Classification of Imputation Methods in Data Mining[END_REF]. For instance, Although BN does not have severe limitations on the number of samples as well as a high classifier efficiency, the prediction

Acknowledgements

The study tour in France is a dream-seeking journey for me. During my master's degree, I lived the worst period of my life, and the huge graduation pressure Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

Imputation Error

In order to evaluate the true imputation errors of imputation methods, only complete datasets are used in this experiment. If a dataset has naturally missing values, we discard incomplete rows. Specifically, missing values are then introduced into each data completely randomly at a specific level. and then imputation methods are employed to impute missing values. Finally, imputation accuracy is evaluated by comparing imputed values and real values. The experimental procedure is shown in Figure 3.8.

Generate Missing Values

Complete Data Imputation Methods

Evaluate Imputation Errors

Incomplete Data Here, three benchmarks are applied including k-Nearest Neighbors Imputation (kNNI) [START_REF] Troyanskaya | Missing value estimation methods for dna microarrays[END_REF], Random Forest Imputation or MissForest (RFI) [START_REF] Daniel | Missforest-non-parametric missing value imputation for mixed-type data[END_REF] and Weighted Nearest Neighbor Imputation using Selected Variables (wN N Sel mix ) [START_REF] Faisal | Imputation methods for high-dimensional mixed-type datasets by nearest neighbors[END_REF]. In the kNNI approach, an imputed value is obtained by taking the average of the values of k candidate samples, called neighbors, chosen based on a distance measure. In order to apply for mixed-type data, Gower's distance [START_REF] Gower | A general coefficient of similarity and some of its properties[END_REF] is employed as the distance measure. The RFI approach is applicable to categorical as well as continuous data even in the case of a high number of predictors. This approach firstly uses simple imputation like mean imputation as an initial method, (5) The selected models M are stacked, and the performance of the ensemble model is evaluated using five-fold cross-validation.

(6) The performance of previously selected models and newly selected models are compared. If the performance is improved and the difference is greater than a threshold β, then repeat steps 3 to 5. Otherwise, the last added model is pushed out.

In the proposed MoItMS approach, the threshold β=0.01 is implemented to balance accuracy and complexity, which means that when the performance increase is insufficient, we sacrifice performance and maintain complexity low.

In general, the proposed algorithm has lower computational complexity. The

Performance Evaluation

The precision, specificity, recall (sensitivity), accuracy, F1-measure, and AUC are the metrics that are applied in this research to evaluate the models that are suggested and compared. To begin, these metrics will be described with the help of a confusion matrix, which can be seen in Figure 3 After optimization, the AUC values of models using default parameters and optimized parameters are shown in Figure 4.4.

In Figure 4.4, we found that while the performance of machine learning models like KNN, DT, and SVM is significantly impacted by varying parameter values, the identifying power of MLP, RF, and LRM utilizing various hyperparameters is comparable. In conclusion, hyperparameter optimization is necessary because it helps machine learning models find better parameters to improve performance. For example, the performance of the decision tree in this figure has

Missing Value Analysis and Processing

First, the missing value module can automatically calculate the missing rate of each dimension in the dataset. First, the missing value module automatically calculates the missing rate of each dimension in the data set. Specifically, the overall missing rate in our case is 13.36%. Subsequently, the missing value module automatically analyzes the absence of the missing rate of the features in the data set, as shown in Figure 5.1.

Figure 5.1 shows that some features' missing rate exceeds the 0.8 cutoff point, which means that 80% of their values are lost. Because of this, we exclude these features, which include L SQ, L SA, L DQ, and L DA. Following that, as

Data Flow of the Prediction Framework

The data flow through the forecasting framework is then examined. In particular, the raw data will be appropriately processed in the proposed prediction framework and utilized to identify essential features and build core models, such as a missing value imputation model and an ensemble prediction model for LRDs.

The prediction framework has three primary data processing components, which we previously analyzed:

1) The original data is converted into data without missing values and available and robust imputation models for missing values in the missing value module after some features and instances are removed and the null values are filled with the proper missing value processing method.

2) The feature selection module selects crucial features for lifestyle-related diseases using advanced feature selection techniques based on machine learning and then turns the data into core experimental data for creating models for the prediction of lifestyle-related diseases.

3 other than hypertension). Assessments in practice (multidisciplinary collaboration with clinicians) will be taken into consideration in the future within the context of this study and will require human or professional analysis. This study is going to create a website for LRD prediction in order to control and prevent

LRDs. The proposed framework can assist the site in offering high-quality healthcare services. The website will include the following 7 key functions:

• User registration/login. Users need to register and log in to use the functions of the website.

• Personal information entry. Users need to enter their basic information, including name, gender, age, height, weight, blood pressure, heart rate, and other indicators.

• Disease selection. Users need to select the type of disease to be predicted, such as hypertension, diabetes, etc.

• Risk prediction. According to the information provided by the user and the type of disease selected, the website will use a predictive model to calculate the probability of the user suffering from the disease and provide corresponding suggestions.

• Health Advice. According to the information and prediction results provided by users, the website will give corresponding health advice, including diet, exercise, living habits, and so on.

LIST OF PUBLICATIONS

The contributions proposed in this thesis have been validated by the publication of two journal papers and two conference papers as shown hereunder.

Journal papers:

Conference on Bioinformatics and Bioengineering (BIBE), pages 859-864. IEEE, 2020.