David Hason
email: david.hasonrudd@uts.edu.au

Huan Huo
email: huan.huo@uts.edu.au

Md Rafiqul Islam
email: r.islam@aih.edu.au

Guandong Xu
email: guandong.xu@uts.edu.au

Churn Prediction via Multimodal Fusion Learning: Integrating Customer Financial Literacy, Voice, and Behavioral Data

Keywords: Churn prediction, multimodal learning, feature fusion, financial literacy, speech emotion recognition, customer behavior

In today's competitive landscape, businesses grapple with customer retention. Churn prediction models, although beneficial, often lack accuracy due to the reliance on a single data source. The intricate nature of human behavior and highdimensional customer data further complicate these efforts. To address these concerns, this paper proposes a multimodal fusion learning model for identifying customer churn risk levels in financial service providers. Our multimodal approach integrates customer sentiments, financial literacy (FL) level, and financial behavioral data, enabling more accurate and bias-free churn prediction models. The proposed FL model utilizes a SMOGN-COREG supervised model to gauge customer FL levels from their financial data. The baseline churn model applies an ensemble artificial neural network and oversampling techniques to predict churn propensity in high-dimensional financial data. We also incorporate a speech emotion recognition model employing a pretrained CNN-VGG16 to recognize customer emotions based on pitch, energy, and tone. To integrate these diverse features while retaining unique insights, we introduced late and hybrid fusion techniques that complementary boost coordinated multimodal colearning. Robust metrics were utilized to evaluate the proposed multimodal fusion model and hence the approach's validity, including mean average precision and macro-averaged F1 score. Our novel approach demonstrates a marked improvement in churn prediction, achieving a test accuracy of 91.2%, a Mean Average Precision (MAP) score of 66, and a Macro-Averaged F1 score of 54 through the proposed hybrid fusion learning technique compared with late fusion and baseline models. Furthermore, the analysis demonstrates a positive correlation between negative emotions, low FL scores, and high-risk customers.

I. INTRODUCTION

Understanding and predicting customer churn has become paramount for long-term success in an increasingly competitive business landscape [START_REF] Chen | Building comprehensible customer churn prediction models: a multiple kernel support vector machines approach[END_REF]. Customer churn, or customer attrition, directly affects a company's revenue and growth potential, making it essential to be accurately predicted and prevented [START_REF] Lemmens | Managing churn to maximize profits[END_REF]. Furthermore, the human cognitive system is inherently susceptible to cognitive biases, such as anchoring bias, the availability heuristic, and the bandwagon effect [START_REF] Oechssler | Cognitive abilities and behavioral biases[END_REF]. These biases can significantly skew customer's feeling and decision-making towards products and services. Misguided customer interactions or suboptimal presentations of complex financial products can trigger these biases, prompting a focus on potential risks over service benefits. For financial service providers, understanding these psychological factors is crucial. It allows them to tailor product design, marketing strategies, and customer relationship management (CRM) to mitigate negative biases and churn rate. Therefore, predicting customer emotion and knowledge about products and services is not just advantageous-it's pivotal for businesses striving for success in today's highly competitive marketplace.

Customer historical data recorded by organizations is increasing exponentially, and data analytics specialists contend that the traditional customer churn analysis method, which heavily relied on transactional data and customer demographics, is no longer a robust solution. A more comprehensive approach is required to better deal with the heterogeneous data input in various modalities.

The current study aims to predict customer churn propensity by incorporating diverse modalities for the first time, to our best knowledge. The proposed approach incorporates survey analysis regarding customer financial literacy level, emotion detection from customer voices (CV), and CRM data to offer a more comprehensive representation of churn risk.

The proposed approach utilizes multimodal machine learning (ML) methods for a more holistic understanding of customer churn. This approach is characterized by considering multiple facets of the customer experience, from interactions with customer service to engagement with a product or service.

Multimodality in ML refers to the integration of two or more distinct inputs, recorded in different media formats, within a single ML model. The unique aspect of these inputs is that they cannot be unambiguously mapped into one another using an algorithm [START_REF] Baltrušaitis | Multimodal machine learning: A survey and taxonomy[END_REF]. We proposed a fusion learning to force similarities of these inputs into a coordinated feature representation space. Subsequently, a decision-level fusion is applied to rank churn risk.

A. Sentiment analysis

Customer sentiment analysis can be divided into para linguistic, using CV analysis, or linguistic or text based approaches. Recorded customer service calls can be analyzed to understand customer satisfaction. Although sentiment analysis using natural language processing (NLP) can be used to transcribe and analyze these calls; meaning is often also conveyed by tone, pitch, and loudness. Thus, humans provide and receive considerable meta-information during a conversation apart from the text. This is an inherent problem for text-based sentiment analysis in NLP, whereas non-linguistic emotion detection through speech signal processing is somewhat more robust [3]. Customers negative emotions are often good indicators of product dissatisfaction, and these customers can be considered to have higher churn risk due to their discernible discontent, warranting special attention. To decode emotion from CV, we introduced a voice-based sentiment analysis model [3]. The central innovation of our work lies in the development of a novel hybrid feature map generator algorithm using harmonic and percussive components of Mel Spectrogram that improves speech emotion recognition (SER).

B. Financial literacy

Customer financial literacy can be a crucial factor to predict customer churn within the financial services industry. Financially literate customers are generally better equipped to comprehend the range of products and services available, enabling them to better select the most suitable option(s) to meet their requirements. This reduces dissatisfaction and subsequent churn probability. In contrast, customers with low financial literacy could struggle to understand the options, and hence make sub-optimal decisions regarding financial products and services, leading to dissatisfaction and potential migration to competing financial institutions perceived to offer superior products. We introduced a SMOGN-COREG model to deal with unlabelled examples via semi-supervised regression (SSR) methods [8]. We could significantly contribute by enhancing prediction precision beyond relying on labeled data in supervised methods.

C. Multimodal modelling

The current study focused on analyzing synergistic relationships between predictors in each unimodal context, with the objective is to enhance churn prediction by utilizing a coordinated representation space that forces similarity for three input data formats: finance network, CRM, and inbound audio call data. We investigated the effectiveness for each previously proposed unimodal. Given the identified challenges and opportunities, this study introduces several significant contributions as below:

-In a first-of-its-kind approach, we proposed a multimodal hybrid fusion model that integrates distinct databases, including CRM, CV, and FL data. This enables us to add an essential layer of understanding customer behavior for churn prediction by discerning customer emotion from their vocal attributes and measuring financial literacy level from their account performance and survey data. -We built a specific coordinated feature representation space and translation matrix that reflects data similarity meaningfully to capture interactions between modalities, with the capability of adding future modalities like textual features in the proposed framework. -Identifying significant correlations between negative emotions, low FL, and increased churn risk. Thus, the proposed multimodal churn model leveraged customer segmentation by detecting negative emotions in CVs during interactions with call center operators, subsequently tailoring the response to address specific customer concerns and requirements, and gaining insights into potential dissatisfaction or frustration causes.

II. RECENT WORKS

Considerable research effort is evolving towards more sophisticated churn prediction methods to better draw advantages from data growth. Hence, multimodal churn modelling will continue to advance, providing ever more refined strategies for churn prediction by encompassing various customer experience facets for holistic customer churn understanding [START_REF] Gopal | A Survey on Customer Churn Prediction using Machine Learning and data mining Techniques in E-commerce[END_REF].

De Caigny explored the benefits from incorporating textual data into customer churn prediction (CCP) models and highlighted superior performance from convolutional neural networks (CNNs) compared with other text mining methods for CCP, achieving 89.87% accuracy. Outcomes from their study will help guide researchers regarding text mining for improved CCP [START_REF] De Caigny | Incorporating textual information in customer churn prediction models based on a convolutional neural network[END_REF].

Nhi NY and Liu proposed predicting client churn by analyzing unstructured data, e.g. audio calls, in machine learning models. Their approach expanded traditional methods, which tend to focus on structured data, such as demographics and transactions. In contrast, their proposed model combined text mining with a gradient boosting tree algorithm to improve the client churn model predictive performance. This approach achieved significant predictive accuracy improvement, increasing at least 5% across various customer datasets [START_REF] Vo | Client churn prediction with call log analysis[END_REF].

Kimura highlighted benefits from combining boosting algorithms with hybrid resampling methods for CP, which could significantly enhance prediction effectiveness and practicality in both academic and industry settings. The challenge from imbalanced datasets was explicitly discussed, and showed that although hybrid resampling methods, including synthetic minority over-sampling technique (SMOTE), SMOTE-ENN, and SMOTE Tomek-Links, offer significant potential, they are yet to be widely adopted for CCP [START_REF] Kimura | CUSTOMER CHURN PREDICTION WITH HY-BRID RESAMPLING AND ENSEMBLE LEARNING[END_REF]. Ahn and Hwang reviewed churn prediction methodologies, highlighting their chronological progression and underscored the necessity for adaptable and flexible approaches contingent on specific data formats. They also showed that very few previous studies adopted a multimodal approach to address churn prediction [START_REF] Ahn | A survey on churn analysis in various business domains[END_REF].

As delineated some recent work in Table I from our comprehensive literature review, many studies primarily focus on using singular structured data sources, such as CRM databases, for churn prediction tasks. Conversely, there needs to be more research that employs multiple data sources for the same purpose. To the best of our knowledge, our current investigation is the first that incorporates three discrete input data sets for the churn prediction problem. Therefore, the present study undertook a gap analysis and subsequently proposed an suitable methodology for CCP. This provides significant contributions by incorporating churn predictors, such as customer positive and negative emotional feedback from audio calls, and customer financial literacy level from survey data. The proposed synergistic hybrid feature fusion techniques and subsequent studies building on that framework, will mitigate bias for CCP. III. METHODOLOGY Our methodology encompasses three modalities to analyze customer behavior: baseline customer churn, SER, and the FL model, which are elaborated in the subsequent sections.

A. Customer financial literacy modelling

Considering substantial impact from inadequate FL on customer churn, our first modality emphasizes FL in particular [8]. We developed a semi-supervised regression model, (SMOGN-COREG) to measure customer financial literacy derived from a large and unlabeled dataset. The proposed method boosts the learning process by generating synthetic samples for minority class. The SMOGN approach was applied during preprocessing and subsequently combined with non-parametric multi learner semi-supervised regression to enhance model performance, balancing the response variable distribution by increasing the number of rare but important cases in the dataset [START_REF] Branco | SMOGN: a pre-processing approach for imbalanced regression[END_REF]. The model output is FL measurement in a range between 0 and 1.

B. Emotion recognition modelling

The second unimodal leverages the voice based sentiment analysis to recognize customer positive and negative emotions during their interaction with the call center [3]. The model employs the Mel Spectrogram to provide insight into voice tone, pitch, and tempo; extracting acoustic features and percussive and harmonic components from the signal frequency domain [START_REF] Chu | Environmental sound recognition with time-frequency audio features[END_REF].

We employed a pre-trained CNN to develop a framework decipher emotions from CV, incorporating a feature map generator function to extract harmonic and percussive components via a median filter applied to the signal spectrum's axes. Two feature vectors were subsequently created by averaging these components, and computing the Mel spectrogram logarithm. Finally, we construct a two-dimensional image feature map as input for the pre-trained CNN-VGG16 network to classify emotions into binary outcomes positive and negative classes, represented as 0 and 1, respectively.

C. Baseline churn model

The third unimodal evaluates customer churn risk using demographic and financial behavior sourced from CRM platforms [10]. The proposed approach integrates recursive feature elimination (RFE), SMOTE, and ANN with Deep Learning (ANN DL) to construct the churn classifier. Thus, the proposed framework predicts potential churn within the upcoming six months using data from the prior 12-month window. A customer was deemed a 'churner' if they terminated their account within six-month prediction period, and the outcome is encapsulated in a binary variable, 0 or 1 indicates active or terminated account throughout the subsequent six months.

D. Proposed multimodal fusion learning

The proposed multimodal is able to analyze customer behavior from a diverse array of data sources highlighting complex interrelationships between different customer behavior facets. The model is designed with different components, including independent unimodal learning, feature representation space, translation or mapping function, and hybrid fusion.

Figure 1, shows how gauge customer financial literacy level, customer emotion, and churn propensity were employed to segment customers as low-risk (loyal or non-churner), Mid-risk (potential churner), and high-risk (high probability churner). For example, a high risk churn customer may exhibit low financial literacy, negative sentiment towards the company's offerings, and heightened propensity to churn.

Hybrid fusion:

The proposed model includes hybrid fusion (multi-level fusion) by incorporating early and late fusion. Hence, hybrid fusion generates input data for the baseline churn model by fusing prediction outcomes from FL and SER models for the CRM dataset.

Late fusion:

We established a prototype multimodal learning model with late fusion to evaluate the proposed hybrid fusion method efficacy compared with other methods. Early fusion data was not utilized as churn model input, with all data sources directly provided to the modalities.

E. Feature representation space and translation

Extracting specific features associated with each modality presents significant challenges due to inherent disparities in scales, units, and distributions. Therefore, the proposed multi-modal model fusion these heterogeneous features while maintaining their unique insights towards churn modeling. We propose a specific coordinated representation space that reflects data similarity meaningfully to capture interactions between modalities with the ultimate goal to understand complementary customer behavior across various touchpoints, such as call center interaction, FL survey, and historic demographic data. The proposed coordinated representation space can be expressed as

f (xi) ∼ c(xi) ∼ v(xi) (1)
where x i is the modalities and functions f , v, and c representing independent unimodal learning networks, including FL, SER, and baseline churn model into the coordinated representation space; and ∼ is similarity or coordination in the projection into this space. We employ a dictionary to map features extracted from each modality while preserving similarity and meaningful patterns. The mapping matrix is represented through several logical propositions defined by experts, each corresponding to different condition(s) on the unimodal features. Each unimodal output data structure is mapped into two level numeric nominal variables based on a predetermined prediction confidence P (pred) threshold, defining a coordinated representation space by logical proposition(s). Constant weights are assigned to each matrix array C i , F i and V i depending on the unimodal feature weight in multimodal learning, which can be expressed as,

∀xi, P 1(xi) : f (xi) < P (pred) =⇒ Fi = 1; ¬P 1(xi) =⇒ 0 ∀xi, P 2(xi) : c(xi) ≤ P (pred) =⇒ Ci = 0; ¬P 2(xi) =⇒ 2 ∀xi, P 3(xi) : v(xi) ∈ {'Happiness', 'Neutral'} =⇒ Vi = 0; ¬P 3(xi) : v(xi) ∈ {'Sadness', 'Anger'} =⇒ Vi = 1
where P 1(x i ), P 2(x i ), and P 3(x i ) denote propositions for each unimodal feature, respectively. Estimating suitable weights for each feature indicator is challenging. This study assigned weights to the indicators guided by expert knowledge regarding churn modeling and superannuation fund business intricacies. Therefore, implicit feature translation alignment, often incorporated as an intermediate step in mapping data, is imperative for any subsequent task.

F. Hybrid fusion strategy

The proposed approach employed hybrid fusion to leverage complementary knowledge from unimodal representations while ensuring logical coherence. This incorporates model and decision-level fusion, creating a multi-level fusion framework. We enforced constant fusion weight C i + F i + V i ≥ 0 and F i = V i by assuming fusion weights for FL and SER models had equal influential contribution to enhance baseline model performance. Therefore, a decision fusion D i was defined to integrate complementary information and hence rank the churn risk as low, mid, or high. The proposed decision fusion matrix D i can be expressed as

Di =   Ilowrisk Imidrisk Ihighrisk   = I *     C1 + F1 + V1 C2 + F2 + V2 . . . Ci + Fi + Vi     (2)
subject to fusion conditions

• Low risk Churner:

Ilowrisk = I(Di = 0) • I(Ci = 0) • I(Fi = 0) • I(Vi = 0) +I(Di = 1) • I(Ci = 0) • I(Fi = 1) • I(Vi = 0) +I(Di = 1) • I(Ci = 0) • I(Fi = 0) • I(Vi = 1)
If I lowrisk = 1, customer is classified as low risk. • Mid risk churner:

Imidrisk = I(Di = 2) • I(Ci = 2) • I(Fi = 0) • I(Vi = 0) +I(Di = 2) • I(Ci = 0) • I(Fi = 1) • I(Vi = 1)
If I midrisk = 1, customer is classified as mid risk. • High risk churner:

Ihighrisk = I(Di = 4) • I(Ci = 2) • I(Fi = 1) • I(Vi = 1) +I(Di = 3) • I(Ci = 2) • I(Fi = 1) • I(Vi = 0) +I(Di = 3) • I(Ci = 2) • I(Fi = 0) • I(Vi = 1)
If I highrisk = 1, customer is classified as high risk.

Where D i ∈ {0, 4} is a fusion output weight that embodies ranked risk for each member i. C i , represents the fundamental customer churn understanding based on historical patterns; F i is quantitative financial behavior understanding, i.e., customer FL; V i is a qualitative measure, reflecting customer emotional state and satisfaction level derived from their phone communication with call center; I(•) is an indicator function where I(•) = 1 if the condition in the brackets is true and 0 otherwise. Thus, the logical operators are designed such that exactly one of I lowrisk , I midrisk , and I highrisk should = 1 for a given customer risk level query. This ensures each customer is classified into exactly one risk category.

The synergistic integration occurs when these modalities fuse distinct learnings to generate a multimodal co-learning allows a coordinated representation of two modalities FL and SER influence complementary the training of another. Thus, this co-learning strategy mitigates the limitations of relying on a single source of information, thereby enhancing the model's robustness. We employed real-world data from multiple sources to evaluate the proposed approach efficiency, including customer historic data from finance networks, demographic data from CRM platforms, and voice data, as shown in Table II.

Customer voices were replaced with similar voice from standard emotion databases to maintain privacy while preserving emotional content in the recorded voice files from inbound calls. The proposed method built a CV database leveraging the correlation between negative emotions and high risk churn customers with low financial literacy, as well as the association between positive emotions and low risk churn customers. The EMODB database, a standardized resource for categorizing emotions based on voice recordings, was utilized to label customer emotions [START_REF] Gopal | A Survey on Customer Churn Prediction using Machine Learning and data mining Techniques in E-commerce[END_REF]. The FL database was built by combining financial network data and an online customer survey to measure FL level. After initial data processing, training data was enhanced to (4154, 140). Table II shows the various data sources and their specification.

B. Evaluation criteria

The mean average precision (MAP) metric was used to objectively evaluate the proposed method solving quality for churn ranking,

MAP = 1 Q Q q=1 1 mq n k=1 P (k) • rel(k) (3)
where Q is the total number of queries; q is the current query, m q is the relevant churn risk level for the q-th query; P (k) is the precision at cut-off k in the list; rel(k) is an indicator function such rel(k) = 1 if the churn risk at rank k is a relevant document, and rel(k) = 0 otherwise. We calculated MAP by taking the mean of average precision (AP) across all risk levels (low, mid, high) for queries or instances. MAP = 1 means the system can find all the churn risk levels accurately. The AP was computed for an individual customer risk rank query as follows.

-Calculate the precision for each relevant risk rank retrieved, i.e., the proportion of relevant retrieved risk ranks up to that point, using constant weight for each risk rank. -Calculate the mean AP across all queries. This provides an overall measure for system performance across a risk rank query set.

We also employed the macro-averaged F1 score (MA-F1), since the datasets were typically considerably imbalanced. It was imperative to evaluate smaller class performance with equivalent weighting to larger classes, and F1 score can be calculated independently for each class,

Macro-Averaged F1 Score = 1 N N i=1 F 1 Scorei (4)
where F 1 -score i indicates F1 for the i th class and N is the number of classes.

C. Correlation analysis

Figure 2(a) shows the linear relationship between modalities and the proposed multimodal model output. Thus, the proposed financial literacy and churn unimodal exhibits approximately balanced influences on model performance; whereas the SER unimodal exhibits strong positive correlation = 0.66, confirming significant impact on the multimodal model performance.

Figure 2(b) shows weightings for low FL and negative emotions (large circles). Thus, clients who renewed their accounts annually exhibited high FL and positive feedback; whereas customers with negative feelings did not renew their accounts or exited.

D. Validation and comparison

To validate the effectiveness of the proposed method, we investigated the impact of modality integration on performance. Various evaluation metrics were used, including test accuracy, recall, F1-score, AUC, MAP, and MA-F1. The Figure 3 (a) shows the highest performance is achieved in hybrid fusion method with contribution of all modalities (FL+Churn+SER) with test accuracy of 91.2%.

Moreover, as shown in Figure 3(a) FL has the greatest impact on performance compared with SER. Figure 3(b) shows that the hybrid fusion approach improved churn risk distribution by shifting customers from low into mid and high risk classes.

Metrics a

Fusion Method None DF (without LF) HF (LF+DF) MAP % ± STD 51 ± 0.8 65 ± 0.7 66 ± 0.1 MA F1 % ± STD 47 ± 0.1 47 ± 1.1 54 ± 0.6 a A higher value implies superior result Python code for our proposed framework with further result visualization is available on our GitHub repository 1 . This resource simplifies the reproduction and enhancement of our study's experimental results.

V. DISCUSSION

This study proposed a multimodal fusion learning model for predicting customer churn. To the authors' best knowledge, this for the first time an attempt to integrate diverse modalities including customer's voice (CV), financial literacy (FL) survey, and CRM record data to predict churn risk at three levels (low, mid, and high). These modalities provide distinct insights, thereby contributing to improved churn prediction precision.

The FL model utilized a SMOGN-COREG supervised model to infer customer FL from their financial behavior data. The baseline churn model, developed on a robust SMOTE and ensemble ANN algorithm combination, accurately predicted churn propensity in the context of massive and high dimensionality data. The proposed SER model employed a pre-trained CNN-VGG16 model to discern customer emotions from their vocal attributes, providing an additional layer to help understand customer behavior. Our findings highlight significant correlations between negative emotions, low FL, and increased churn risk. Furthermore, comparative analysis underscored the superiority of the hybrid fusion technique, achieving a MAP score of 66 and a test accuracy of 91.2%, thus outperforming both non-fusion and late fusion approaches.

VI. CONCLUSION AND FUTURE WORKS

Despite the promising results, the proposed approach has limitations; in particular, the coordinated representation in the multimodal fusion method could lose valuable intermodal information. Future study will explore a joint representation approach, concatenating features from different modalities before commencing learning, providing the model with a broader, richer data set. This could enhance the learning process efficacy, particularly when employing a powerful model such as Deep ANN.

The second area of potential future work involves extending the multimodal approach by integrating textual features as a fourth modality. It can be represented as t(x i ) a textual learning unimodal and embedded into existing coordinated representation space which is expressed in equation ( 1). This would broaden the model's scope and enhance the multifaceted analyses of customer behavior.

Fig. 1 .

 1 Fig. 1. Proposed multimodal hybrid fusion learning method workflow to integrate various modalities

Fig. 2 .

 2 Fig. 2. a) Strong positive relationship between customer emotion recognition and multimodal learning performance. b) Relation of customer emotion feedback and financial literacy level with customer tenure.

Fig. 3 .

 3 Fig. 3. (a) Multimodal model performance is significantly improved by combining early and late fusion (i.e., hybrid fusion) considering all features; (b) Risk levels for various multimodal fusion methods, more customers identified as mid and high risk using the proposed hybrid fusion method.

Figure 4

 4 Figure 4 compares ROC curves for late and hybrid fusion. Higher AUC in Fig. 4 (b) confirms the advantage of integrating multiple modalities, leveraging multi-level fusion benefits.

Fig. 4 .

 4 Fig. 4. Hybrid fusion exhibits higher AUC than other methods considered.

Table III

 III

compares risk identification accuracy using MAP and MA F1 metrics. Multimodal learning with hybrid fusion strategy achieved significantly improved results; MAP score = 66 and MA-F1 = 54.

TABLE III METRICS

 III FOR THE DIFFERENT FUSION METHODS CONSIDERED.

https://github.com/DavidHason/multimodal churn model

ACKNOWLEDGMENT

This work is partially supported by the Australian Research Council under grant number: DP22010371, LE220100078, DP200101374 and LP170100891


Bekkouri
email: hind.bekkouri@supergrid-institute.com

Abdelkrim Benchaib

Florent Morel

Pierre Rault

Juan- Carlos Gonzalez-Torres

Florian Errigo

Energy-based and non-energy-based control strategies for modular multilevel converters with embedded energy storage

Keywords: «Modular Multilevel Converters (MMC)», «Energy storage», «Control strategy», «Modelling», «Grid-connected converter»

This paper deals with a high voltage direct current (HVDC) system based on modular multilevel converter (MMC) with embedded energy storage capacity. The dynamic behavior of energy-based (EBC) and non-energy-based (NEBC) controls is analyzed and compared for balanced and unbalanced energy storage distribution. EMTP simulations results show an equivalent performance for both controls when energy storage is balanced. While the NEBC control highlights limitations for unbalanced energy storage management, the EBC shows satisfactory overall performance.

Introduction

The integration of power electronics (PE) induces a paradigm shift in the operation and control of electrical networks. This integration is mainly due to the connection of renewable energy sources and the multiplication of interconnections through the power network. High voltage direct current (HVDC) systems are considered as the best solution for interconnecting grids and far renewable energy sources thanks to their low transmission loss and high power-transmission capacity. In addition, voltage source converters (VSCs) based on modular multilevel converter (MMC) technologies highlight useful control flexibilities to participate positively to the overall stability enhancement. As a matter fact, the replacement of synchronous machines, which provide stability to the power grid through their kinetic energy and inherently maintain the balance between electricity production and consumption leads to serious stability issues, especially in the event of a fault or disturbance. To deal with these stability issues, one solution is to add energy storage systems (ESSs) to the grid. ESSs are usually connected to the AC system through a power electronic converter but connection to an HVDC link [START_REF] Sau-Bassols | A Single branch of energy storage submodules to integrate energy storage devices in HVDC systems[END_REF] and integration in an MMC [START_REF] Errigo | A submodule with integrated supercapacitors for HVDC-MMC providing fast frequency response[END_REF] can also be considered. This work is focused on the integration of ESSs in the submodules of the MMC to provide ancillary services needed for the AC power system. The ESSs are based on batteries or supercapacitors interfaced thanks to dc/dc converters. Considering ESSs lifetime and discrepancies in aging, their distribution among the MMC submodules will become unbalanced at some point. Balanced and a sort of extreme unbalanced energy storage distributions can be implemented [START_REF] Henke | Balancing of modular multilevel converters with unbalanced integration of energy storage devices[END_REF]. Unbalanced energy storage leads to uneven constraints for the converter devices which has been investigated in [START_REF] Errigo | Assessment of the Impact of Split Storage within Modular Multilevel Converter[END_REF]. Two control philosophies of the MMC exist in the literature, namely non-energy-based (NEBC) [START_REF] Shinoda | A New Energy Management Control of Modular Multilevel Converters for Coping with Voltage Stress on Sub-Modules[END_REF] and energy-based (EBC) [START_REF] Bergna | An energy-based controller for HVDC modular multilevel converter in decoupled double synchronous reference frame for voltage oscillation reduction[END_REF] controls. Both controls require modifications for additional ESS management and results have been shown in the literature but, to the best knowledge of authors, performances, and limitations of these control approaches for MMCs with embedded unbalanced energy storage have not been compared. This is the main contribution of this paper. This paper is organized as follows. First, the MMC converter topology is described. Secondly, the associated NEBC and EBC philosophies are discussed. Thereafter, physical and control layers modifications for ESS integration in the MMC are highlighted. Then, simulation results of three use cases, for balanced and unbalanced energy storage integration, are presented and analyzed. Finally, some concluding remarks end the paper.

MMC converter topology and modeling

Each phase of the half-bridge MMC outlined in Figure 1 can be divided into upper and lower parts, which are called arms [7][8]. In order to analyze the dynamic behavior of an MMC, it is necessary to solve a large number of simultaneous differential equations which can require enormous computational effort. Hence, some simplification techniques are needed. According to [START_REF] Saad | Modular Multilevel Converter Models for Electromagnetic Transients[END_REF], for system-level simulations, each MMC arm can be represented with sufficient accuracy by an ideal DC/DC chopper associated with an equivalent capacitor. This model assumes that the voltages of all the sub-module capacitors 𝑣 𝐶i 𝑢,𝑙 are maintained within a close range by an appropriate voltage balancing control, i.e., 𝑣 𝐶1 𝑢,𝑙 = 𝑣 𝐶2 𝑢,𝑙 = ⋯ = 𝑣 𝐶𝑁 𝑢,𝑙 in each arm, and that the capacitance of the sub-modules are sized identically. The assumptions above make it possible to grasp the general behavior of the hundreds of sub-modules by considering only the sum of all the capacitor voltages in each arm, i.e., 𝑣 𝐶𝑠𝑢𝑚𝑖 𝑢,𝑙 = 𝑣 𝐶1 𝑢,𝑙 + 𝑣 𝐶2 𝑢,𝑙 + ⋯ + 𝑣 𝐶𝑁 𝑢,𝑙 , which is applied on an equivalent capacitance given by 𝐶 𝑎𝑟𝑚 = 𝐶/𝑁, where C is the capacitance of a submodule and N is the number of sub-modules in the arm. The arm average model (AAM) of the three-phase MMC is thus obtained as depicted in Figure 1 bottom right-hand side.

MMC control philosophies

Unlike classical VSC converters, the large number of SMs in HVDC-MMCs allows an almost continuous control of the valve voltage. MMC control differs then from classical VSCs and requires more controllers because the converter stores energy into the submodule capacitors, which are not directly connected to the DC terminal [START_REF] Friedrich | Modern HVDC PLUS application of VSC in Modular Multilevel Converter topology[END_REF]. Thus, the energy stored in the distributed submodules can be decoupled from the associated DC converter terminal voltage [START_REF] Delarue | Energetic macroscopic representation and inversion based control of a modular multilevel converter[END_REF]. This brings a new degree of freedom but to take advantage of it, the regulation of this energy is an important requirement for the MMC control.

In the following, NEBC and EBC of the MMC are described.

NEBC of MMC

For NEBC, since no MMC energy control is explicitly included, the regulation of the DC current essentially depends on the amount of transferred active power and DC voltage level of the MMC unit. This results in a coupling between the AC network response and the DC grid dynamics. The modulation index calculation is based on the measured DC voltage [START_REF] Bergna | An energy-based controller for HVDC modular multilevel converter in decoupled double synchronous reference frame for voltage oscillation reduction[END_REF]. The associated control loops are shown in Figure 2.

EBC of MMC

For EBC, and unlike NEBC response, a variation of the DC voltage does not necessarily translate into energy variations for the sub-module capacitors. This decoupling effect occurs since the energy in the arm is a function of the number of capacitors introduced (by the insertion index), and the DC voltage level. This double dependency has been mentioned in several studies as a beneficial feature capable of decoupling the AC network dynamics and the DC network dynamics [START_REF] Shinoda | A New Energy Management Control of Modular Multilevel Converters for Coping with Voltage Stress on Sub-Modules[END_REF]. The principles of EBC which were presented in [START_REF] Shinoda | A New Energy Management Control of Modular Multilevel Converters for Coping with Voltage Stress on Sub-Modules[END_REF] and [START_REF] Delarue | Energetic macroscopic representation and inversion based control of a modular multilevel converter[END_REF], are shown with the control loops in Figure 3.

Energy storage system in the MMC

Compared to the MMC schematic presented above, some modifications at the physical layer as well as at the control layer have been done to integrate the energy storage systems in the MMC submodules.

Sub-module

Equivalent arm

Modifications at the physical layer

For the considered MMC model, which is based on average arm models, a balanced distribution of the energy storage within one arm is considered, i.e., if the energy storage is integrated in one arm of the MMC, it is integrated in all the submodules of that arm. Figure 4 shows the topology of an MMC with embedded energy storage system in all arms and the related arm model where equivalent energy storage system is added to the MMC equivalent arm model that has been presented in Figure 1. For the energy storage elements, several technologies could be considered, e.g., supercapacitor or battery. In this paper, the energy storage elements are interfaced with the SM capacitor thanks to a dc/dc converter.

Since the focus of this study is on the MMC control, the ESS (energy storage elements and interface dc/dc converter) is reduced to a controllable current source as shown in Figure 5.

The arm ESS power reference is filtered to account for the DC/DC converter dynamics.

Modifications in the control layer

For both controls, NEBC and EBC, the modifications concern mainly the outer control loop: a supplementary power reference 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓 is added to the AC power reference of the converter. The supplementary power reference can be injected either if the converter is on 𝑉 𝑑𝑐 control or in 𝑃 control mode, see Figure 6.

𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓 is also distributed among the MMC arms where the ESS is activated, Figure 5.

Case study Benchmark description

Typically, symmetric monopole HVDC link is composed of two MMCs, one controlling the DC voltage while the second controls the active power. In this study, the energy storage is integrated in the MMC in P control. The energy storage power reference is equally shared among the arms with energy storage systems.

In simulations, the voltage controlled MMC, and the DC cable are replaced by a DC voltage source, as shown in Figure 7 to focus on the energy storage integration in the MMC. The AC grid is modeled by a Thevenin equivalent. The MMC with enhanced energy storage is simulated in EMTP. The simulation parameters used throughout this paper are detailed in Table I and Table II. In Table I, the electrostatic constant is defined as the total of energy stored in submodule capacitors divided by the rated power. The aim of these simulations is to address the performance of NEBC and EBC to handle balanced and unbalanced energy storage integration. In the following, three energy storage arrangements in the MMC are used, see Figure 8.

-Use case A: Energy storage elements are included in all the arms of the converter.

-Use case B: Energy storage elements are only included in phase B (in lower and upper arms of the leg B).

-Use case C: Energy storage elements are only included in the upper arm of phase A. In order to evaluate the performance of MMC control strategies with embedded energy storage, the ESS power and the HVDC link power reference profiles shown in Figure 9 are considered in all cases.

The energy storage power reference 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓 is defined in per-unit of the HVDC rated power. It corresponds to the total ESS power reference from the MMC. This power reference is equally shared between ESS arms according to the number of arms which have ESS embedded. Thus for use case A, the energy storage power reference for each arm of the MMC is 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓 6 ⁄ , for use case B, the energy storage power reference for the upper and lower arms of leg B is 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓 2 ⁄ and 0 for the remaining arms, while for case C the power reference of upper arm phase A is equal to 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓 and 0 for the other arms. To cover the two operating modes of the MMC and consider the charge and discharge states of the energy storage system in both modes, the power references for all simulations are explained hereafter and given in Figure 9.

-A positive step of 0.9 pu for the HVDC link power reference 𝑃 𝐷𝐶 𝑟𝑒𝑓 from (a) -NEBC, (b) -EBC t = 0.5s to 3.5s, during this phase the power 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓 goes up from 0 to 0.1 pu between t = 1s and 2s and then goes down from 0 to -0.1 pu between t = 2.5s and 3.5s. -A negative step of 0.9 pu for 𝑃 𝐷𝐶 𝑟𝑒𝑓 from t = 3.75s to 6.75s, during this phase 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓 goes up from 0 to 0.1 pu between t = 4.25s and 5.25s and then goes down from 0 to -0.1 pu between t = 6.75s and 6.75s.

In steady state, if losses are neglected, the AC power is egal to the ESS power reference plus the HVDC link power reference

Use case A: Energy storage elements are equally inserted in all the arms of the converter

Use case B: Energy storage elements are only inserted in one phase

In this use case, the storage system is unbalanced among the MMC arms, the ESS is available only in one phase of the MMC (leg B). Applying the same power reference profile, for MMC with EBC, the energy storage power is injected into the AC grid without disturbing the DC side. As shown in Figure 12, both 𝑃 𝑎𝑐 and 𝑃 𝑑𝑐 follow their respective references. However, with NEBC, power oscillations at 100Hz are observed on the AC and DC powers when the energy storage power reference is not null. To understand the effect of the unbalanced integrated energy storage on the internal dynamics of the MMC with NEBC, the Circulating Current Suppressing Control (CCSC-dq) is disabled to observe the behavior of the differential currents in open loop.

The differential currents 𝑖 𝑑𝑖𝑓𝑓 𝑖 (also named circulating currents), defined in (1), comprise the DC currents, along with the additional AC currents that circulate within the converter as a result of the interaction between the arms.

𝑖 𝑑𝑖𝑓𝑓𝑖 = (𝑖 𝑖 𝑢 + 𝑖 𝑖 𝑙 ) 2 (1)
For the following, to highlight the undesirable phenomena at the AC side of the MMC, 𝑃 𝐴𝐶 𝑟𝑒𝑓 is set to zero and 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓 is defined as previously. In this case, the energy storage power is injected to the DC side. Consequently, if active power is observed at the AC side, it is due to uncontrolled variables. As reference case, the previous case with balanced energy storage distribution is simulated with CCSC-dq disabled, which results are shown in Figure 13. As expected, when the CCSC-dq is disabled and ESS is evenly distributed, 100Hz oscillations are observed in the differential currents. ESS, in Case A, the 100Hz oscillations are also balanced between the three-phases, and their magnitude varies according to the power from the ESS due to the change in the arm currents (Figure 13). Considering Case B, the differential currents 𝑖 𝑑𝑖𝑓𝑓 𝑖 are not balanced anymore.The phase B current is much larger when the power from the ESS is not null. This explains why the classical CCSC-dq is not working properly as it is designed to control balanced currents since it is based only on the positive sequence abc-dq transformation.

To eliminate the AC component of the unbalanced differential currents, a solution based on the ideas presented in [START_REF] Wang | Circulating current suppression for MMC-HVDC under unbalanced grid conditions[END_REF] is applied in the following. The control scheme consists of using a Proportional-Resonant (PR) controller per leg, as depicted in Figure 14. The PR control per phase allows to filter the oscillating component at a given frequency (in this case 100 Hz) per phase and it does not affect components at other oscillations are created in the grid currents leading to AC power oscillations, see Figure 16. In [START_REF] Deng | Circulating current suppression for MMC-HVDC systems with asymmetric arm impedance[END_REF], circulating current suppression for HVDC-MMC systems with asymmetric arm impedance shows similar phenomena. In [START_REF] Soong | Modular multilevel converters with integrated energy storage[END_REF], some strategies are proposed for balancing the energy among the different arms of the MMC while still using the DC voltage for computing the modulation indexes. The addition of supplementary control loops to ensure the horizontal and vertical energy balancing of the legs and arms leads to very similar strategies to the EBC strategies. It can be concluded that, in case of unbalanced energy storage, an EBC is somehow indispensable for the purpose of arm energy balance while allowing balanced grid currents.

Use case C: Energy storage elements are only inserted in one arm

To assess the balancing mechanism of the EBC in arm energy management, the energy storage integration in a single arm of the MMC, which presents the most constraining case for energy balancing, is investigated. The simulated scenario is ESS integration in the upper arm of phase A. only. Power references are as described earlier and shown in Figure 9. Even with the ESS integration in only one arm, the AC active and DC power profiles in Figure 18 are identical to the ones in Figure 10 and Figure 12 for ESS in all the arms of the MMC and in one phase of the MMC, respectively. In other words, the EBC manages effectively to balance the energy among arms despite the most challenging case

Figure 19 shows the voltage of the equivalent capacitor of the MMC arms 𝑣 𝐶 𝑠𝑢𝑚 𝑖 𝑢,𝑙 . The average values of these voltages are balanced as expected, but the ripples are different (in shape and in amplitude). This is also expected [START_REF] Errigo | Assessment of the Impact of Split Storage within Modular Multilevel Converter[END_REF] since the currents and modulation index are different in each arm according to the energy storage power. Consequently, the voltage ripples for equivalent capacitors are not balanced among the upper and lower arms of leg A.

Highlights of Findings

Integrating energy storage within the MMC represents a challenge in terms of controlling and balancing energies in each arm. Differential currents 𝑖 𝑑𝑖𝑓𝑓 𝑖 serve as an indicator of this energy distribution process. Depending on the adopted control strategies, such as EBC or NEBC, the management approach for these currents differs, potentially exerting significant influence on system controllability. These considerations motivate the selection of these currents as a key performance indicator (KPI), see Table III.

The impact of the energy storage integration on the MMC design would serve as an interesting KPI for comparing the three use cases for the MMC with EBC. The analysis in [START_REF] Errigo | Convertisseurs de puissance avec stockage d'énergie intégré pour réseaux haute tension à courant continu[END_REF] covers deeply all key elements of this comparison, mainly the impact of energy storage distribution on the sizing elements of the MMC.

Conclusion

In this paper, MMC model with embedded ESS into its submodules was presented and simulated to assess NEBC and EBC strategy performance. Situations where the ESS is evenly or unevenly balanced between converter arms were investigated.

The comparisons are based on a power profile including combination of both HVDC link power and energy storage power. It has been observed that both NEBC and EBC are well suited for controlling MMC with balanced ESS within the arms. However, the NEBC has poor performance when ESS are unbalanced among MMC arms. Both CCSC-dq and PR-CCSC leads to power oscillations on the AC and/or DC sides. In one case, the CCSC-dq is unable to suppress the 100Hz oscillations on the differential current leading to power oscillations, while in the other case, the PR-CCSC is controlling the circulating current to zero, but leading to unbalance the AC grid currents and power. In general, the NEBC is inefficient when the energies of ESS are not perfectly balanced among the MMC arms, which is always the case when considering ESS lifetime (due to uneven component ageing). The three use cases simulations have outlined the good performance of EBC for ESS integration into MMC arms for balanced and extremely unbalanced situations.

Table III: Summary of results based on the differential currents as a KPI Use case NEBC EBC

A Energy balance among the MMC arms is implicitly maintained, even though the primary focus of this control lies in achieving objectives such as output DC voltage or DC current or grid current regulation. However, this approach leads to less effective management of differential currents. See Figure 20.

Energy balance among the MMC arms results in a response where for all the legs, the upper and lower AC arm currents exhibit the same amplitude. These currents show a smoother response following variations in active power demand. See

Figure 1 :

 1 Figure 1: Three-phase MMC

Figure 2 :Figure 3 :

 23 Figure 2: Considered non-energy-based control (NEBC) of MMC converter

Figure 4 :

 4 Figure 4: Three phase MMC with embedded energy storage in all arms

Figure 5 :

 5 Figure 5: MMC equivalent arm connected to energy storage system represented as a controlled current source

Figure 6 :

 6 Figure 6: Outer control loop of NEBC and EBC of MMC with embedded energy storage

Figure 7 :

 7 Figure 7: MMC converter connected to a DC source and an equivalent 3-ph AC source -Simulated model

Figure 8 :Figure 9 :

 89 Figure 8: MMC with energy storage devices in all arms for use case A, only in one phase (leg B) for use case B and only in one arm for use case C

Figure 10 :

 10 Figure 10: MMC active and DC powers: 𝑃 𝐴𝐶 , 𝑃 𝐷𝐶 and 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓

Figure 11 :

 11 Figure 11: Voltage of the equivalent capacitor of the MMC arms 𝑣 𝑐 𝑠𝑢𝑚 𝑖 𝑢,𝑙

Figure 10 shows

 10 Figure10shows the active and DC powers of the MMC. It is observed that for both controls (EBC and NEBC) the energy storage power 𝑃 𝑠𝑡𝑜 is injected to the AC side via 𝑃 𝐴𝐶 without modifying the power 𝑃 𝐷𝐶 injected to the DC side. With EBC as for NEBC, the voltage ripples in the

Figure 12 :

 12 Figure 12: MMC Active power 𝑃 𝐴𝐶 , 𝑃 𝐷𝐶 and 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓

Figure 13 :

 13 Figure 13: 𝑖 𝑑𝑖𝑓𝑓 𝑖 with NEBC and CCSC disabled (a) -Use case A, (b) -Use caseThe aim of CCSC-dq integration in the NEBC structure is the elimination of the inner balancing currents 𝑖 𝑑𝑖𝑓𝑓 𝑖 in the MMC. In the absence of the CCSC-dq, 𝑖 𝑑𝑖𝑓𝑓 𝑖 are balanced and are composed of a DC component and an AC component at a double-fundamental-frequency i.e. 100Hz. With ESS, in Case A, the 100Hz oscillations are also balanced between the three-phases, and their magnitude varies according to the power from the ESS due to the change in the arm currents (Figure13). Considering Case B, the differential currents 𝑖 𝑑𝑖𝑓𝑓 𝑖 are not balanced anymore.The phase B current is much larger when the power from the ESS is not null. This explains why the classical CCSC-dq is not working properly as it is designed to control balanced currents since it is based only on the positive sequence abc-dq transformation. To eliminate the AC component of the unbalanced differential currents, a solution based on the ideas presented in[START_REF] Wang | Circulating current suppression for MMC-HVDC under unbalanced grid conditions[END_REF] is applied in the following. The control scheme consists of using a Proportional-Resonant (PR) controller per leg, as depicted in Figure14. The PR control per phase allows to filter the oscillating component at a given frequency (in this case 100 Hz) per phase and it does not affect components at other

Figure 14 :

 14 Figure 14: Inner control loop for NEBC using a PR-CSCC frequencies such as the DC components. In this section, this control is named PR-CSCC. Regarding the PR control, 𝑘 𝑝 𝑐𝑐 and 𝑘 𝑟 𝑐𝑐 are the control gains, and 𝜔 0 is the resonant frequency tuned at 100Hz to control the 100Hz oscillations. By comparison of Figure 13 and Figure 15, the effectiveness of the PR-CSCC is demonstrated as this controller allows in one hand to suppress the oscillating component at 100Hz and the DC component of the current in phase B can flow to the DC side. Although the differential currents are correctly controlled, when looking into the grid

Figure 15 :

 15 Figure 15: 𝑖 𝑑𝑖𝑓𝑓 𝑖 in use case B (a) -EBC , (b) -NEBC with PR-CSCC

Figure 18 :

 18 Figure 18: MMC Active power 𝑃 𝐴𝐶 , 𝑃 𝐷𝐶 and 𝑃 𝑆𝑡𝑜 𝑟𝑒𝑓

  Figure 20. B Horizontal energy storage unbalance among the MMC legs results in unbalanced phaseshifted differential currents, oscillating at a frequency of 100Hz, rendering the use of the classical CCSC-dq inefficient. The implementation of CCSC-PR control results in the transfer of oscillations from the differential currents to the AC grid currents. See Figure 13. Horizontal energy storage unbalance among the MMC legs leads to a response where for each leg, the upper and lower AC arm currents exhibit the same amplitude, resulting in differential currents displaying a DC offset. See Figure 21. C Given the unsatisfactory outcomes of employing CCSC-dq and CCSC-PR based NEBC in handling horizontal energy unbalance, it is nonsense to extend such control to this case. Horizontal and vertical energy storage unbalances result in a response characterized by varying amplitudes in the upper and lower AC arm currents within each leg, leading to balanced phase-shifted AC components with different amplitudes observed in the differential currents. See Figure 22.

Figure 20 :

 20 Figure 20: 𝑖 𝑑𝑖𝑓𝑓 𝑖 for use case A (a) -EBC, (b) -NEBC

Table I : 1050 MVA MMC model parameters

 I

	Parameter	Value	Unit
	Rated power 𝑺 𝒏	1050	MVA
	SM per arm 𝑵	400	-
	SM capacitance 𝑪 𝒔𝒎	11.3	mF
	Electrostatic constant 𝑯 𝒄	33	ms
	Arm inductance 𝑳 𝒂𝒓𝒎	46.6	mH
	Arm resistance 𝑹 𝒂𝒓𝒎	0.4	Ω
	AC filter inductance 𝑳 𝒇	84.2	mH
	AC filter resistance 𝑹 𝒇	0.14	Ω

Table II : Grid parameters

 II

	Parameter	Value	Unit
	DC grid voltage 𝑽 𝒅𝒄	640	kV
	AC grid voltage 𝑽 𝒂𝒄	400	VLLrms
	AC grid short circuit level	10000	MVA
	AC grid frequency 𝒇	50	Hz