Ludovic Javet
email: ludovic.javet@inria.fr

Nicolas Anciaux
email: nicolas.anciaux@inria.fr

Luc Bouganim
email: luc.bouganim@inria.fr

Léo Lamoureux

Philippe Pucheral
email: philippe.pucheral@inria.fr

Pushing Edge Computing one Step Further: Resilient and Privacy-Preserving Processing on Personal Devices

Can we push Edge computing one step further? This demonstration paper proposes an answer to this question by leveraging the generalization of Trusted Execution Environments at the very edge of the network to enable resilient and privacy-preserving computation on personal devices. Based on preliminary published results, we show that this can drastically change the way distributed processing over personal data is conceived and achieved. The platform presented here demonstrates the pertinence of the approach through execution scenarios integrating heterogeneous secure personal devices.

INTRODUCTION

With forecasts at 29.3 billion connected devices by 2023 [START_REF]Cisco Annual Internet Report[END_REF], the current reliable, server-based, and infrastructure-centric data management approach exhibits its limits in terms of efficiency, privacy, and energy consumption. While Edge computing aims at letting the data close to the sources and leverage in situ computing resources [START_REF] Shi | Edge Computing: Vision and Challenges[END_REF], data is usually gathered again on cloud or cloudlet servers [START_REF] Satyanarayanan | The Case for VM-Based Cloudlets in Mobile Computing[END_REF] whenever multi-sources computation is required (e.g., to perform statistics or machine learning).

A game changer is the generalization of Trusted Execution Environments (TEEs) [START_REF] Sabt | Trusted Execution Environment: What It is, and What It is Not[END_REF] at the extreme edge of the network: Intel SGX [START_REF] Carpentier | An Extensive and Secure Personal Data Management System Using SGX[END_REF] on PC and tablets, ARM's TrustZone [START_REF] Pinto | Demystifying Arm TrustZone: A Comprehensive Survey[END_REF] on smartphones (Figure 1.a) and Trusted Platform Module (TPM) on smart objects (Figure 1.b). The security features of TEEs enable generic and scalable computations, processing cleartext data without any tradeoff between privacy and accuracy. Hence, large-scale privacy-preserving computations are within reach in a more generic and flexible way than with homomorphic cryptography [START_REF] Boneh | Private Database Queries Using Somewhat Homomorphic Encryption[END_REF], differential privacy [START_REF] Dwork | Differential Privacy[END_REF], or secure multiparty computation protocols [START_REF] Bater | SMCQL: Secure Query Processing for Private Data Networks[END_REF].

In [START_REF] Javet | Edgelet Computing: Pushing Query Processing and Liability at the Extreme Edge of the Network[END_REF], we advocate that the convergence between TEEs and Opportunistic Networks (OppNets) [START_REF] Pelusi | Opportunistic networking: data forwarding in disconnected mobile ad hoc networks[END_REF] can be leveraged to organize fully decentralized and secure query computations among data scattered on multiple personal devices, without resorting to any central authority. We call this new data management paradigm Edgelet computing and propose dedicated execution strategies that guarantee three fundamental properties, namely Resiliency -a query completes before a given deadline according to a given fault presumption rate -, Validity -the query result is equivalent to the one obtained in a centralized context -and Crowd Liability -the liability of the processing is equally distributed among all query participants -. The shift of responsibility from a unique data controller (in the GDPR sense) to the crowd is a salient feature of Edgelet computing. These properties provide tangible guarantees to both Data Contributors (individuals supplying the data) and Data Processors (people whose personal device performs the processing), but also to the beneficiaries of the query results.

This research work is being validated in the field through the DomYcile project [START_REF]DomYcile Project[END_REF]: 8,000 elderly people receiving home care in the French Yvelines district are equipped with a secure box (see Figure 1.b) where their medical records are stored and processed (local queries today, clustering algorithms crossing the data of several patients tomorrow); currently, the boxes are not connected to the Internet for subscription cost and acceptability reasons, but are connected opportunistically by caregivers during their visits. Today, we aim at enlarging the use cases targeted by Edgelet computing, considering that the solutions proposed in [START_REF] Javet | Edgelet Computing: Pushing Query Processing and Liability at the Extreme Edge of the Network[END_REF] apply whenever decentralized computations satisfying Resiliency, Validity, and Crowd Liability need to be performed among TEEenabled personal devices (edgelets) connected through "uncertain" communications. OppNets are an extreme case of uncertain communications in terms of latency and then fault presumption, but uncertainty is inherent in any decentralized computation over a crowd of personal devices, because devices can be disconnected at will, be temporarily out of reach, or simply fail. We sketch below two promising use cases as motivating examples:

Opportunistic polling: during events that welcome a large audience (e.g., a conference, a museum, a concert, a match, a protest), the participants using a TrustZone-based smartphone could contribute with their data (e.g., their centers of interest, nationality, age, etc.) to a global processing to improve the user experience in real time (i.e., adapting the services to the characteristics of the audience).

Data altruism: the Data Governance Act [START_REF]Proposal for a Regulation of the European Parliament and of the Concil on European data governance (Data Governance Act)[END_REF] fosters data subjects to give consent to process their personal data for purposes such as scientific research or public services improvement (e.g., a health survey organized by Santé Publique France). The Edgelet framework allows individuals to provide their health data with strong guarantees of confidentiality, through a secure box, a smartphone, or a PC, and to benefit from the results obtained.

The goal of this demo paper is twofold. First, it exemplifies the versatility of the approach by demonstrating the Edgelet computing mechanisms running on different TEE-enabled devices (from PC with SGX up to smart objects with TPM). Second, it presents the internals of Edgelet computing and let the audience play with important parameters related to resiliency and data privacy and observe the outcome by themselves. A video of the demonstration is available online 1 .

EDGELET COMPUTING FRAMEWORK

A lot of research work has been produced on the different areas encompassed by Edgelet computing: Computing architectures: from cloud, fog, edge computing to mobile edge [START_REF] Ferrer | Towards the Decentralised Cloud: Survey on Approaches and Challenges for Mobile, Ad hoc, and Edge Computing[END_REF], mobile ad-hoc cloud computing [START_REF] Yaqoob | Mobile ad hoc cloud: A survey[END_REF] and cross-device federated machine learning [START_REF] Kallista | Towards Federated Learning at Scale: System Design[END_REF]; Private/Confidential computing: using homomorphic encryption [START_REF] Boneh | Private Database Queries Using Somewhat Homomorphic Encryption[END_REF], secure multiparty computation [START_REF] Bater | SMCQL: Secure Query Processing for Private Data Networks[END_REF], differential privacy techniques [START_REF] Dwork | Differential Privacy[END_REF] or TEEs (in the cloud [START_REF] Schuster | VC3: Trustworthy Data Analytics in the Cloud Using SGX[END_REF], or at the edges [START_REF] Bouganim | Highly distributed and privacy-preserving queries on personal data management systems[END_REF][START_REF] Ladjel | Trustworthy Distributed Computations on Personal Data Using Trusted Execution Environments[END_REF]). To the best of our knowledge, none of the aforementioned papers and surveys address the challenge raised by Edgelet computing: how to perform complex and scalable distributed queries on data scattered on multiple personal devices while ensuring fault and disconnection tolerance?

In this section, we first recall from [START_REF] Javet | Edgelet Computing: Pushing Query Processing and Liability at the Extreme Edge of the Network[END_REF] the basics of Edgelet execution strategies that can scale to a large number of participants while providing privacy guarantees. Then, we present a resilient execution strategy that handles device failures as well as temporary disconnections, and produces accurate results.

Design of Edgelet Computation Plans

Contrary to participatory sensing or sensor networks which focus on stream queries over elementary data, Edgelet computing aims at handling rich data (e.g., healthcare folders, spending habits) and complex processing (e.g., machine learning, data mining) over edgelets data seen as a horizontal partitioning of a shared database (i.e., edgelets data conform to a common schema). We represent these computations in the form of a Query Execution Plan (QEP), that is a directed graph where vertices materialize the operators to be computed and edges represent the dataflow among them. A secure assignment of these operators is then essential to avoid any targeted attacks, but is not detailed here for the sake of brevity.

The simplest form of a QEP is a tree with Data Contributors at the leaves (one per edgelet contributing to the query with its data) sending the requested data to a Snapshot Builder operator, the role of which is to collect a representative snapshot of cardinality 𝐶 (e.g., select tuples from 𝐶 = 2000 patients over 65 years old).

The Snapshot Builder then sends the representative snapshot to a Computer operator which, in turn, computes the final result and eventually delivers it to the Querier.

The Data Processors (here, the Snapshot Builder and the Computer) can be decomposed into sub-operators assigned to different edgelets (i.e., TEE-enabled devices) for performance or privacy concerns. Indeed, although highly difficult to conduct and requiring physical instrumentation, side-channel attacks compromising data confidentiality cannot be totally ignored [START_REF] Wang | Leaky Cauldron on the Dark Land: Understanding Memory Side-Channel Hazards in SGX[END_REF], placing the TEE in a "sealed glass" mode [START_REF] Tramer | Sealed-Glass Proofs: Using Transparent Enclaves to Prove and Sell Knowledge[END_REF] (the integrity guarantee is preserved, but not the confidentiality). This decomposition can help minimizing the amount of data exposed at each edgelet by horizontally partitioning the dataset. This can also preclude the concomitant exposure, in the same edgelet, of data items that become sensitive when combined (e.g., a quasi-identifier) by vertically partitioning the dataset. This can finally help minimizing the workload (e.g., when energy consumption matters) or exploit the inherent Edgelet computing parallelism. A Computing Combiner operator must then be added in the QEP to combine the outputs of all sub-operators.

Figure 2 presents the QEP for an example query using horizontal partitioning (Data Contributors are assigned to Snapshot Builders, e.g., by hashing their public key) and vertical partitioning (each Computer manages a single statistic, e.g., Age, BMI).

Edgelet Query Execution

During the execution of a QEP, each edgelet acting as a Data Processor is a potential Single Point of Failure, either because it really fails or because of a temporary disconnection.

To solve this problem, we explored in [START_REF] Javet | Edgelet Computing: Pushing Query Processing and Liability at the Extreme Edge of the Network[END_REF] two resiliency strategies, namely Backup and Overcollection, which both enable the execution of a query while respecting the resiliency, validity and crowd liability properties introduced earlier. These two strategies differ on complexity, performance and on the way privacy is enforced, making each strategy best adapted to different contexts (see [START_REF] Javet | Edgelet Computing: Pushing Query Processing and Liability at the Extreme Edge of the Network[END_REF] for a complete taxonomy). In this demonstration, we focus on the Overcollection strategy which is best adapted to any use case where performance matters and approximate results are acceptable (e.g., statistics, machine learning processes).

We illustrate the Overcollection principle by considering a query over a sample dataset (e.g., 𝐶 individuals with age > 65) where Data Processors (i.e., Snapshot Builder and Computers) execute distributive operators. Instead of executing the operators on single edgelets, we distribute (using hashing) its execution over 𝑛 + 𝑚 edgelets where each one processes a partition of the original dataset, with 𝑛 the minimum number of partitions to be collected and 𝑚 the overcollection parameter (see Figure 3). The validity is then trivially preserved as long as (1) each of the 𝑛 + 𝑚 partitions is representative and has a cardinality 𝐶/𝑛, and

(2) less than m partitions are lost. Note that we need to add to the Computing Combiner an Active Backup, i.e., a replicated instance running in parallel with it, in order to handle its potential failure. Many data-intensive queries of interest are distributive by nature and can accommodate Overcollection. This is also the case of general interest Machine Learning (ML) algorithms. However, ML algorithms are often iterative and require to exchange partial results computed over different data partitions. A strict validity between the different iterations would require that a valid snapshot is processed at each iteration, and that the same valid snapshot (same partitions) is kept during the subsequent iterations, which is not easily ensured due to various message loss at each iteration. Fortunately, strict validity is not a prerequisite for these algorithms, and resampling at each iteration sometimes even produces better accuracy (as in Mini-batch K-Means [START_REF] Sculley | Web-scale k-means clustering[END_REF]).

The method for executing an iterative algorithm A (e.g., K-Means [START_REF] Inderjit | A Data-Clustering Algorithm on Distributed Memory Multiprocessors[END_REF]) in the Edgelet context is the following: each edgelet implementing a Computer iterates on (1) a local convergence phase where it computes A on its local partition, improving its local knowledge (e.g., its centroids), and broadcasts this knowledge to all others Computers, and (2) a synchronization phase where it receives the knowledge of the other Computers it has heard of and integrates them in its own knowledge (e.g., the barycenter for each centroid). Right before the query deadline, the knowledge is sent to the Computing Combiner which combines all received knowledges and sends the final result to the Querier. We enforce the progression of the algorithm on all edgelets thanks to a Heartbeat, that is each iteration is cadenced by a clock, whatever the local state of the processing, i.e., the Computers move to the next iteration even if few or no messages were received (failures or disconnections).

DEMONSTRATION 3.1 Demonstration Platform

The first objective of this demonstration is to present the computational mechanisms of the Edgelet framework using TEE-enabled devices ranging from high-end device (PC) to low-end device (home box). Therefore, we deploy the following hardware:

• PC (Intel SGX). A laptop with an Intel Core i5-9400H 2.5GHz 4 Cores with SGX 1-FLC runs secure enclaves in an Ubuntu Linux 20.04 environment. The code is written on top of Open Enclave [START_REF]Open Enclave SDK[END_REF], an SDK for developing enclave applications in C/C++. Open Enclave provides support for Intel SGX as well as preview support for ARM TrustZone, thus aiming to generalize the development of enclave applications across TEEs.

• Home boxes (TPM). These secure boxes (see Figure 1.b, identical to those deployed in the field by the DomYcile project [START_REF]DomYcile Project[END_REF], incorporate a STM32F417 microcontroller. The latter is connected to a 𝜇-SD card hosting the owner's raw data and to a tamper-proof Trusted Platform Module (TPM). This TPM secures the cryptographic keys and guarantees, during the secure boot, that the embedded code has not been tampered with.

The second objective is to present the internal aspects of the solution, we thus develop the following software: (1) a GUI implemented in Dash Python that allows interactive configuration and visualization of Edgelet queries; (2) an Edgelet manager that orchestrates executions and communications between simulated and real edgelets (PC and home boxes); (3) a web client accessible to attendees' smartphones via a QR code allowing them to visualize the processed data and interact in real-time with the execution.

Demonstration Scenario

In this demonstration, we take the motivating example presented in the introduction. Santé Publique France (Querier) wants to perform a set of queries on population health data to improve the quality of its services. Some individuals are equipped with a PC, others with a secure home box, but all are interconnected by uncertain communications. The scenario consists of two interactive parts related to the configuration and execution of Edgelet computations. In the first part, the attendees will understand the impact of privacy and resiliency parameters on the QEPs. In the second part, they will follow the execution in real time and visualize the results obtained. These two parts are represented on Figure 4.

Part 1: QEP Configuration. The attendees are first invited to select a query among the 2 proposed ones: (i) a Grouping Sets query [START_REF]Group By Grouping Sets -Snowflake Documentation[END_REF] which allows multiple Group-By clauses to be evaluated within a single SQL query (e.g., to cross multiple statistics over the same data sample); or (ii) a K-Means [START_REF] Inderjit | A Data-Clustering Algorithm on Distributed Memory Multiprocessors[END_REF] followed by a Group By on the resulting clusters (e.g., to identify which characteristics most influence the dependency level of an elderly person). Then, following their intuition, the attendees can try to improve the privacy of the QEP of the selected query to reduce data exposure in case of TEEs compromise. To do so, they can adjust the horizontal and vertical partitioning parameters presented in Section 2.1, by specifying the maximum number of raw data per edgelet and selecting the attribute pairs to be separated. Finally, the attendees can vary the failure probability value of the scenario and observe automatic changes in the execution plan to keep it resilient. We illustrate the Overcollection principle by considering a query over a sample dataset (e.g., C individuals with age > 65) where Data Processors (i.e., Snapshot Builder and Computers) execute distributive operators. Instead of executing the operators on single edgelets, we distribute (using hashing) its execution over n+m edgelets where each one processes a partition of the original dataset, with n the minimum number of partitions to be collected and m the overcollection parameter (see Figure 3). The validity is then trivially preserved as long as (1) each of the n+m partitions is representative and has a cardinality C/n, and (2) less than m partitions are lost. Note that we need to add to the Computing Combiner an Active Backup, i.e., a replicated instance running in parallel with it, in order to handle its potential failure.

Many data-intensive queries of interest are distributive by nature and can accommodate Overcollection. This is also the case of general interest Machine Learning (ML) algorithms. However, ML algorithms are often iterative and require to exchange partial results computed over different data partitions. A strict validity between the different iterations would require that a valid snapshot is processed at each iteration, and that the same valid snapshot (same partitions) is kept during the subsequent iterations, which is not easily ensured due to various message loss at each iteration. Fortunately, strict validity is not a prerequisite for these algorithms, and resampling at each iteration sometimes even produces better accuracy (as in Mini-batch K-Means [START_REF] Schuster | VC3: Trustworthy Data Analytics in the Cloud Using SGX[END_REF]).

The method for executing an iterative algorithm A (e.g., K-Means [START_REF] Kallista | Towards Federated Learning at Scale: System Design[END_REF]) in the Edgelet context is the following: each edgelet implementing a Computer iterates on (1) a local convergence phase where it computes A on its local partition, improving its local knowledge (e.g., its centroids), and broadcasts this knowledge to all others Computers, and (2) a synchronization phase where it receives the knowledge of the other Computers it has heard of and integrates them in its own knowledge (e.g., the barycenter for each centroid). Right before the query deadline, the knowledge is sent to the Computing Combiner which combines all received knowledges and sends the final result to the Querier. We enforce the progression of the algorithm on all edgelets thanks to a Heartbeat, that is each iteration is cadenced by a clock, whatever the local state of the processing, i.e., the Computers move to the next iteration even if few or no messages were received (failures or disconnections).

DEMONSTRATION

Demonstration Platform

The first objective of this demonstration is to present the computational mechanisms of the Edgelet framework using TEEenabled devices ranging from high-end device (PC) to low-end device (home box). Therefore, we deploy the following hardware:

• PC (Intel SGX). A laptop with an Intel Core i5-9400H 2.5GHz

4 Cores with SGX 1-FLC runs secure enclaves in an Ubuntu Linux 20.04 environment. The code is written on top of Open Enclave [START_REF] Javet | Edgelet Computing: Pushing Query Processing and Liability at the Extreme Edge of the Network[END_REF], an SDK for developing enclave applications in C/C++. Open Enclave provides support for Intel SGX as well as preview support for ARM TrustZone, thus aiming to generalize the development of enclave applications across TEEs. • Home boxes (TPM). These secure boxes (see Figure 1.b), identical to those deployed in the field by the DomYcile project [START_REF] Boneh | Private Database Queries Using Somewhat Homomorphic Encryption[END_REF], incorporate a STM32F417 microcontroller. The latter is connected to a μ-SD card hosting the owner's raw data and to a tamper-proof Trusted Platform Module (TPM). This TPM secures the cryptographic keys and guarantees, during the secure boot, that the embedded code has not been tampered with. The second objective is to present the internal aspects of the solution, we thus develop the following software: (1) a GUI implemented in Dash Python that allows interactive configuration and visualization of Edgelet queries; [START_REF]DomYcile Project[END_REF] an Edgelet manager that orchestrates executions and communications between simulated and real edgelets (PC and home boxes); (3) a web client accessible to attendees' smartphones via a QR code allowing them to visualize the processed data and interact in real-time with the execution.

Demonstration Scenario

In this demonstration, we take the motivating example presented in the introduction. Santé Publique France (Querier) wants to perform a set of queries on population health data to improve the quality of its services. Some individuals are equipped with a PC, others with a secure home box, but all are interconnected by uncertain communications. The scenario consists of two interactive parts related to the configuration and execution of Edgelet computations. In the first part, the attendees will understand the impact of privacy and resiliency parameters on the QEPs. In the second part, they will follow the execution in real time and visualize the results obtained. These two parts are represented on the figure below. Part 2: Execution of an Edgelet computation. After the attendees have configured the query as desired, we proceed to its real-time execution on the heterogeneous personal devices available for the demonstration (concrete edgelets), the rest of the operators being associated to a configurable number of simulated edgelets to attest scalability. The demonstration platform allows the attendees to visualize, step by step, the query execution. Initially, we launch the collection phase where Snapshot Builders receive contributions from thousands of simulated Data Contributors and build representative snapshots. Next, the Edgelet platform redistributes the data and launches the computation phase with the corresponding Computers. At each step, the user can interact with the execution using their own smartphone to analyze the input and output data. At the end, the aggregated data is transmitted to the Computing Combiners for the combination phase and the query is completed. In case of failures or disconnections, the attendees will be able to directly identify the concerned edgelets on the QEP and understand the impacts on the execution. For example, we can intentionally power off some concrete devices to generate a failure at will. In order to verify the results, the attendees can take the same dataset used with the distributed edgelets and run the processing centrally on the demonstration platform.

Demonstration Results

The demonstration shows the internals of the Edgelet computing framework applied to the fully decentralized context and illustrates its usage through execution scenarios on multiple personal devices. It helps to answer the following questions:

Does Edgelet computing concretely make sense? The practical implementation on high-end and low-end personal devices demonstrates both its applicability and versatility. It shows that large-scale general-purpose computations can be performed over devices while providing high-security guarantees. This opens up important opportunities in terms of personal data management.

Can any form of computation be handled? Edgelet computing leverages the TEE security to perform computations on clear-text data (once decrypted locally). It can then combine computation generality -demonstrated by our demonstration queries -and scalability -demonstrated by the number of simulated edgelets -, contrary to homomorphic encryption, secure multiparty computation, or local differential privacy solutions. Note that the Overcollection strategy only applies if the processing is distributive; otherwise, the Backup strategy can be used at the price of a higher complexity and lower performance (see [START_REF] Javet | Edgelet Computing: Pushing Query Processing and Liability at the Extreme Edge of the Network[END_REF] for details).

Is privacy protected whatever the attack? While highly difficult to implement, side-channel attacks on TEEs could compromise the confidentiality of the manipulated data. Edgelet computing counter-measures are horizontal and vertical partitioning. Through the demonstration, attendees will be able to visualize the distribution of data among the edgelets and measure the respective benefit of both types of partitioning. They will also understand that only the results of the computations, i.e. the aggregated data, are sent (encrypted) to the successor operators.

Can a query always proceed despite the failures? Providing fault tolerance in a distributed context where messages are sent among weakly connected personal devices is a real challenge, either because they are down or because they are temporarily unavailable (e.g., offline smartphones). The demonstration shows that the Overcollection strategy can answer this: attendees will be allowed to vary the failure context (e.g., disconnection probability) and see the impact on the overcollection degree as well as the effects on the results accuracy with respect to the number of heartbeats.

Figure 1 :

 1 Figure 1: Instances of Trusted Execution Environments.

Figure 2 :

 2 Figure 2: Vertically and Horizontally partitioned QEP.

Figure 3 :

 3 Figure 3: Overcollection for the QEP of Figure 2.

Figure 3 :

 3 Figure 3: Overcollection for the QEP of Figure 2

Part 1 :

 1 Configuration of a QEP Part 2: Execution of an Edgelet Computation

Figure 4 :Figure 4 :

 44 Figure 4: Edgelet computing demonstration platform