Andy Dolan

Indrakshi Ray

Suryadipta Majumdar

Proactively Extracting IoT Device Capabilities: An Application to Smart Homes

Internet of Things (IoT) device adoption is on the rise. Such devices are mostly self-operated and require minimum user interventions. This is achieved by abstracting away their design complexities and functionalities from the users. However, this abstraction significantly limits a user's insights on evaluating the true capabilities (i.e., what actions a device can perform) of a device and hence, its potential security and privacy threats. Most existing works evaluate the security of those devices by analyzing the environment data (e.g., network traffic, sensor data, etc.). However, such approaches entail collecting data from encrypted traffic, relying on the quality of the collected data for their accuracy, and facing difficulties in preserving both utility and privacy of the data. We overcome the above-mentioned challenges and propose a proactive approach to extract IoT device capabilities from their informational specifications to verify their potential threats, even before a device is installed. We apply our approach to the context of a smart home and evaluate its accuracy and efficiency on the devices from three different vendors.

Introduction

The popularity of IoT devices is gaining momentum (e.g., projections of 75.44 billion devices worldwide by 2025 [START_REF]Internet of Things (IoT) connected devices installed base worldwide from 2015 to 2025[END_REF]). This large ecosystem is comprised of a variety of devices that are being used in diverse environments including healthcare, industrial control, and homes. Manufacturers emphasize certain features and characteristics of the IoT devices and often abstract away their actual design complexity and functionalities from the user. Many IoT devices are equipped with an extended set of sensors and actuators which allows them to perform different functionalities. For example, a smart light with a microphone and motion detector can possibly perform far more than just light sensing.

Such abstraction and extended (and in many cases hidden) functionalities of an IoT device result in a blind spot for the consumers and leave an IoT system vulnerable to various security and privacy threats. Installing the abovementioned smart light necessitates that the consumer understand its potential security and privacy consequences. This requires the consumer to study its design specifications to find out what sensors it possess. Furthermore, she must have the insights to realize the security and privacy consequences of having a microphone and motion detector in a light, and determine if any of those consequences violate the policies of the household or organization. Performing all these steps is infeasible for most IoT users due to either their time constraints or lack of knowledge. Therefore, IoT consumers need assistance to properly interpret the underlying security and privacy threats from these devices. Our work aims to fill this gap by providing consumers information on IoT device capabilities.

A comprehensive knowledge of device capabilities can be used in various security applications, including security verification, monitoring, risk analysis, and digital forensics. One example application is proactively verifying the security and privacy of IoT devices in a smart home or in an organization. Specifically, once we know the capabilities of a device, we can check if any of those capabilities violate any of the security and privacy policies in an organization or a household. We can also ensure that the deployment of an IoT device in some location or under some configuration does not cause any security or privacy breaches and can take adaptive measures to mitigate that risk.

Several works [START_REF] Lopez-Martin | Network Traffic Classifier With Convolutional and Recurrent Neural Networks for Internet of Things[END_REF][START_REF] Oconnor | HomeSnitch: Behavior Transparency and Control for Smart Home IoT Devices[END_REF][START_REF] Zhang | HoMonit: Monitoring Smart Home Apps from Encrypted Traffic[END_REF][START_REF] Bezawada | Behavioral Fingerprinting of IoT Devices[END_REF][START_REF] Miettinen | IoT SENTINEL: Automated Device-Type Identification for Security Enforcement in IoT[END_REF][START_REF] Marchal | AuDI: Toward Autonomous IoT Device-Type Identification Using Periodic Communication[END_REF][START_REF] Ortiz | DeviceMien: Network Device Behavior Modeling for Identifying Unknown IoT Devices[END_REF][START_REF] Wang | Fear and Logging in the Internet of Things[END_REF][START_REF] Birnbach | Peeves: Physical Event Verification in Smart Homes[END_REF] that profile IoT devices and their behaviors to detect security breaches and/or monitor an IoT environment pose two limitations: (i) Collecting and interpreting data from an IoT system is extremely challenging. Existing solutions [START_REF] Zhang | HoMonit: Monitoring Smart Home Apps from Encrypted Traffic[END_REF][START_REF] Bezawada | Behavioral Fingerprinting of IoT Devices[END_REF] either perform entropy analysis of encrypted traffic or use only the unencrypted features of network traffic (e.g., TCP headers and flow metadata). Due to its great reliance on data inference, false positives/negatives are a legitimate concern. Providing better accuracy in these security solutions is a critical challenge. (ii) Such approaches may reveal sensitive information (e.g., daily routines of smart home users [START_REF] Birnbach | Peeves: Physical Event Verification in Smart Homes[END_REF]) about an IoT system and its users, threatening their privacy. Preserving privacy while sharing sensitive data for security analysis is another challenge.

We overcome these limitations and propose an approach to proactively extract IoT device capabilities from their design specifications. We first define the notion of device capability in the context of IoT. Second, we extract the transducer (e.g., sensors and actuators) information for each device using vendorprovided specification materials. Third, we identify the capabilities of a device by deriving the capabilities of each sensor and actuator of that device. We discuss our approach in the context of smart homes, an important IoT domain (with projections of 505 million active smart home devices worldwide by this year [START_REF]Smart home -United States[END_REF]) and evaluate its efficiency and accuracy. The main contributions of this paper are as follows.

-We propose a new approach to proactively extract the device capabilities from design specifications. The key advantages of this approach over existing works are: (i) this approach does not rely on environment data and is therefore not directly affected by the difficulties of collecting and interpreting IoT data, and further is free from the privacy concerns of data sharing; and (ii) this approach enables proactive security verification of an IoT device even before it is installed or deployed. -We are the first to define this concept of device capability in IoT, which can potentially be applied in the future security solutions for various IoT applications (e.g., smart grid, autonomous vehicle, smart health, etc.) to offer proactive security guarantee.

-As a proof of concept, we apply our approach in the context of smart homes. We demonstrate the applicability of our approach by applying it to devices from various vendors (e.g., Google, Ring, and Alro), and we evaluate it in terms of its efficiency and accuracy.

The remainder of the paper is organized as follows. Section 2 summarizes related work. Section 3 provides background on vendor materials. Section 4 presents our methodology. Section 5 describes its implementation. Section 6 presents the evaluation results. Section 7 concludes the paper.

Related Work

Research on IoT security has gained significant interest. These studies [START_REF] Lopez-Martin | Network Traffic Classifier With Convolutional and Recurrent Neural Networks for Internet of Things[END_REF][START_REF] Oconnor | HomeSnitch: Behavior Transparency and Control for Smart Home IoT Devices[END_REF][START_REF] Zhang | HoMonit: Monitoring Smart Home Apps from Encrypted Traffic[END_REF][START_REF] Bezawada | Behavioral Fingerprinting of IoT Devices[END_REF][START_REF] Miettinen | IoT SENTINEL: Automated Device-Type Identification for Security Enforcement in IoT[END_REF][START_REF] Marchal | AuDI: Toward Autonomous IoT Device-Type Identification Using Periodic Communication[END_REF][START_REF] Ortiz | DeviceMien: Network Device Behavior Modeling for Identifying Unknown IoT Devices[END_REF][START_REF] Wang | Fear and Logging in the Internet of Things[END_REF][START_REF] Choi | Detecting and identifying faulty IoT devices in smart home with context extraction[END_REF][START_REF] Yang | Hide your hackable smart home from remote attacks: The multipath onion IoT Gateways[END_REF][START_REF] Zhang | Modeling virtual channel to enforce runtime properties for IoT services[END_REF][START_REF] Bhatt | An access control framework for cloud-enabled wearable Internet of Things[END_REF][START_REF] Celik | IoTGuard: Dynamic Enforcement of Security and Safety Policy in Commodity IoT[END_REF][START_REF] Serror | Towards in-network security for smart homes[END_REF][START_REF] Román-Castro | Evolution and trends in IoT security[END_REF]) are categorized into device fingerprinting, application monitoring, intrusion detection, and access control.

The existing device fingerprinting techniques [START_REF] Lopez-Martin | Network Traffic Classifier With Convolutional and Recurrent Neural Networks for Internet of Things[END_REF][START_REF] Oconnor | HomeSnitch: Behavior Transparency and Control for Smart Home IoT Devices[END_REF][START_REF] Zhang | HoMonit: Monitoring Smart Home Apps from Encrypted Traffic[END_REF][START_REF] Bezawada | Behavioral Fingerprinting of IoT Devices[END_REF][START_REF] Miettinen | IoT SENTINEL: Automated Device-Type Identification for Security Enforcement in IoT[END_REF][START_REF] Marchal | AuDI: Toward Autonomous IoT Device-Type Identification Using Periodic Communication[END_REF][START_REF] Ortiz | DeviceMien: Network Device Behavior Modeling for Identifying Unknown IoT Devices[END_REF] monitor and analyze network traffic in IoT. More specifically, [START_REF] Lopez-Martin | Network Traffic Classifier With Convolutional and Recurrent Neural Networks for Internet of Things[END_REF][START_REF] Oconnor | HomeSnitch: Behavior Transparency and Control for Smart Home IoT Devices[END_REF] automatically discover and profile device behaviors by building machine learning models trained on network traffic according to their service (e.g., DNS, HTTP) and the semantic behaviors of devices (e.g., detected motion), respectively. Similar analysis is performed in Zhang et al. [START_REF] Zhang | HoMonit: Monitoring Smart Home Apps from Encrypted Traffic[END_REF], where the fingerprints of a particular smart home device are built using its network traffic. Other works (e.g., [START_REF] Bezawada | Behavioral Fingerprinting of IoT Devices[END_REF][START_REF] Miettinen | IoT SENTINEL: Automated Device-Type Identification for Security Enforcement in IoT[END_REF][START_REF] Marchal | AuDI: Toward Autonomous IoT Device-Type Identification Using Periodic Communication[END_REF][START_REF] Ortiz | DeviceMien: Network Device Behavior Modeling for Identifying Unknown IoT Devices[END_REF]) use similar techniques to automatically determine device identity or typical aggregate behaviors (as opposed to specific behavior). Bezawada et al. [START_REF] Bezawada | Behavioral Fingerprinting of IoT Devices[END_REF] utilize machine learning to build behavior profiles based on network traffic for devices using the device category and device type. IoTSentinel [START_REF] Miettinen | IoT SENTINEL: Automated Device-Type Identification for Security Enforcement in IoT[END_REF], AuDI [START_REF] Marchal | AuDI: Toward Autonomous IoT Device-Type Identification Using Periodic Communication[END_REF] and DeviceMien [START_REF] Ortiz | DeviceMien: Network Device Behavior Modeling for Identifying Unknown IoT Devices[END_REF] use unsupervised learning to build models for individual device-types based on network traffic captured during a device connection.

There exist several other security solutions (e.g., [START_REF] Jia | ContexloT: Towards Providing Contextual Integrity to Appified IoT Platforms[END_REF][START_REF] Tian | Smar-tAuth: User-Centered Authorization for the Internet of Things[END_REF][START_REF] Celik | IoTGuard: Dynamic Enforcement of Security and Safety Policy in Commodity IoT[END_REF][START_REF] Zhang | Modeling virtual channel to enforce runtime properties for IoT services[END_REF]) for smart homes. The existing application monitoring techniques (e.g., [START_REF] Jia | ContexloT: Towards Providing Contextual Integrity to Appified IoT Platforms[END_REF][START_REF] Tian | Smar-tAuth: User-Centered Authorization for the Internet of Things[END_REF]) run on source code of IoT applications and analyze these applications. More specifically, ContextIoT [START_REF] Jia | ContexloT: Towards Providing Contextual Integrity to Appified IoT Platforms[END_REF] and SmartAuth [START_REF] Tian | Smar-tAuth: User-Centered Authorization for the Internet of Things[END_REF] offer permission-based systems to monitor an individual app. ProvThings [START_REF] Wang | Fear and Logging in the Internet of Things[END_REF] builds provenance graphs using securitycritical APIs for IoT forensics. Soteria [START_REF] Celik | Soteria: Automated IoT Safety and Security Analysis[END_REF] and IoTGuard [START_REF] Celik | IoTGuard: Dynamic Enforcement of Security and Safety Policy in Commodity IoT[END_REF] verify security and safety policies by performing static and dynamic code analysis, respectively. Zhang et al. [START_REF] Zhang | Modeling virtual channel to enforce runtime properties for IoT services[END_REF] monitor isolation-related properties among IoT devices through a virtual channel. Yang et al. [START_REF] Yang | Hide your hackable smart home from remote attacks: The multipath onion IoT Gateways[END_REF] protect IoT devices from remote attacks by hiding them inside onion gateways.

Limitations of Existing Work. First, most of the solutions above rely on a great amount of inference, especially when considering encrypted network traffic. Many solutions either perform entropy analysis of encrypted traffic [START_REF] Bezawada | Behavioral Fingerprinting of IoT Devices[END_REF] or use only the unencrypted features of network traffic such as TCP headers and other packet and flow metadata [START_REF] Lopez-Martin | Network Traffic Classifier With Convolutional and Recurrent Neural Networks for Internet of Things[END_REF][START_REF] Oconnor | HomeSnitch: Behavior Transparency and Control for Smart Home IoT Devices[END_REF][START_REF] Zhang | HoMonit: Monitoring Smart Home Apps from Encrypted Traffic[END_REF][START_REF] Marchal | AuDI: Toward Autonomous IoT Device-Type Identification Using Periodic Communication[END_REF][START_REF] Miettinen | IoT SENTINEL: Automated Device-Type Identification for Security Enforcement in IoT[END_REF][START_REF] Ortiz | DeviceMien: Network Device Behavior Modeling for Identifying Unknown IoT Devices[END_REF]. Because of this inference, false positives and false negatives are a legitimate concern of these solutions. Second, as most of the existing works rely on the application of inferential models (machine learning or otherwise), they are vulnerable to deceptive attacks, where an adversary may craft an attack that conforms to the model's expectation of legitimate traffic or behavior, thereby circumventing the model. An attack at the other end of this spectrum would be to simply conduct a denial-of-service attack by, for example, inundating the system with purposefully malicious traffic to overwhelm the model and prevent the processing of any legitimate traffic. Third, these related works cannot detect/prevent the critical safety or privacy implications that are not observable from the network traffic.

Our paper, on the other hand, is complementary to those existing works, and targets a different threat model where we extract IoT device capabilities from their design specifications that will facilitate evaluating potential security threats even before a device is installed.

3 Vendor Materials

Vendor Material Description

We consider vendor materials including product webpages, technical specifications, and developer documentations which are publicly available and contain aspects of the specifications (sensors, actuators, or related features) of a device. Product Webpages. Product webpages are official marketing pages from which a consumer can purchase the product, and contain the summary information about a device. For instance, Google has an online store for its smart home products (e.g., [START_REF]Nest cam indoor -home security camera -google store[END_REF]). These pages can be an initial source of information about a smart home device and its specifications.

Technical Specifications. Technical specification pages provide details about a device and its hardware specifications. For instance, Google has a technical specification page for its smart home products (e.g., [START_REF]Nest cam indoor -installation and tech specs -google store[END_REF]). This work considers these technical specification pages as one of the most significant sources of information about a device's hardware components.

Developer Documentations. Developer documentations provide information to developers who create applications for the smart home devices. Even though these materials are intended for application developers, they can be used as a source for the extraction of information about the hardware and capabilities of a device.

Investigation on the Real-World Vendor Materials

Analysis of the Vendor Materials. We analyze the contents of several vendor materials by leveraging natural language processing techniques. These analyses result in insights on the challenges that come with the extraction of vendor materials, which is illustrated through the following examples.

Figure 1a shows the term frequency distribution for the Google Nest Cam Indoor's vendor materials as a word cloud, where the larger terms appear more frequently across the corpus. This particular corpus is constructed from the Nest Cam Indoor main product page, technical specifications page, technical specifications support page, and the Nest Cam Developer API documentation [START_REF]Nest cam indoor -home security camera -google store[END_REF][START_REF]Nest cam indoor -installation and tech specs -google store[END_REF][START_REF]Technical specifications for nest cameras and video doorbells -google nest help[END_REF][START_REF]Camera API -nest developers[END_REF]. The full corpus contains 4,175 words after pre-processing. The word cloud suggests that terms that would intuitively be assumed to appear frequently, such as "camera" and "nest" appear often, as these terms are directly related to the primary functionality of the device. However, terms that are indicative of other transducers and their capabilities appear less often, and even appear less often than terms that are unrelated or potentially indicative of transducers that the device does not have. Figure 1b illustrates the frequency distribution of only a subset of notable terms. Additionally, term frequency-inverse document frequency (TF-IDF) calculations are also performed within individual corpora for a device. Specifically, each separate vendor material for a device is treated as an article in the corpus, and the TF-IDF metric is computed for an individual corpus. Overall, TF-IDF fails to find terms that are most indicative of a device's transducers and capabilities due to their infrequent appearances. We also apply TF-IDF on each section of a vendor material as an individual document. However, this technique also does not produce any conclusive results.

Challenges in Extracting Capabilities from Vendor Materials. Based on the outcome of the analysis above, we enumerate the major challenges in extracting capabilities from vendor materials as follows.

-No Standaridized Template. Each vendor follows different templates for their materials and furthermore, different materials of the same vendor follow different formats. There is no standard template or specification for how to describe different generic features or hardware components. This implies significant effort to learn those different templates to enable their extraction. -Brevity of the Materials. Vendor materials are usually expressed in a brief manner and do not include all explicit specifications of a device. Therefore, extracting device information from them requires more interpretation of the contents. Additionally, terms that are indicative of particular hardware components may only appear a limited number of times within the materials, especially if they are not related to the primary function of the device. -Vendor-Specific Jargons. Each vendor tends to use their own set of terminologies for their devices. Mainly due to their business policy, vendors craft languages around what information they believe is the most useful to or well-received by the consumer, and include terminology that may be unique to only their line of products. Accordingly, learning the vocabularies used for various vendors and then normalizing them to infer their capabilities presents additional challenges. -Interpreting Visual Contents. Several contexts (e.g. device type) of a material is visually represented and is therefore very difficult to encode automatically. An interesting aspect of the marketing and technical specification pages for IoT devices is the way that page layout and structure provide contextual information in the form of visual cues and hierarchical organization. Therefore, text processing alone becomes insufficient in those cases. -Distributed Materials. The information about a device is distributed over various materials (e.g., product webpage, technical specifications, user manual, and developer documentation), and it is essential to obtain information from as many different materials as possible, normalize their formats, and extract capabilities for the most thorough extraction.

Methodology

We first present our threat model and the assumptions of our approach, followed by the overview, and finally the details.

Threat Model

We focus on smart homes, an IoT application, in this work. We assume that the sensors and actuators in a smart home device may be used to conduct various security and privacy attacks. Our approach, therefore, builds the device capabilities (i.e., the actions that a device can perform) which can later be used to detect/prevent the adversaries that exploit these sensors or actuators. Our approach does not consider the threats from a malicious or vulnerable transducer; which includes misbehavior and malfunction. Also, any network attack that does not involve the transducers is beyond the scope of this paper. In this work, we derive the device capabilities from the vendor-provided materials that are publicly available. In this paper, the impact of negation in the language of this materials is not considered in extracting capabilities. Therefore, any missing information about a device in those materials may affect the effectiveness of our approach.

Overview

Figure 2 illustrates an overview of our approach to extract the capabilities of a smart home device from its specifications. The three steps are described below, and we provide the example of a motion-activated smart camera throughout.

[Step 1: Defining Capability Model for IoT Devices] We first define IoT devices, then define their transducers (e.g., sensors and actuators), and finally define the capability models that map transducers to their set of capabilities. (See Section 4.3). [Step 2: Normalizing Specifications from Vendor Materials] We first extract the device specifications from various vendor materials, then prune the extracted data to eliminate irrelevant contents (e.g., stop words and site navigation links), and finally normalize the pruned contents to refer to the transducer information. (See Section 4.4). [Step 3: Building IoT Device Capabilities] We first build the ontology of the device specifications, then derive an enumeration of transducers for a device by applying this ontology on the processed vendor materials, and finally map these transducers to their capabilities. (See Section 4.5).

Defining Capability Models for IoT Devices

This work defines a transducer as a sensor or actuator (partly inspired by the definitions in NIST 8228 [START_REF] Shen | Considerations for Managing Internet of Things (IoT) Cybersecurity and Privacy Risks[END_REF]). A sensor holds the core functionality of sensing or measuring various aspects of a physical environment and converting it to a digital signal. For example, image sensors, motion sensors, and microphones sense light, motion, and sound from a physical environment, respectively. An actuator converts a digital signal to various physical actions (e.g., emitting light, producing sound, actuating a lock to toggle its state).

The set of all transducers T is partitioned into two sets, where S is the set of all sensors and A is the set of all actuators, where T = S ∪ A and S ∩ A = ∅. A capability of a sensor is denoted as c si and the capability of an actuator is denoted as c aj . Note that, ∀i, j, c si = c aj . However, for any two sensors s m and s n , where m = n and s m , s n ∈ S, their set of capabilities may overlap.

Definition 1. [Transducer]:

A transducer t i is either a sensor s i or an actuator a i . That is, if t i = s i , then t i = a i . Also, if t i = a i , then t i = s i . Each sensor s i and each actuator a i consists of a non-zero finite set of capabilities, denoted as s i = {c s1 , c s2 , . . . , c sp } and a i = {c a1 , c a2 , . . . , c aq }. Also s i = {} and a i = {}. Multiple devices can have common capabilities. For example, a smart camera has an image sensor and therefore holds the capability of capturing an image. On the other hand, a smart light has both light and motion sensors; therefore it holds the capabilities of sensing lights and detecting motion. In the following, the two devices D p and D q have common capabilities c s3 and c a2 :

D p = {c s1 , c s2 , c s3 , c a2 , c a3 }; D q = {c s3 , c s4 , c s5 , c a1 , c a2 }.

Normalizing Specifications from Vendor Materials

To prepare the vendor materials for building device capabilities, we first extract the device specifications from the vendor materials, then remove irrelevant information (e.g., external navigation links or copyright information) from those extracted specifications, and finally refine them into a more homogeneous, and machine-friendly format.

Selective Extractions of Device Specifications. The initial extraction of the device specification is a process that operates on the input vendor materials. The vendor materials must be parsed for their contents, which can be defined in terms of semantics as well as more abstract information such as document structures and page layouts. This work extracts the vendor materials in HTML and/or text formats.

For HTML documents, we first remove the non-HTML contents from each web page, such as style and script blocks. We then extract the raw text from the resulting HTML elements. We parameterize this step so that specific sections of a page can be extracted. For example, the technical specifications page for a smart camera may contain page elements unrelated to the device, such as navigation links, or even additional specification information for similar products (e.g., video doorbells). With our parameterized method, we are able to extract only the specification information for the smart camera. We also tailor the parameters to specific vendor pages; these parameters are often reusable, as web design under a single vendor is often homogeneous.

Normalizing the Data. To ensure that the contents extracted from the vendor materials are best suited for the transducer enumeration technique, our approach normalizes those contents by removing elements that are not critical to the enumeration process, including punctuation, non-alphanumeric, and non-white-space characters, as well as "stop words" (i.e., common articles and prepositions in English). The term case and plurality are also normalized through lemmatization, a process of linguistics that simply involves homogenizing different inflections of the same term to the dictionary form of the term. The content output by the normalization step contains a more homogeneous sequence of terms in the original order that they appeared in the vendor materials.

For example, a motion activated camera's marketing page may contain the following text: "with the camera's array of motion sensors, video will be recorded automatically". After this step, the same string will read "camera array motion sensor video record automatically".

Building IoT Device Capabilities

We now describe how to derive the device capabilities after pre-processing.

Building the Ontology. Our approach understands the language and structure of the vendor materials, and then builds an ontology. The ontology will contain an understanding of the terminologies used to refer to specific components of a device. It is possible to include other information as well that can be derived from the vendor products.

Enumerating Transducers. To enumerate device transducers using the abovementioned ontology, we devise three algorithms, namely, ranked key term set matching (rKTSM), unranked key term set matching (KTSM), and unguided key term set matching, where key terms are one or more words related to a transducer. These algorithms use two types of key terms: indicative terms and related terms. The indicative terms are unambiguously indicative of the presence of a transducer. The related terms are related to a transducer, but may be more ambiguous, and hence are not sufficient in drawing conclusions about its presence. In the case of a motion-activated camera, the term "camera" is considered indicative of an image sensor, while the term "video" is considered related to an image sensor. We describe each algorithm below, in which the notation "x[y]" indicates the mapping or membership of a field y in x.

-Ranked Key Term Set Matching (rKTSM) Algorithm: Algorithm 1 first filters the key term sets for different abstract device types, and then performs key term matching based on the most relevant set of indicative terms. Specifically, Lines 1-11 outline the first matching step, which uses both indicative and related terms to determine which abstract device type is most likely being represented by the corpus of vendor materials. The "get matches" function extracts any matching key terms in the provided set that are contained within the input corpus. The indicative terms of the device type that is ranked as the best match candidate are considered to be the terms that refer to the transducers of the device. A reverse-mapping step is then performed on Line 12, where only these indicative terms are used to determine which transducers are present. Note that the output of the first matching step is a subset of indicative terms for the selected abstract device type. The reverse mapping step determines which transducer each indicative term refers to, where some transducers may be referred to by more than one indicative term. The final results contain an enumeration of transducer identifiers, which are returned at Line 12. Algorithm 1: Ranked Key Term Set Matching (rKTSM) The rKTSM algorithm is able to better exploit context clues found in the related terms while avoiding erroneous matches that can be introduced by their ambiguity. Additionally, the approach accounts for the fact that terms that are most directly indicative of the presence of a transducer may have a frequency that is much lower than that of other terms. For example, even if the indicative term "microphone" appears only twice within the entire corpus, the presence of related terms such as "audio" or "voice" can help bolster confidence when concluding that a microphone transducer is present. -Unranked Key Term Set Matching (KTSM) Algorithm: Algorithm 2 uses only the indicative terms without ranking term sets. Specifically, it evaluates the corpus for any matching indicative terms of any transducer, and identifies the transducers through the same reverse-mapping process (as in Algorithm 1).

Algorithm 2: Unranked Key Term Set Matching (KTSM)

1 all indicative terms ← d d[transducers][indicative terms]
2 ind matches ← get matches(all indicative terms, corpus) 3 transducer identifiers ← reverse map(ind matches)

-Unguided Key Term Set Matching: Additionally, we consider a completely unguided KTSM alrogithm, shown in Algorithm 3, that performs the same matching as unranked KTSM, but also matches on the related terms. We consider this algorithm to be the least focused and exact, as it attempts to match using an entire vocabulary. The transducers that are enumerated from the extraction step must be represented in a standardized way, (e.g., using the same identifier for the transducer type), where each extracted transducer is completely decoupled from the device instance it was extracted from. This is to ensure that the transducers fit into the model that is described in Section 4.3.

Mapping to Device Capabilities. Capabilities of a device are enumerated from its constituent transducers. This work assumes that transducers are associated with a static, finite set of capabilities that are established during the creation of the ontology. Each transducer can be directly mapped to its set of capabilities as per Section 4.3. The capabilities contained in the final output set represent a device's functionality unambiguously. Table 1 provides examples of the outputs of our methodology.

Implementation

We build the ontology of vendor materials for seven smart home products: Arlo Ultra Camera, Nest Cam Indoor, Nest Hello Doorbell, Nest Protect, Nest Learning Thermostat, Nest X Yale Lock, and Ring Indoor Camera [START_REF]Arlo ultra -hd security camera -wireless camera system[END_REF][START_REF]Nest cam indoor -home security camera -google store[END_REF][START_REF]Nest hello video doorbell -know who's knocking -google store[END_REF][START_REF]Nest protect 2ng gen -installation and tech specs -google store[END_REF][START_REF]Nest learning thermostat -installation and tech specs -google store[END_REF][START_REF]Nest x yale lock -key-free smart deadbolt -google store[END_REF][START_REF]Indoor cam -indoor security cameras -ring[END_REF]. Our pre-processing step is implemented (in Python) to fetch the product web pages directly over the network via the requests library [START_REF]HTTP for humans[END_REF] by way of their URI, or to read pages fetched previously and saved locally. To extract the textual content from those pages, we utilize the BeautifulSoup package [11] which allows us to extract only specific sections of vendor material pages by providing parameters with specific HTML tags and attributes used to identify page portions. Our current implementation supports the static elements in web pages (i.e., HTML), which is the current format for most vendor materials. However, if some vendor materials only display the content dynamically, a simple workaround would be to use a web engine to first internally render any dynamic content before processing the resulting HTML. To normalize the text, a separate Python function replaces stop words and non-alphanumeric characters via regular expressions. For the normalization of term plurality, lemmatization is performed using the Spacy natural language processing package [31].

The KTSM algorithms described in Section 4.5 are also implemented as Python functions that take a corpus as input from which to enumerate transducers. To encode the ontology created during the manual review process (as described in Appendix A), a data model is created to represent abstract device types, transducers, their capabilities, and key term sets. Each transducer is represented in the data model as having a static set of capabilities, a set of indicative terms, and a set of related terms. These data models act as additional parameters to our KTSM functions. Given the corpus of a device's vendor materials and the data model, the implemented KTSM functions utilize the tree-based flashtext algorithm [START_REF] Singh | Replace or Retrieve Keywords In Documents at Scale[END_REF] to perform key term matching. In the case of ranked KTSM, the number of matches is used to determine the best abstract device type. In unranked and unguided KTSM, only the matching step takes place with all indicative terms, and with all indicative and related terms, respectively. Any matches are mapped to their associated transducer's identifier automatically (enabled by the Python implementation of flashtext).

Evaluation

This section discusses the performance of our implemented solution, which is used to extract enumerations of transducers for the seven devices. All extractions are performed on a system with an Intel Core i7-8550U processor @ 1.80 GHz and 8 GB of memory. We evaluate the performance of our implementation in terms of its efficiency, the enumeration accuracy for each device, and the proportion of incorrect transducer matches for each device. Efficiency. The goal of the first set of our experiments is to measure the efficiency of our approach. The efficiency of our implementation refers to the total time required to perform the pre-processing step on the input vendor materials for a particular device. Displayed in Figure 3a, the extraction step is the most significant source of processing time in our methodology, ranging from less than a second to nearly 20 seconds.

The time required for the extraction step depends on the size of the vendor materials that are used as input. HTML files for devices may exhibit a large range of sizes; for example, pages on Google Nest devices have the largest range of sizes on disk, from 38 KB to 2.6 MB. The HTML extraction portion of the pre-processing step takes the largest amount of time, between 15 and 18 seconds, for the largest collections of web pages (over 4 MB total). Comparatively, smaller collections of pages (totalling 200 KB or less) take less than a second for extraction. Figure 3a indicates that extraction time scales with the size linearly.

For most evaluated devices, the times required to perform the text normalization step, displayed in Figure 3b, are negligible when compared to those of the extraction step. Consistently, across all devices, the text normalization step is performed in less than a second, increasing slightly with the total number of lemmas extracted. We do not analyze the impact of the automatic enumeration of device transducers from normalized texts, due to the efficiency of the tree-based flashtext algorithm for keyword extraction which we use. Additionally, we do not consider the mapping from enumerated transducers to their capabilities as having any impact on efficiency, due to the fact that this mapping is statically defined in the ontology. That is, once the transducers have been enumerated, establishing their corresponding capabilities requires a trivial lookup in the ontology.

Enumeration Accuracy. For the purposes of evaluation, a sample set of transducers for the evaluated devices are enumerated manually as ground truth. Enumeration accuracy is computed as the ratio between the number of these ground truth transducers that were correctly identified by an extraction approach to the total number of ground truth transducers. Figure 4a displays the results of our enumerations on the seven different devices from three different vendors, grouped by the enumeration algorithms.

Figure 4a shows that both the ranked and unranked KTSM algorithms provide similar proportions of correctly identified transducers, averaging about 60% (b) Fig. 5. A comparison of the proportions of indicative and related terms per document for (a) the Nest Cam Indoor [START_REF]Nest cam indoor -installation and tech specs -google store[END_REF] and (b) the Nest X Yale Lock [START_REF]Nest x yale lock -key-free smart deadbolt -google store[END_REF]. and 62%, respectively, of ground truth transducers identified among all devices with a standard deviation of 24% and 27%, respectively.

For each extraction approach, there is a large disparity between the transducer enumeration rate for the Nest Cam Indoor and the Nest Yale Lock. We present a comparison of these two devices in Figure 5, showing the proportion of indicative and related terms for each document per device. An interesting property that follows from this comparison is the correlation between this property and the enumeration accuracy for each device. As can be seen in Figure 6a, as the proportion of indicative terms grows larger, the overall enumeration accuracy of the rKTSM algorithm generally does as well. This is fairly intuitive, in that a set of vendor materials with a larger number of terms that more explicitly reference a particular transducer will better inform a reader of the association of that transducer with the device. This means that a more refined ontology will contribute to an increased number of indicative terms and improve the enumeration accuracy.

Similarly, Figure 6b displays the negative relationship between proportion of lemmas that are related terms and the transducer enumeration rate. This relationship follows from that of indicative terms and transducer enumeration rate, as a higher proportion of related terms in a corpus is likely accompanied by a lower proportion of indicative terms in the corpus.

Proportion of Incorrect Transducer Matches. To evaluate the tendencies of our algorithms to incorrectly identify transducers, we also measure the proportion of matched transducers that are not within the ground truth set. In other words, we measure the rates at which our algorithms enumerate transducers that are not actually associated with the device.

Figure 4b shows the proportion of incorrect matches. The unguided KTSM approach suffers from the highest proportions of incorrect matches across all devices, with an average of 36% of all transducer matches for each device. Un- ranked KTSM, on the other hand, averages only 3% incorrect transducers, and ranked KTSM does not incorrectly attribute any transducers to any of the new evaluated devices.

The advantage that comes from applying ranked KTSM on texts extracted from vendor materials is the fact that only the indicative terms that are associated with certain devices as defined in the ontology are applied for matching. This ensures that only the most relevant terms with the least ambiguity will be used to draw conclusions about the device's transducers. If ranked KTSM behaves in a way that is too restrictive and fails to enumerate transducers that unranked or unguided KTSM can enumerate, it may be the case that the term sets used for matching are not thorough enough, or a sign that the ontology's representation of the device in question should be improved.

Conclusion and Future Work

With the growing popularity of IoT, the necessity of ensuring its security becomes important than ever. We proposed a proactive approach to extract IoT device capabilities from their design specifications to verify their potential threats even before a device is installed. More specifically, we defined the notion of device capability in the context of IoT, extracted the transducer information for each device using vendor-provided design specifications and finally identified the capabilities of a device.

Our current work relies only on vendor provided materials that are publicly available and may be missing some information. In future, we plan to augment our approach with information from software specifications, device configurations, and firmware versions. Our future work also includes adapting our methodology to other IoT applications including smart grid, smart health, and autonomous vehicle.

ities. The goal of the manual review step is to enumerate the different hardware components and capabilities of a device for a baseline of ground truth, and also to enumerate and analyze the inferences and assumptions that are required for the reader to draw these conclusions. These results are captured and become the basis for the ontology that will be used as a parameter for the automated enumeration process.

The process of manually reviewing vendor materials is the same for all devices. This process involves reading through different documents that are associated with each device and interpreting from them the set of transducers (and capabilities) on the device. During this process, any insights, inferences, or context clues that are used to make this interpretation are also captured. The most important of these features that are key terms that act as indicators of the presence of a particular transducer or capability. In some cases, key terms are only indicative of a transducer or capability when considered in conjunction with another key term. For example, during the manual review of the Nest Cam Indoor's technical specifications on the Google Nest support forums [START_REF]Technical specifications for nest cameras and video doorbells -google nest help[END_REF], simple terms such as "camera" are indicative of the presence of an image sensor. On the other hand, a term like "video" is more ambiguous, and could refer to the video captured by the image sensor, or video displayed on some kind of screen. In this case, it can be "inferred" that this term refers to an image sensor because of context clues provided by additional related terms such as "1080p," which refers to the resolution of the video captured by the sensor, and "lens," which refers to the lens of the camera. On their own, these additional terms do not necessarily suggest the presence of an image sensor, but they can provide context when considered in conjunction with other camera-related terms to suggest with higher confidence the presence of the sensor. The understanding of these terms as context clues carries an assumed level of prerequisite knowledge of camera-related terminology. A sample outcome for the Google Nest Cam Indoor is summarized in Table 2.

The visual cues provided by page structure can also help a reader understand the context around the terms. For the Nest family of products, it is common for the term "temperature" to appear on technical specification pages for devices that have no sensors or actuators related to the measurement or alteration of any temperature. Instead, these instances of the term are used to describe the "operating" constraints of the device (the theoretical range of temperature in which it can operate). The only real indicator of this difference is in the table Fig. 7. Different sections of the Nest Cam Indoor technical specifications page [START_REF]Technical specifications for nest cameras and video doorbells -google nest help[END_REF], where the terms "temperature" and "humidity" can be seen multiple times, but only in reference to the theoretical temperature range in which the device can regularly operate and be stored. layout of the Nest Cam Indoor's technical specifications page, where a reader can see by way of the row's label "operation" that the temperature in this case refers to the device's operating temperature and not any sensor. This part of the page is illustrated in Figure 7.

Perhaps the most abstract feature that is considered during manual review, which is largely dependent on the individual reviewer, is the use of technical background knowledge to infer, from a described concept, how a certain feature of a device may be implemented. The term "motion detection," for example, could indicate the presence of a motion sensor or of software that enables an image sensor to perform motion detection. Regardless, the ability for the reviewer to conceive of these possibilities is dependent on their technical background knowledge.

Fig. 1 .

 1 Fig. 1. (a) The term frequencies for the corpus of vendor materials on the Nest Cam Indoor, visualized as a word cloud. (b)The term frequencies for a subset of terms from vendor materials on the Nest Cam Indoor. Terms that are more directly related to the transducer they refer to appear in blue, while other terms appear in red.

Fig. 2 .

 2 Fig.2. An overview of our methodology, including 1) development of the model, 2) preprocessing of vendor materials, and 3) applying an ontology to extracted specifications by way of an automatic enumeration function.

Definition 2 .

 2 [Device:] A device is an embedded system that consists of a set of transducers. A device D i consists of a set of sensors S i and actuators A i where S i ⊆ S and A i ⊆ A and S i = {s 1 , s 2 , . . . , s n } and A i = {a 1 , a 2 , . . . a m }. The number of transducers in D i equals n + m. Multiple devices may have common sensors or actuators. For example, smart cameras and smart video doorbells both have a camera sensor. Two devices D r and D s shown below have common sensor s 3 and common actuator a 2 . D r = S 1 ∪ A 1 = {s 1 , s 2 , s 3 , a 1 , a 2 }; D s = S 2 ∪ A 2 = {s 3 , s 4 , a 2 , a 3 }. Definition 3. [Device Capability]: A capability of device D i is a function the device can perform. The set of capabilities for D i is computed as the union of the set of capabilities of the transducers comprising the device.

Algorithm 3 :

 3 Unguided Key Term Set Matching 1 all indicative terms ← d d[transducers][indicative terms] 2 all related terms ← d d[transducers][related terms]3 all terms ← all indicative terms + all related terms 4 all matches ← get matches(all terms, corpus) 5 transducer identifiers ← reverse map(all matches)

Fig. 3 .

 3 Fig. 3. Average text extraction time of the vendor materials for different devices by their size on disk, computed over 5 trials. (b) Average text normalization time by the total number lemmas in a corpus, computed over 5 trials.

Fig. 4 .

 4 Fig. 4. For each device, grouped by KTSM algorithm (a) the proportion of ground truth transducers that are correctly enumerated, and (b) the proportion of matching transducers that are incorrectly identified.

Fig. 6 .

 6 Fig. 6. The proportion of corpus terms that are (a) indicative and (b) related correlated with transducer enumeration rate of ranked KTSM.

Table 1 .

 1 An excerpt of outputs from our approach.

	Device	Category	Transducer	Atomic Capabilities
			Image Sensor	Capture image, Capture video, Detect
		Sensors		light
			Microphone	Capture sound
	Arlo Ultra [1]		Motion Sensor	Detect motion
			Speaker	Produce sound
		Actuators	LED Light Infrared Light	Produce light Produce IR light (enabling night vision)
			Siren	Produce high-volume siren
		Sensors	Image Sensor	Capture image (take photo), Capture video, Detect light
	Nest Cam Indoor [5]		Microphone	Capture sound
			Speaker	Produce sound
		Actuators	LED Light	Produce light
			Infrared Light	Produce IR light (enabling night vision)
			Smoke Sensor	Detect smoke
			Carbon Monoxide	Detect carbon monoxide
			Sensor
	Nest Protect 2nd Gen [8]	Sensors	Temperature Sensor Humidity Sensor Microphone	Measure temperature Measure humidity, detect steam Capture sound
			Motion Sensor	Detect motion
			Light Sensor	Detect light
		Actuators	Speaker LED Light	Produce sound Produce light
		Sensors	Light sensor Touch Sensor	Detect light Detect (capacitive) contact
	Nest X Yale Lock [9]		Lock	Lock and unlock door
		Actuators	Speaker	Produce sound
			LED Light	Produce light

  Mean Extraction Time by Input Size

				Mean Text Normalization Times
	Nest Thermostat		0.6	Arlo Ultra
	Nest Doorbell			Nest Cam Indoor
	Ring Indoor Camera Nest Protect 2nd Gen Nest Yale Lock Arlo Ultra Nest Cam Indoor	Normalization Time (Seconds)	0.1 0.2 0.3 0.4 0.5	Nest Doorbell Nest Protect 2nd Gen Nest Thermostat Nest Yale Lock Ring Indoor Camera
				2.5	5.0	7.5	10.0	12.5
				Total Number of Lemmas
	(a)

Table 2 .

 2 Obtained ontology for the Nest Cam Indoor

	Category Transducer	Capabilities
	Sensors	Image Sensor Capture Image, Capture Video, Detect Light Microphone Capture Sound
		Speaker	Produce Sound
	Actuators	LED Light	Produce Light
		Infrared Light	Produce Infrared Light

Acknowledgement

The authors thank the anonymous reviewers for their comments. This work was funded in part by NIST under Contract Number 60NANB18D204, by funds from NSF under Award Number CNS 1650573, CNS 1822118, and funds from AFRL, SecureNok, Furuno Electric Company, and CableLabs. We would also like to thank Upakar Paudel for help with code development.


Y J Meijaard
email: y.j.meijaard@student.tue.nl

B C M Cappers
email: b.c.m.cappers@tue.nl

J G M Mengerink
email: j.mengerink@analyzedata.com

N Zannone
email: n.zannone@tue.nl

Predictive Analytics to Prevent Voice Over IP International Revenue Sharing Fraud

International Revenue Sharing Fraud (IRSF) is the most persistent type of fraud in the telco industry. Hackers try to gain access to an operator's network in order to make expensive unauthorized phone calls on behalf of someone else. This results in massive phone bills that victims have to pay while number owners earn the money. Current anti-fraud solutions enable the detection of IRSF afterwards by detecting deviations in the overall caller's expenses and block phone devices to prevent attack escalation. These solutions suffer from two main drawbacks: (i) they act only when financial damage is done and (ii) they offer no protection against future attacks. In this paper, we demonstrate how unsupervised machine learning can be used to discover fraudulent calls at the moment of their establishment, thereby preventing IRSF from happening. Specifically, we investigate the use of Isolation Forests for the detection of frauds before calls are initiated and compare the results to an existing industrial post-mortem anti-fraud solution.

Introduction

International Revenue Sharing Fraud (also known as Toll fraud) is a multi-billion dollar scheme [START_REF] Gibson | Europol Cyber Fraud Intelligence[END_REF] where fraudsters use telecommunications products or services without the intent to pay. With Voice Over IP (VoIP) enabling devices to communicate over the Internet, VoIP devices and services are a popular target for hackers to abuse. The result is typically phone bills of thousands of euros [START_REF] Pelroth | Phone hackers dial and redial to steal billions[END_REF] that victims have to pay. In practice, network operators often covers these expenses either because they were responsible for the vulnerability or they want to maintain customer satisfaction. To minimize this fraud, they often install anti-fraud detection software to detect signs of fraud as quickly as possible. Current anti-fraud solutions typically use pricing information and features such as call duration to determine whether a call was fraudulent. Although such detection turns out to be accurate [START_REF] Becker | Fraud detection in telecommunications: History and lessons learned[END_REF], those features are only available after the call has been made. Since the detection of a fraudulent call does not provide any information about the severity of the next call, operators are often forced to shutdown the phone device or underlying VoIP server (e.g., a Private Branch eXchange) until they are sure that the call was legitimate. The result is that customers can no longer make (regular) phone calls until the fraud has been resolved.

The goal of this work is to explore to what extent fraud can be detected when a call is established, compared to existing industrial (post-mortem) anti-fraud solutions. To this end, we study how we can use unsupervised machine learning solutions and, in particular, Isolation Forests [START_REF] Liu | Isolation forest[END_REF] to determine the severity of a phone call using features that are only available at the start of a call. In contrast to traditional anomaly detection algorithms (see [START_REF] Chandola | Anomaly detection: A survey[END_REF] for a survey), Isolation Forests enable the detection of anomalies without having to construct profiles of normal behavior. This makes the algorithm particularly suitable for highly dynamic environments where construction of normal profiles can be too resource intensive. In addition, with linear time classification Isolation Forests have also shown to be suitable for the analysis of large data streams [START_REF] Ding | An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window[END_REF]. However, in order for machine learning solutions to be applied for fraud detection, the number of false alarms need to be kept to a minimum. This typically requires properly tuning such solutions and underlying parameters, which is a non-trivial task [START_REF] Friedman | There's no such thing as a free lunch[END_REF].

In this work, we perform a latitudinal study to assess the detection capabilities of Isolation Forests in the context of IRSF detection and to study how different parameter settings and feature spaces affect performance. Specifically, our main contributions are:

the application of Isolation Forest anomaly detection for the early-stage detection of IRSF;

a case study of the approach demonstrating the effect of different parameter settings, feature sets, and the use of derived features to improve detection rates;

a comparison of the detection with respect to existing post-mortem analysis;

a number of lessons learned on how to use Isolation Forests for the detection of anomalies in multivariate data.

Our approach offers several benefits compared to existing anti-fraud solutions. In particular, it enables network operators to detect signs of fraudulent calls before they are established, providing operators the opportunity to block the call preemptively rather than blocking a phone device entirely when the fraud has happened.

The remainder of the paper is structured as follows. The next section introduces background on VoIP and related frauds. Section 3 discusses related work. Section 4 presents our methodology and Section 5 presents its experimental evaluation. Finally, Section 6 discusses the results along with the limitations of the approach and Section 7 concludes the paper and provides directions for future work.

Background & Motivations

Voice over IP (VoIP) enables users to communicate audio and video over the Internet. Compared to physical phone lines, the Internet provides a cheaper alternative and is widely available. The VoIP infrastructure consists of four main types of components: VoIP devices, Private Branch eXchanges (PBX's), the Public Switched Telephony Network (PSTN), i.e. the legacy phone infrastructure, and VoIP gateways. A VoIP device connects over the Internet to a PBX operated by a network operator, which in turn connects to other PBX's. Over this network, a VoIP device is able to call other VoIP devices. VoIP gateways connect the VoIP network to the PSTN to enable VoIP devices to call legacy phones.

Over the years, several vulnerabilities in VoIP and underpinning network protocols have been discovered. Sahin et al. [START_REF] Sahin | Sok: Fraud in telephony networks[END_REF] define a taxonomy of VoIP fraud schemes in which these schemes are categorized with respect to their root causes along with their weaknesses and techniques to exploit them. Given that IRSF has been recognized as the largest class of frauds in practice [START_REF] Gibson | Europol Cyber Fraud Intelligence[END_REF], in this work we focus on the detection of this class.

In IRSF an attacker breaks into the VoIP system of the victim and places numerous calls to a premium phone number, i.e. a number that charges a fee in addition to the regular cost of the call, which is owned by the attacker or a colluding entity. These calls are charged to the victim and the revenue made from this call is shared amongst the attackers. Blacklisting premium numbers is virtually impossible, due to them being ill-defined internationally [START_REF] Sahin | Sok: Fraud in telephony networks[END_REF]. Therefore, network operators need methods for the detection of IRSF. Ideally, these methods should be able to detect the fraud when the call is established to prevent it from happening.

Real-time anomaly detection in telco industry, however, is challenging due to the variety and volumes of data [START_REF] Elagib | CDR analysis using Big Data technology[END_REF]. Depending on the type of users, the number of calls can vary from tens to hundreds per month. In addition, data volumes are often too large to analyze all network packets individually. As a result the use of profiling techniques can be too computational intensive.

In order for anomaly detection to be effective in this field, the evaluation of a data point needs to be reliable and efficient. False positives need to be kept to a minimum to avoid operators from being overloaded with false alarms. In particular, the maximum allowable false positive rate should be <2% [START_REF] Kübler | Toll Fraud Detection in Voice over IP Networks Using Behavior Patterns on Unlabeled Data[END_REF]. The false negative ratio (i.e., the number of missed fraud) is less critical here. We wish to catch as much fraud beforehand as possible, but missed fraud cases could still be detected by a post-mortem detector. In addition, some features such as the call duration and cost are only available after the call has been made. Most data fields in VoIP calls consists of categorical features. In order to enable real-time detection, the evaluation of a new data point and model updating must be efficient [START_REF] Ahmad | Real-time anomaly detection for streaming analytics[END_REF].

In summary, in order to make the detection of IRSF during call establishment effective, the anomaly detection method should meet several requirements. Specifically, a solution for online IRSF detection should: R1 Be resilient to ill-balanced label distribution (e.g., 99% normal and 1% fraud). R2 Support the analysis of categorical features. R3 Enable fast classification of (new) individual calls. R4 Provide a computational efficient method for training the classification model. R5 Generate an operationally feasible number of false positives. R6 Not rely on call features that are only available after the end of a call.

In the next sections we discuss to what extent existing solutions meet the requirements and show how Isolation Forests can be used to solve the detection task.

Related Work

Fraud detection is an extensively studied field covering a wide variety of techniques [START_REF] Kou | Survey of fraud detection techniques[END_REF]. We first give a broad overview on data analysis techniques used to detect IRSF, followed by a detailed discussion on existing anti-fraud solutions that are based on the analysis of Call Detail Record (CDR) logs. An evaluation of existing solutions with respect to the requirements identified in Section 2 is given in Table 1. For a more detailed overview on the use of anomaly detection techniques for the detection of telecom fraud, we refer to [START_REF] Ighneiwa | Bypass fraud detection: Artificial intelligence approach[END_REF]. Wiens et al. [START_REF] Wiens | A new unsupervised user profiling approach for detecting toll fraud in VoIP networks[END_REF] Kuble [START_REF] Kübler | Toll Fraud Detection in Voice over IP Networks Using Behavior Patterns on Unlabeled Data[END_REF] Olszewski [START_REF] Olszewski | Employing self-organizing map for fraud detection[END_REF] Wiens et al. [START_REF] Wiens | Improvement of User Profiling, Call Destination Profiling and Behavior Pattern Recognition Approaches for Telephony Toll Fraud Detection[END_REF] Becker et al. [START_REF] Becker | Clustering anonymized mobile call detail records to find usage groups[END_REF] Fraud Detection. Similar to intrusion detection [START_REF] Etalle | From intrusion detection to software design[END_REF], approaches for the detection of (IRSF) fraud can be classified in two main categories, namely signature-based detection and behavioral-based detection [START_REF] Burge | Detecting cellular fraud using adaptive prototypes[END_REF]. Signature-based detection uses fixed rules to identify whether a call adheres to predefined patterns. For instance, a suddenly increase in a user's dial expenses compared to his average phone bill provides an indicator for IRSF. For more details on the typical signatures used for fraud detection, we refer to [START_REF] Ferreira | Establishing fraud detection patterns based on signatures[END_REF]. Although the use of signatures provides an effective approach to fraud detection, it is unable to discover new patterns enabling the detection of IRSF when, for instance, the cost of the call is unknown. These shortcomings are typically addressed by behavioral-based detection techniques. These techniques focus on the (statistical) profiling of entities so that the regularity of new data points can be determined according to a model of expectation derived from historical data. This enables analysts to automatically derive models that are tailored on a per profile basis. In case of IRSF detection, we can identify two main types of behavioral-based techniques, namely offline and online [START_REF] Cappers | Interactive visualization of event logs for cybersecurity[END_REF][START_REF] Widmer | Learning in the presence of concept drift and hidden contexts[END_REF]. Offline behavioral-based techniques (also referred to as post-mortem detection) stores incoming data and do the analysis after all calls have been made. Online techniques (a.k.a. (near)-real time analysis) perform the analysis the moment new data arrives (e.g., during call establishment).

Our work can be positioned as online behavioral-based method. In particular, we investigate the possibilities of applying online anomaly detection at the start of a call, enabling the early stage detection of IRSF. This is in contrast to current literature, which mainly focuses on the offline analysis of calls by applying anomaly detection on Call Detail Record (CDR) logs [START_REF] Peterson | Business telecom systems: A guide to choosing the best technologies and services[END_REF]. Next, we review those solutions.

CDR-based fraud analysis.

A common approach in the telecommunication domain for offline behavioral-based fraud detection is through the analysis of Call Detail Records (CDR). For instance, Modani et al. [START_REF] Modani | CDR Analysis Based Telco Churn Prediction and Customer Behavior Insights: A Case Study[END_REF] use decision trees and logistic regression to predict the churn rate of companies in CDR records whereas Wiens et al. [START_REF] Wiens | Improvement of User Profiling, Call Destination Profiling and Behavior Pattern Recognition Approaches for Telephony Toll Fraud Detection[END_REF][START_REF] Wiens | A new unsupervised user profiling approach for detecting toll fraud in VoIP networks[END_REF] apply statistical profiling on user call behavior to detect exploited FRITZ!Boxes in a network. The use of anomaly detection techniques are also shown to be useful for the discovery of unknown patterns in call records. For instance, Becker et al. [START_REF] Becker | Clustering anonymized mobile call detail records to find usage groups[END_REF] are one of the first to use unsupervised learning on CDRs to discover cellphone usage patterns. CDRs have also been used to discover relationships between criminals based on the assumption that criminals interact with each other. Specifically, Kumar and colleagues propose a model to construct a CDR database in which these relationships are captured [START_REF] Kumar | Crime investigation and criminal network analysis using archive call detail records[END_REF]. Kübler et al. [START_REF] Kübler | Toll Fraud Detection in Voice over IP Networks Using Behavior Patterns on Unlabeled Data[END_REF] analyze toll fraud hindsight by clustering user behavior using unsupervised learning techniques such as k-means clustering and EM mixture models [START_REF] Alldrin | Clustering with EM and K-means[END_REF]. Instead of using cost-based features, they build user-profiles based on destination number and duration of the call. Although this approach to discover behavioral patterns through feature engineering is similar to ours, it requires the construction of user profiles to determine the severity of a call. In addition, the clustering algorithms proposed in [START_REF] Kübler | Toll Fraud Detection in Voice over IP Networks Using Behavior Patterns on Unlabeled Data[END_REF] are too computational intensive to be used in online settings. Especially in dynamic environments where users register and leave phone operators on a regular basis, building "normal" profiles can be too resource intensive to be applied in practice. The advantage of using Isolation Forests is that this technique does not require a normal baseline to determine the severity of an anomaly, but aims to isolate "few and different" points from the rest of the data [START_REF] Liu | Isolation forest[END_REF].

In summary, current approaches for fraud detection are signature-based or rely on an offline behavioral-based analysis of calls using CDR logs. The absence of offline features such as cost and duration in online detection requires adaptation of existing fraud detection techniques both in terms of the feature space to analyze as well as computational requirements. Our work overcomes these limitations by using Isolation Forests.

Methodology

The goal of the methodology is to explore to what extent we are able to detect fraudulent IRSF calls before they are established, giving operators the opportunity to block the call and prevent the fraud from happening.

We enable online IRSF detection through a real-time analysis of the network traffic from and to Private Branch eXchange servers to identify suspicious patterns in the establishment of VoIP calls. The establishment of a VoIP call is achieved by means of a handshake using the Session Initiation Protocol (SIP) [START_REF] Rosenberg | SIP: Session initiation protocol[END_REF]. This protocol contains call meta-data similar to Call Detail Records such as call start time, source and destination numbers, and user identifier. In addition, more advanced features can be derived such as the country of origin or in/outside office hours (we refer to Section 5.4 for more details). Since the analysis is done during call establishment, we assume that features such as call duration, cost of a call and call end time are unavailable.

Our methodology for online detection of IRSF comprises three steps, as depicted at the top of Figure 1: 1. From the VoIP data we extract the features to use for training of the isolation forest. 2. We generate an isolation forest from the training data, which assigns an anomaly score to each call. 3. We classify all calls with an anomaly score lower than a given threshold as anomalies.

Anomalous records are marked as fraud while the other are marked as normal.

To validate experimental results, we use an industry post-mortem fraud detector as a baseline. Validation comprises three steps:

5. After the call ended, a CDR record is created by the underpinning PBX server. 6. The industrial anti-fraud detector labels the call records either as fraud or normal. 7. The evaluation phase compares the labels from our approach with the ones given by the industrial detector and generates a performance report.

Start of call

Train forest

Call Detail Record

In the remainder of the section, we first introduce Isolation Forests after which we discuss the dataset used for our experiments.

Isolation Forests

Traditional anomaly detection techniques typically construct a profile of the data that is considered "normal" and evaluate any new data point against that model [START_REF] Chandola | Anomaly detection: A survey[END_REF]. Isolation Forests use a different approach as it aims to separate anomalous data points in the dataset without building a distribution or profile. Instead, data is sub-sampled in a tree structure by evaluating data points on randomly chosen features and split on those features. Specifically, if a node contains two or more records, it is split according to a randomly chosen feature. This causes an anomalous data point to reside in their own leaf node. This is also illustrated in Figure 2. The main idea is that similar data points require more splits before they can be separated from one another while anomalies remain close to the root. Since purely random splits can lead to poor decision trees, data points are evaluated against a collection of generated trees to determine whether they are statistically significantly different from the rest.

The anomaly score of a data point is based on its average depth in all isolation trees. A high score indicates that many splits are required in order to separate a data point from the rest and is therefore considered to be similar to other records. Similarly, if the score is low, only few splits are required to separate the data point and, therefore, it is considered different from the other data points. A user defined threshold is used to determine whether a score is low or high, i.e. whether a data point should be considered anomalous.

In our work, we evaluate the following parameters for the generation of Isolation Forests (illustrated in Figure 3):

-nrTrees represents the number of trees in the forest.

-sample_size represents the size of the sample set. The sample size determines the number of records for the construction of a tree in the forest. -max_depth represents the maximum depth for each isolation tree. Fig. 3. Parameters in isolation forests manipulate the forest by specifying the number of trees, number of nodes per tree, the maximum depth of generated trees along with the number of samples that must be used for the construction of a tree. The dot in red is anomalous with respect to the chosen threshold.

threshold is a user-defined threshold that determines the average tree depth that should be exceeded in order for a data point to be considered normal. -feature_set represents the set of features used for the construction of the trees in the forest.

Isolation Forests provide several advantages compared to other machine learning techniques. Compared to black-box techniques such as neural networks, Isolation Forests are transparent. By inspecting the different trees, a VoIP operator is able to provide an explanation on why a data point was considered anomalous, thereby enabling means to judge the quality of a model using domain knowledge. Adaptations of Isolation Forests have been proposed to make them suitable for streaming data [START_REF] Tan | Fast anomaly detection for streaming data[END_REF]. In contrast to other unsupervised learning methods such as k-means and EM-Mixture models [START_REF] Alldrin | Clustering with EM and K-means[END_REF], the construction of the forest is computationally efficient, since distances between data points are not computed, but features and data splits for the trees are chosen randomly. In addition, the classification of a new data point requires at most O(max_depth) steps per tree and can be run in parallel for every tree in the forest. Compared to supervised learning techniques such as decision tree classifiers and SVM, Isolation forest do not require labels to identify anomalous data points.

Data characteristics

We performed a number of experiments to assess the effectiveness of Isolation Forests for the online detection of IRSF. Our dataset consists of over 10.000 VoIP calls, out of which the industrial anti-fraud solution marked 150 calls as fraudulent. The dataset consists of nine Dutch users where a user can represent a physical person, a server, or an entire phone operator. The data was recorded for a month by mirroring network traffic from a Private Branch eXchange server of a Dutch VoIP provider. Each call is characterized by the following online (i.e., pre-call) features:

-record_id is the identifier of the VoIP record.

-user_id is the identifier of the user that made the call.

-srcNr is an anonymized representation of the source number.

-srcCtry represents the country of call source.

-dstNr is an anonymized representation of the destination number.

-dstCtry represents the country of the call destination.

disposition indicates whether the call was answered, canceled or busy.

time is a timestamp indicating when call took place.

To test the suitability of Isolation Forests for the detection of IRSF in general, we also collected the following offline (i.e., post-call) features:

duration indicates how long the call lasted in seconds.

billsec indicates the cost of the call per second.

We performed an analysis of the dataset to verify that it does not contain any artefacts that could disqualify the experiments. For the sake of space, here we only report our analysis with respect to features time and dstCtry. As shown in Figure 4, calls are typically made during work hours whereas fraud cases mainly occur outside of work hours. We can observe in Figure 5 that the destination of the calls is predominantly to the Netherlands, but there is a wide variety of calls to different countries. The plot also shows that simply blocking calls to certain countries is not a viable solution to fraud prevention, as it would incorrectly block at least as much normal traffic.

Experiments

This work aims to evaluate the effectiveness of Isolation Forests for the detection of IRSF. In particular, we are interested in answering the following research questions: Q1 Are Isolation Forests suitable for offline and online detection of IRSF ? Q2 What is the effect of different parameter settings towards detection rates? Q3 What is the effect of different feature sets towards detection rates? Q4 To what extent does the use of derived features improve detection rates? Questions Q1 and Q2 validate whether Isolation Forests can be used in general for IRSF detection. Q3 aims to determine which features in the input data should be taken into account during classification. The naïve usage of features such as time or categorical features can significantly blow up the state space in which the algorithm needs to operate and can lead to sub optimal results [START_REF] Dong | Feature engineering for machine learning and data analytics[END_REF][START_REF] Indyk | Approximate nearest neighbors: towards removing the curse of dimensionality[END_REF]. Similar to Kübler et al. [START_REF] Kübler | Toll Fraud Detection in Voice over IP Networks Using Behavior Patterns on Unlabeled Data[END_REF], in Q4 we test the effect of discretizing the time feature to see how this influences detection rates.

For the implementation of Isolation Forests we used the h2o framework (version 3.26.0.10) [START_REF] Candel | Deep learning with h2o. H2O[END_REF]. This framework offers out-of-the-box support for categorical data and is commonly used in academia to study machine learning techniques.

Settings.

In our experiments, we tested different configurations for the generation of Isolation Forests by varying the parameters presented in Section 4.1. An overview of the parameters for each experiment is given in Table 2. It is worth noting that increasing nrTrees does not influence the outcome of the model, since it is statistically unlikely to generate significantly different anomaly scores when averaging over 100 trees given the feature set of VoIP calls. Therefore, the recommended default settings for this parameter is used, i.e. 100. In Experiments 1, 3, and 4 we set parameter sample_size to 256, which we deem sufficiently large. This choice is further motivated by Experiment 2 in which we varied this parameter to study its effect on detection rates.

The choice for parameters max_depth and threshold is based on two observations: Isolation Forests are unable to classify anomalies from normal behaviour with a small max_depth, e.g., ≤ 5. On the other hand, a large max_depth (e.g., ≥ 15) can result in problematic overfitting, as mentioned in the original paper [START_REF] Liu | Isolation forest[END_REF]. Simultaneously, the value of threshold cannot exceed the max_depth value and, therefore, its domain is restricted. Specifically, we varied threshold in the range [5,max_depth], with steps of 0.1.

Evaluation metrics.

In general, the evaluation of an unsupervised learning algorithm is a challenge due to the lack of labeled data or a proper baseline. Inspired by Wang et al. [START_REF] Wang | CVAP: validation for cluster analyses[END_REF] and Dudoit et al. [START_REF] Dudoit | A prediction-based resampling method for estimating the number of clusters in a dataset[END_REF], we measure the quality of the resulting model by means of an external index using the labels of the existing anti-fraud solution as the baseline.

The most common evaluation metric to assess the performance of a machine learning algorithm is the Receiver Operating Characteristics (ROC) curve [START_REF] Hanley | The meaning and use of the area under a receiver operating characteristic (ROC) curve[END_REF]. In this curve the true positive rates and false positive rates are plotted for an algorithm at various thresholds (i.e., the Isolation Forest threshold parameter). In our application domain, the goal is to have a curve where the true positive rate is high and false positive rate is close to 0. The Area Under the Curve (AUC) is an indicator how well the algorithm can discriminate between normal and fraudulent traffic. The AUC can vary between 0.0 and 1.0 and the larger the area, the better the performance.

Experiment 1: Effectiveness of Isolation Forests for IRSF detection

In this experiment, we assess whether the classification capabilities of Isolation Forests are suited to fraud detection in VoIP. We test this by applying Isolation Forests to both an online and an offline setting. Specifically, we conduct two tests: one using all data available at the establishment of the call (e.g., pre-call features) and one also including post-call features (cf. Section 4.2). Based on R5, we require that the false positive rate is less than 2%, while the true positive rate should be sufficient to outweigh operational costs.

The full construction of an isolation forest for the dataset is efficient and takes approximately 1 second (R4). Figure 6 reports the ROC curves when using pre-call and post-call features. The figure shows that, at a 2% false positive rate, the true positive rate obtained using pre-call features is insufficient. However, in the post-mortem setting we can detect up to 58% of all fraud cases. At a false positive rate of 5%, we can detect up to 33% of all fraud cases in the pre-call setting, compared to 82% in the post-call setting. The true and false positive rates for the post-call features are comparable to earlier work by Kübler [START_REF] Wiens | A new unsupervised user profiling approach for detecting toll fraud in VoIP networks[END_REF] and shows that, with respect to requirements R1 and R5, Isolation Forests have potential for the detection of IRSF. Enabling detection of IRSF using only pre-call features, however, requires additional tuning in order to meet the requirements.

Experiment 2: Effect of parameter settings

This experiments is conducted to evaluate the effect of parameters max_depth and sample_size on the detection capabilities of Isolation Forests. To this end, we first computed the ROC curves for each value of max_depth while varying parameter threshold with steps of 0.1 and fixing the sample_size parameter to 256. We also computed the ROC curves for each value of sample_size while keeping the max_depth parameter fixed to log(sample_size) and varying the threshold with steps of 0.1. For this experiment, we used pre-call features. Figure 7 reports the results of the experiment.

From the plot at the top of Figure 7, we can observe that, when max_depth is lower than 7, the detection rate drops significantly. We believe this is twofold. First, if the max_depth becomes smaller than the number of features, there is an increased risk of missing crucial feature splits during the tree generation phase of the algorithm. Second, in Section 4.1 we showed that anomalous data points occur alone in the leafs of an isolation tree. With a max_depth of 6, at most 2 6 = 64 anomalous data points can be detected. Given that the dataset includes over 100 cases marked as fraud, the size of the tree is insufficient to assign every anomalous point to an empty leaf node. As a consequence, it is more likely for two inherently different data points to end up in the same leaf node, which leads to an overgeneralization of the resulting model.

A second observation is that increasing the max_depth does not significantly affect performance. By default, Liu et al. [START_REF] Liu | Isolation forest[END_REF] recommend max_depth = log 2 (sample_size) to obtain a balanced tree. In our experiments this recommended value would correspond to log 2 (256) = 8. The idea of increasing the depth would be to give the algorithm more slack to add additional splits to better discriminate between normal and fraudulent points. Results show, however, that for max_depth = 8 the number of splits is already sufficient to identify the anomalies.

The plot at the bottom of Figure 7 shows that the increase of parameter sample_size does not lead to any significant improvement of the performance. This illustrates that taking a sample size of 256 is sufficiently large to obtain a data subset which is representative for the generation of one tree.

Experiment 3: Effect of different features

We conducted this experiment to investigate the impact of including certain features on the effectiveness of Isolation Forests for fraud detection. We compare different sets of features and measure the false positive and true positive rates. In particular, we consider the following feature sets: 1. Pre-call, no time: record_id, user_id, srcNr, srcCtry, dstNr, dstCtry, disposition, which allows us to study the influence of the time of the call on the detection rate. 2. Pre-call, no user: record_id, srcNr, srcCtry, dstNr, dstCtry, time, disposition, which allows us to study the influence of the user identity on the detection rate. 3. Pre-call, no disposition: record_id, user_id, srcNr, srcCtry, dstNr, dstCtry, time, which allows us to study the effect of feature disposition on the detection rate.

We use the pre-call ROC curve from Experiment 1 as the baseline for the comparison. If a feature set performs as the pre-call feature set or better, then we conclude that the feature set is effective and the removed feature is not relevant for fraud detection; otherwise the feature set is not effective and the removed feature should be used for the classification of calls.

Figure 8 shows the result of the experiment. The removal of feature disposition has a slightly positive effect on the detection rate, as is the effect of removing user_id. A possible explanation for this could be that both features are not heavily correlated with fraud. Removing time has a negative effect on the detection rate, which coincides with the observation in Section 4.2 that the time of the call is a key factor for fraud detection. In the next experiment, we investigate how we can derive additional features from time.

Experiment 4: Effect of derived features

In this experiment we explore how to improve detection rates by deriving additional features from the call's start time field. We extract three kinds of features, namely: Specifically, the curve obtained using all pre-call features is compared to the curves obtained when additionally to the pre-call features, features OfficeHours, HourOfDay or FocusHourOfDay are included.

1. HourOfDay: 24 Categorical features where every feature corresponds to a one hour interval (24 hours notation). 2. OfficeHours: A Boolean feature that indicates whether the call has happened between 09:00-17:00. 3. FocusHourOfDay: These features are similar to HoursOfDay but in this case only the hours just before 09:00 and just after 17:00 are considered, specifically, in the interval [START_REF] Burge | Detecting cellular fraud using adaptive prototypes[END_REF][START_REF] Ding | An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window[END_REF] and [START_REF] Hanley | The meaning and use of the area under a receiver operating characteristic (ROC) curve[END_REF][START_REF] Kou | Survey of fraud detection techniques[END_REF].

These features were introduced to exploit domain knowledge in the classification of calls. From the analysis of the dataset in Section 4.2 we observed that most fraud cases occurred outside office hours. We attempted to capture this information is several different ways. Initially, we only tried to capture this in a Boolean feature representing the office hour time-interval, i.e. OfficeHours. We stretched the idea of a Boolean feature per time interval all the way to including such a feature for every hour of a day. Finally, we focused this broad set of features to only including the time-intervals we know are correlated with fraud.

We applied Isolation Forests to the aforementioned feature spaces and compared the results to pre-call results in Figure 6. Specifically, in each experiment we consider the pre-call data set and replace the time feature with one of the derived features. The time feature is replaced to isolate the effect of including the derived features. The results are given in Figure 9.

We can observe that the features HourOfDay and FocusHourOfDay significantly improve the detection rates. In particular, with the latter we have a true positive rate of 10. Summary of the results. Specifically, the ROC curve for fraud detection using all pre-call features is compared to the ROC curve obtained using the feature FocusHourOfDay.

over 45%, at a false positive rate of 2%, which is comparable to the results obtained using post-call features. A possible explanation for this is that by increasing the number of features drastically, the chance of an isolation forest splitting on time increases. Since time is correlated with fraud, this could explain the performance increase.

Discussion and limitations

The experiments in Section 5 show that with careful choice of the parameters and feature space tuning, Isolation Forests provide a reliable tool for fraud detection. Figure 10 shows the false positive and true positive rates of Isolation Forests before and after the proposed optimizations as described in Experiments 1, 2, 3, and 4. Here, we can observe that at least 45% of the fraud can be detected reliably using the pre-call feature set at the cost of a 2% false positive rate and up to 87% at the cost of a 5% false positive rate. Given that the damage of such phone calls can be in the order of thousands of euros per call [START_REF] Fgsservices | Are you at risk from Toll Fraud?[END_REF][START_REF]Evety Business needs to know Toll Fraud and VoIP[END_REF][START_REF] Phithakkitnukoon | VoIP Security-Attacks and Solutions[END_REF][START_REF]Business At Risk of Toll Fraud[END_REF], the discovery of a few fraudulent calls is already sufficient to cover operational expenses.

The introduction of time based derived features improves the detection and coincides with the observation that most of the fraud cases happened outside office hours. Experiment 2 has shown that decreasing max_depth does not yield a significant increase in performance. This is desirable since it gives less branching possible and clusters tend to become larger.

With respect to the requirements presented in Section 2, our evaluation shows that Isolation Forests are resilient to ill-balanced label distribution (R1) while enabling the support of categorical features with over 50 values per features (R2). Isolation Forests have a linear time complexity with a low constant and a low memory requirement which is ideal for high volume datasets (R3). Although the training of the forest was a matter of seconds, a larger data sample is needed to test whether requirement R4 is fully satisfied. Experiments have also shown that, considering only pre-call features (R6), we are able to detect almost half of the fraud reliably (R5).

Isolation Forest do not limit itself to the analysis of features before call establishment, but can also be used for post-mortem analysis on CDR records. In Experiment 1 we showed that, for post-mortem detection, Isolation Forests shows promising results. However, more future work is required to explore the boundaries of the technique. The computation efficiency of the algorithm enables fast retraining of the model to avoid phenomena such as concept drift [START_REF] Widmer | Learning in the presence of concept drift and hidden contexts[END_REF].

Although Isolation Forests have shown promising results in this application, the approach has some limitations. First, the construction of an isolation forest is done by randomly choosing features and splitting on values. Depending on the domain, however, certain splits might be more favorable than others. Currently, Isolation Forests do not allow enforcing certain splits or influencing the order in which features should be considered. One workaround would be to run the algorithm separately for every data subset of interest (e.g., per user); however, this no longer enables the discovery of any overlapping patterns between the subsets. Another workaround to enforce a data split would be to convert such split into a set of features (like it was done for feature OfficeHours in Experiment 4). This solves the problem only partially, since it is not possible to enforce the split at a certain location in the tree. Another limitation of Isolation Forests is that, like other unsupervised learning methods, this technique does not use labeled data in the training phase; therefore, it cannot leverage this information to learn how to distinguish between normal and fraudulent traffic.

Conclusions and Future Work

In this paper, we explored the boundaries of Isolation Forest for online detection of IRSF. We have performed a number of experiments to assess the effectiveness of the approach for both offline and online analysis of VoIP traffic and validated results against an existing industrial fraud detector. The results in Figure 10 show that Isolation Forest can identify up to 45% of IRSF traffic before the calls have been established. Since the proposed approach makes no underlying assumptions on the data to analyze, it is general and flexible enough to be applied in other domains such as uselogin analysis or clickstream analytics.

An interesting direction for future work is the study of other derived features, for example based on the countries involved. We also plan to perform a performance analysis of Isolation Forests on a larger-scale, e.g., with dozens of calls per second.

Fig. 1 .

 1 Fig.1. At the start of a call the isolation forest is trained and anomaly scores are computed on the data points. At the end of a call, the post-mortem anti-fraud detector evaluates the call after which the labels of the isolation forest are compared to the existing solution.

Fig. 2 .

 2 Fig. 2.Example model where seven data points are clustered according to two randomly chosen data splits. Regular data points are assumed to have overlap in values and are therefore harder to separate from one another compared to anomalous data points. Isolation forest uses the path length from the root to determine data points' regularity. Anomalous data points have their own leaf node.

Fig. 4 .

 4 Fig. 4. Distribution of calls over the time of the day in our dataset.

Fig. 5 .

 5 Fig.5. Distribution of calls over countries (expressed using ISO 3166-1 country codes) in our dataset.

Fig. 6 .

 6 Fig. 6. Comparison of ROC curves for fraud detection using pre-call and post-call features. The dashed line represents the ROC curve for a random classifier.

Fig. 7 .

 7 Fig. 7. Comparison of ROC curves for fraud detection using pre-call features at the variations of max_depth (top) and sample_size (bottom).

Fig. 8 .

 8 Fig. 8.Comparison of ROC curves for fraud detection using different feature sets. Specifically, the curve obtained using all pre-call features is compared to the curves obtained when either disposition, time or user_id are removed.

Fig. 9 .

 9 Fig. 9.Comparison of ROC curves for fraud detection using different discretization of feature time. Specifically, the curve obtained using all pre-call features is compared to the curves obtained when additionally to the pre-call features, features OfficeHours, HourOfDay or FocusHourOfDay are included.

  Fig.10. Summary of the results. Specifically, the ROC curve for fraud detection using all pre-call features is compared to the ROC curve obtained using the feature FocusHourOfDay.

Table 1 .

 1 Comparison of fraud detection techniques with respect to the requirements for online IRSF detection, as formulated in Section 2. In the table, means "support", "partially support", "no support".

	Requirements	R1 R2 R3 R4 R5 R6

Table 2 .

 2 Summary of parameter settings used for the experiments described in Section 5.

	Parameter	Experiment 1	Experiment 2	Experiment 3	Experiment 4
	nrTrees	100	100	100	100
	sample_size 256	64, 128, 256, 512 256	256
	max_depth	[6, 15] ∈ N	[6, 15] ∈ N	[6, 15] ∈ N	[6, 15] ∈ N
	threshold	[5, 15] ∈ R	[5, 15] ∈ R	[5, 15] ∈ R	[5, 15] ∈ R
	feature_set pre-call, post-call pre-call	subsets of pre-call pre-call + derived features