E Baccelli

G Bartl

A Danilkina

V Ebner

F Gendry

C Guettier

O Hahm

U Kriegel

G Hege

M Palkow

H Petersen

T C Schmidt

A Voisard

M Wählisch

H Ziegler

Area & Perimeter Surveillance in SAFEST using Sensors and the Internet of Things

SAFEST is a project aiming to provide a comprehensive solution to ensure the safety and security of the general public and critical infrastructures. The approach of the project is to design a lightweight, distributed system using heterogeneous, networked sensors, able to aggregate the input of a wide variety of signals (e.g. camera, PIR, radar, magnetic, seismic, acoustic). The project aims for a proof-of-concept demonstration focusing on a concrete scenario: crowd monitoring, area and perimeter surveillance in an airport, realized with a prototype of the system, which must be deployable and foldable overnight, and leverage autoconfiguration based on wireless communications and Internet of Things. This paper reviews the progress towards reaching this goal, which is planned for 2015.

Introduction

This paper describes the current status of SAFEST [START_REF]The SAFEST project: Social-Area Framework for Early Security Triggers at Airports[END_REF], a project aiming at providing a comprehensive solution to ensure the safety and security of the general public and critical infrastructures. Specifically, SAFEST addresses the problems of crowd analysis and monitoring, as well as intrusion detection, using heterogeneous, networked sensors. As various distributed embedded systems have emerged recently (e.g., in home automation, building automation, healthcare automation, and intelligent transport systems) power-line communications and spontaneous wireless networks are indeed expected to connect heterogeneous devices. These include sensors, home appliances, handhelds, and vehicles, giving birth to the Internet of Things (IoT) [START_REF] Sundmaeker | Vision and challenges for realising the Internet of Things[END_REF], a concept heavily leveraged in SAFEST.

The approach taken in the SAFEST project is interdisciplinary in the sense that it complements a technical part (i.e., designing a distributed system for sensing and alerting) with a part focusing on social science aspects (i.e., analyzing the public's awareness and acceptance of such sensing and alerting systems). Scientific and technical challenges in this endeavor are thus very diverse. The key challenges that were identified concern the following fields: sensor hardware design, sensor software platform and techniques for local even detection (such as video processing), communication protocols and knowledge fusion techniques for complex distributed event detection, and methods to evaluate and analyze social acceptance. Figure 1 depicts the overall architecture of SAFEST.

In order to evaluate SAFEST's project realizations, we tightly involve potential end-users of such a sensing and alerting system, over the complete duration of the project. To that end, the consortium includes FBB, which manages Berlin airports. FBB provided continuous feedback and input on the system's requirement analysis and validation, and provides applicable venues to conduct interviews for the socio-cultural analysis. Applicability of the designed system will be further verified by the means of a demonstrator deployed at Berlin's international airport. This setting is particularly adequate since airports present a very challenging and diverse use case with the highest security requirements [START_REF] Wählisch | Technische Dimensionen der Flughafensicherheit[END_REF]: operational challenges include the protection of passengers, staff, and critical infrastructure from serious risks such as mass panic, criminal or terrorist activities in a busy, crowded environment. It is furthermore well conceivable that the SAFEST approach will be adaptable to other types of public infrastructures, such as railway stations or stadiums.

The remainder of this paper reviews the current progress of SAFEST in the key domains of sensor hardware design (Section 2), embedded software platform design (Section 3), video processing techniques (Section 4), distributed cooperation platform design (Section 5), knowledge fusion techniques (Section 6), social science studies (Section 7), and simulation and demonstrations (Section 8). Finally, Section 9 presents the next steps of the project.

Sensor Node Hardware Design

Design requirements of hardware systems must meet ad hoc deployments of security and crowd monitoring perimeters. Communications can be wireless and must support various types of traffics (unicast, multicast, broadcast, image, video and other perception data). Processing must support detection, as well as knowledge fusion and maintenance algorithms. The hardware design must trade off performance with low power consumptions goals. Its multi-core processing must dynamically adjust to application needs, while supporting distributed processing.

The hardware specifically designed for SAFEST includes (i) an uncooled infrared camera and (ii) a processing node called Smartnode.

The infrared (IR) camera architecture follows a compact design and has been optimized to tackle a large variety of public areas like airports or commercial centers with many windows. These buildings are subject to a large range of light and temperature conditions. The camera must indeed support large fields of view; but the IR detection is also very sensitive to the scene, the object to visualize and the environment in terms of light, contrast and temperature. Therefore, dedicated (remote) calibration tools and procedures are required for efficient operation in various deployment environments. The design of electronic boards and video processor must also meet both low power consumption goals and low response times. The IR camera developed and used in SAFEST satisfies these constraints, and integrates a data control layer for video broadcast / multicast in surveillance applications.

The Smartnode architecture must constantly cope with both (i) application-driven distributed processing involved in target detection and tracking, data/knowledge fusion and (ii) sporadic or periodic system-level tasks for ad hoc routing and transport layers, synchronization of processes, shared data updates, monitoring of remote sensors etc. We thus proposed a hybrid processing architecture where a low power consumption processor achieves continuous minimal activity to maintain system synchronization and consistency network wise, while a more powerful processor, dedicated to numerical processing, is woken-up on demand to satisfy system-wide distributed applications. Alternatively, load balancing can also be achieved over the processors if needed.

A prototype of this hybrid architecture designed for the Smartnode was built and is used in SAFEST. This prototype leverages state-of-the-art hardware:

• A Dual-core Cortex-A9 800MHz Nvidia processor to perform sporadic/periodic system-level tasks. • A 1.6GHz Atom processor to support heavier processing loads.

To ease sensor control and network connectivity, the prototype features a variety of I/O ports types: USB 2.0 (Type A), UART TTL, multi-standard serial port RS422/RS485 half and full-duplex, Ethernet 10/100 ports (RJ45) and PAL video input (BNC).

Embedded Software Platform

The SAFEST project couples two categories of systems: (i) a visual and audio surveillance system that monitors large crowds in order to provide guidance in case of unexpected events that trigger mass panic, and (ii) a perimeter protection system that uses distributed event detection algorithms to detect unauthorized intrusions.

For the first task, rather powerful hardware is required: The system must be capable of audio-video processing and the amount of data that has to be transferred can be substantial. For the second task, on the other hand, hardware requirements are quite different: light-weight nodes should be scattered over a large area, in which wired power supply may not be feasible and the amount of data that has to be transferred is much less substantial. In order to fit these diverging requirements, very heterogeneous hardware platforms are used. Our analysis [START_REF] Baccelli | RIOT OS: Towards an OS for the Internet of Things[END_REF] concluded that neither conventional operating systems, nor existing compact operating systems (e.g., designed for sensor networks) could meet the diversity of needs in terms of heterogeneous hardware and software integration for SAFEST in particular, and for the Internet of Things in general. We have thus designed an novel IoT middleware: RIOT [START_REF]RIOT OS -An operating system for the Internet of Things[END_REF], applicable both in the context of SAFEST and in a wide range of Internet of Things scenarios, which aims at networking together heterogeneous hardware, from nodes based on low-power 16-bit microcontrollers, to nodes powered by new generations of energy-efficient 32-bit processors.

RIOT provides a uniform programming interface across this wide range of devices, allowing multi-threading with standard API with very small memory footprint, starting from 1,5kB RAM and 5kB ROM (without network stack). By design it provides energy-efficiency, reliability, and real-time capabilities, based on a modular, microkernel architecture.

On the high end in terms of hardware CPU/memory capacities, RIOT compares mainly with Linux and FreeRTOS [START_REF]The FreeRTOS Operating System[END_REF]. Compared to Linux, RIOT can scale down to orders of magnitude less memory requirements and offers native real-time capabilities as well as built-in energy efficiency. Compared to FreeRTOS, RIOT offers built-in energy efficiency and a full-featured OS including cutting-edge, free, open-source interoperable network stacks (6LoWPAN, IPv6, and CCN stacks), instead of just a kernel.

On the low end in terms of hardware CPU/memory capacities, RIOT compares mainly with Contiki [START_REF] Dunkels | Contiki -a lightweight and flexible operating system for tiny networked sensors[END_REF] and TinyOS [START_REF]The TinyOS Operating System[END_REF]. Compared to these operating systems, RIOT offers real-time capabilities and multi-threading. RIOT also offers standard POSIX APIs and the ability to code in standard programming languages (i.e., C and C++) using standard debugging tools, thus drastically reduces the learning curve of developers and the software development lifecycle process.

Video Processing

SAFEST aims at the protection of people, the goal being to detect unexpected and unusual situations. For monitoring crowds in the context of SAFEST we use infrared cameras (QVGA, λ=8-12µm).

A grid-based approach allows us to model high-level data; the alerting system defines and applies rules on density information. Therefore we divide each scene into nonoverlapping regions of the same size and approximate the density by number of persons in each cell.

The raw video data is processed directly on camera nodes in order to reduce the amount of data, which needs to be sent to a central complex event-processing unit. We do not store or transmit any original video frames -the camera node preprocesses the images to anonymized time-stamped objects.

The architecture of the system is designed as follows: first the camera node processes raw video data and produces a stream of objects, which contain the size and the position of detected persons. This information is transformed by a gateway node into a grid-based density map, which is sent to the central event processing unit producing alarms in case of detecting critical density and other predefined dangerous situations. The gateway node does not only aggregate data flows, but also merges information from different nodes and sends information to a knowledge fusion component for further processing. Thus, we call the gateway a merging component. Crowd detection and counting is a challenging task. Since our approach should be privacy-friendly, we do not focus on systems using analysis of special parts of bodies such as eyes, faces, or silhouettes. We also do not focus on texture analysis such as [START_REF] Marana | Automatic estimation of crowd density using texture[END_REF] due to their inability to detect single persons but only roughly estimating number of people or [START_REF] Wu | Crowd density estimation using texture analysis and learning[END_REF] since they are using high-weighted learning algorithms on texture and are not sensor network and realtime capable. Likewise we do not focus on systems using motion analysis [START_REF] Rabaud | Counting Crowded Moving Objects[END_REF]. We apply the idea of a multistage approach leading to object-level analysis.

Since the camera is placed on the ceiling and looks vertically down to the scene, people appear as moving "blobs" in the scene. Therefore the main aim of the crowd analysis is to first detect and then count individual persons. The foreground objects -people -need to be extracted from the background. To achieve this, we apply a mixture of Gauss functions for statistical background modeling [START_REF] Zivkovic | Improved adaptive Gaussian mixture model for background subtraction[END_REF]. Each pixel's intensity is evaluated frame-wise to parameterize several Gauss functions. Adaptive Gauss functions are able to describe different surfaces, lighting conditions, and their changes and as a weighted mixture model the background.

Having a background model we compare it to the image of the current scene. The resulting differences are the foreground pixels, which are then merged into connected regions using connected component analysis [START_REF] Dillencourt | A general approach to connected-component labeling for arbitrary image representations[END_REF]. Connected regions represent parts of detected people such as legs or arms. These are then clustered in order to find the number of individuals in the scene -each cluster represents one or multiple persons. Our clustering technique is based on the principles of density-based reachability [START_REF] Kriegel | Density-based Clustering[END_REF], which also allows to find non-linearly separable clusters.

While moving, people can come very close to each other or even enter the scene holding hands. In this case, the described approach will find connected regions, which actually represent several people. In such cases, we apply different heuristics to split too large or wrongly merged clusters.

Looking down to people, they can be approximated by circles. Therefore we compute the position and the radius of the approximating circle for each cluster. The result of the video analysis is a list of detected people, their positions and radii. We smooth the data by averaging these parameters over 25 frames per second. This list is then sent by each camera to the merging component. Our approach provides suitable data for further analysis of crowd information, its shape, density, position and behavior without sending whole images throughout the network. Another advantage is that it is impossible to identify or track individual people, reducing privacy concerns.

In the merging component, we compute the crowd density for each grid separately. The discretized density is represented by the number of people per grid cell (see Figure 4). Some people may belong to several grid cells, in case they are positioned on the lines separating the cells. We assign them to the overlapped cells proportional to the area occupied in each cell. We developed geometrical solutions for each of the 16 cases how the grid can divide a person into areas. Furthermore, the merging component has to correct geometrical distortion of the grid at the image edges. It is planned to eventually use more than a single camera to monitor wider areas. Therefore, the merging component computes a single grid-based density map from multiple lists from different cameras. The output of the merging component is one density map per second which is sent to the event processing component described in Section 6 and to the data-mining component for further offline data analysis.

We evaluated our approach using sequences recorded during testing of a live demo [START_REF]The SAFEST project mid-term demonstration[END_REF], which we annotated manually. The results are satisfactory -with a maximum of 11 people in the frame, the number of people counted per frame deviated from the expected value on average by 0.839 persons. The variance of the error equals 0.567. The actual number of people in the scene was determined by manual counting for 320 frames chosen at constant intervals from recorded test sequences. We expect decreasing results for counting by evaluating our approach with default parameters and random sequences. Our current evaluation shows that the algorithm we developed for people counting is able to deal with video scenes showing single people, sparse groups of people, and dense crowds (assuming that the background is still present in the scene, i.e. people are not too densely distributed, and don't form a non-separable "blob" covering the whole image).

Distributed Cooperation Platform

Communication in the IoT is required to enfold spontaneously and organize autonomously without infrastructure provisioning. To meet this objective in lowpower and lossy networks (LLN), the standard routing protocol RPL [START_REF] Winter | RPL: IPv6 Routing Protocol for Low-Power and Lossy Networks[END_REF] has been designed and implemented on the RIOT operating system platform [START_REF] Baccelli | RIOT OS: Towards an OS for the Internet of Things[END_REF] of SAFEST.

We have furthermore designed and standardized an extension to the RPL protocol, P2P-RPL [START_REF] Goyal | Reactive Discovery of Point-to-Point Routes in Low Power and Lossy Networks[END_REF] [START_REF] Goyal | A Mechanism to Measure the Quality of a Point-to-point Route in a Low Power and Lossy Network[END_REF], which improves the performance of RPL in scenarios where application data traffic between arbitrary nodes in the network would benefit from not flowing systematically through a central node (the root of the RPL tree). For this purpose, based on a reactive approach, P2P-RPL enables on-demand establishment of shorter sensor-to-sensor paths that do not necessarily follow the basic RPL tree structure. This type of traffic is necessary in SAFEST to enable nodes to interact and take decisions locally without central control.

Enabling Spontaneous, Secure Communication in the IoT

RPL in its current state of standardization is however vulnerable to a variety of severe attacks that build on topological infringements. To cure the deficits, we designed and evaluated TRAIL (Trust Anchor Interconnection Loop) [START_REF] Perrey | Topology Authentication in RPL[END_REF], which can discover and isolate bogus nodes while they attack the RPL routing hierarchy. TRAIL is derived from first hand principles and shall resolve the issues of topological infringements. Using proper reachability tests, TRAIL reliably identifies any topological attacker without strong cryptographic efforts in a scalable manner. It has been implemented and made openly available on the RIOT platform.

Programming the IoT

In the IoT, a large number of constraint devices typically cooperate to perform common tasks. This dedication requires a new approach to (highly) distributed programming, which complies to the following environment-specific requirements:

1. High scalability up to thousands of nodes 2. Loose coupling of components to allow for a high degree of independent tasks 3. Enhanced fault tolerance to keep the overall system working even at repeated node failures 4. Low overheads compliant to the constraint environment 5. Native implementations to avoid additional software overheads The SAFEST consortium has followed the concept of Actor programming as a lightweight approach to highly scalable distributed programming. The actor model is a formalism describing concurrent entities -"actors" -that communicate by asynchronous message passing. An actor can send messages to addresses of other actors and can create new actors. Actors do not share state and are executed concurrently. Because Actors are self-contained and do not rely on shared resources, race conditions are avoided by design. The message passing communication style also allows network transparency and thus applies to both concurrency, if actors run on the same host on different processors, and distribution, if actors run on different nodes connected via the network.

We have contributed libcppa [START_REF] Charousset | Native Actors --A Scalable Software Platform for Distributed Heterogeneous Environments[END_REF], a native actor library written in C++ that adapts the original model to heterogeneous and constraint environments. libcppa enables lightweight distributed programming on embedded devices without introducing interdependencies and faulty conditions in distribution. This new distributed programming environment is extremely light-weight and demonstrates its high scalability in thorough evaluations.

Knowledge Fusion Techniques

A knowledge fusion subsystem receives a continuous stream of information from attached sensors, augments and correlates them to extract higher level information, which is beyond the scope of a single sensor. The basis of the knowledge processing subsystem in SAFEST is a distributable implicit middleware for knowledge processing components (KPC), developed at Fraunhofer FOKUS [START_REF] Kriegel | Practical Aspects of the Use of a Knowledge Fusion Toolkit in Safety Applications[END_REF]. A KPC is a worker component in an eventdriven architecture [START_REF] Everding | An Event Driven Architecture for Decision Support[END_REF]; it is activated on arrival of new events on a given event input stream and it publishes its processing results on an event output stream. In the context of SAFEST, two types of KPC are used: (i) KPCs programmed according to the KPC component model to preprocess and filter the incoming event stream, and (ii) rule-based KPCs that encapsulate the industry-standard ofthe-shelf open source rule-based software Drools Fusion [START_REF]Drools Fusion[END_REF] to be able to easily adapt fusion logic to new and changing situations. One or more KPC controllers (which can be distributed over different execution environments) load their KPCs, provide a runtime environment for them, and connect them to dedicated event input and output channels. As a communication platform, Redis [START_REF] Carlson | Redis in Action[END_REF] was chosen. Redis is a distributed and scalable open source key-value store with publish/subscribe features, for which a client implementation exist for almost all mainstream programming languages. This makes it an ideal platform for asynchronous inter-platform communication in SAFEST.

In the context of SAFEST, the knowledge fusion subsystem is used to detect unusual and critical crowd behavior. It receives a stream of density maps from the video processing subsystem and issues alerts when a critical situation is detected according to one of the two implemented mechanisms. The first mechanism is based on static patterns, allowing the definition of observation regions around critical areas, such as exits, security areas, or evacuation tunnels. Hereby an average person density value will be calculated and observed via a rule engine, which initiates actions (e.g., alert notifications) on fulfillment of certain conditions, such as exceeding density thresholds. This is particularly useful for identification of bottleneck characteristics based on density/velocity ratios. Furthermore, the uncomplicated setup is quite advantageous.

The second mechanism is built on dynamic detection. Opposed to the static patterns, it can be applied to the full camera-observed area and is able to locate and track crowds and to observe size changes. With regards to the implementation on the one hand data clustering algorithms were considered, on the other hand a contour detection mechanism for two-dimensional scalar fields (i.e., marching squares) was investigated. The clustering algorithms included centroid models, distribution models, density models, hierarchical density models and subspace models. The comparison was performed by application to different crowding scenarios and crowd types. The results demonstrate that data clustering techniques have limitations regarding special crowd types, such as low density crowds and non-centric crowds, but also outliers caused difficulties. In contrast, the results of the marching squares algorithm were very satisfying, as even unusual crowd types were analyzed correctly.

As consequence the latter was chosen and integrated into the system. The algorithm was adapted in order to support the specification of density ranges, but also to support recursive analysis, which is required for crowd decomposition and inner structure analysis.

Social Science Studies

The goal of the social study is to find out whether surveillance techniques used for crowd monitoring affect privacy issues and thus influence the users' acceptance of security measures at airports [START_REF] Bartl | Soziale Dimensionen der Flughafensicherheit[END_REF]. Besides, the study aims at evaluating how privacy relates to other factors of acceptance such as transparency, health, time and effort, or discrimination.

On the one hand, the process of technical development was reflected by making use of expert interviews with security experts from BER airport. On the other hand, an interview study with airport passengers consisting of a qualitative and a quantitative part aims at identifying relevant dimensions of acceptance of security measures at airports. Thus, the technical development process can be oriented on the passengers' needs and preferences what is seen as a precondition for an efficient use of the SAFEST technology.

The interview guideline of the expert interviews focused on two core topics: technical attributes (e.g., technical edge conditions, and technical product attributes) and social aspects (e.g., social impacts of video surveillance and security culture at the airport). Concerning the technical requirements attributes such as compatibility, adaptability, usability, reduction of complexity, or the exchangeability of technical components were brought up by the experts. Besides, spatial constraints and access to electricity resulting from the architecture of the airport were seen as important aspects. The changing of organizational structures through the implementation of new technical systems was another interesting point that revealed the inseparability of technical and social issues in the context of security measures at the airport.

The first part of the study of acceptance with airport passengers has also been completed. 18 problem-centered interviews could be conducted in September 2013 at the Berlin Airport of Schönefeld [START_REF] Witzel | The problem-centred interview[END_REF]. These interviews were analyzed by making use of the method of qualitative content analysis [START_REF] Mayring | Qualitative Inhaltsanalyse: Grundlagen und Techniken[END_REF]. The results show that acceptance is a multi-factorial construct that varies significantly on the individual and socio-cultural level. This finding will be explored in greater detail in the quantitative survey with flight passengers. Additionally, a preliminary model of acceptance was derived from the interview material that structures various factors of acceptance on different levels like types of acceptance, dimensions of acceptance, subjective perception, personal experience, values and security culture [START_REF] Bartl | Die Bevölkerung als Adressat der Sicherheitsforschung[END_REF] (see Figure 5).

Simulations and Demonstrations

One of the main tasks of a SAFEST infrastructure is to detect unexpected situations. Therefore, a dedicated simulation environment has been developed, to be able to simulate various scenarios. The simulation output provides series of density maps that can be used to describe the expected course of events, e.g., in case of an incoming flight, an incoming train or an emergency evacuation situation. The results can be used to (1) specify the relevant event patterns and to parameterize the event detection rules, and (2) to test and validate alert conditions. The scenarios can be parameterized by, e.g., the modal split of inbound and outbound passengers (coming or going by car, taxi, bus, train, and so on), by gender aspects, or by the ratio of Schengen to Non-Schengen visitors. The solution is based on an industry-strength micro-simulation for pedestrian streams in 3D space (PTV Viswalk [PTV]) that has been customized by SAFEST specific output modules. Scenarios have been tested using different types of pedestrians featuring different parameters like size and behavior representing the spectrum of airport visitors (business travelers, tourists, or couples and roll chair users). The result is a generic process model combined with best practice approaches describing how to develop an evacuation monitoring solution for an airport based on a SAFEST infrastructure. The process model has been validated using the BER airport as an example.

We have furthermore recently demonstrated an early version of the full system during a live demo at FU Berlin [START_REF]The SAFEST project mid-term demonstration[END_REF], which included a demonstration of (i) the distributed spontaneous wireless network of nodes, using our proposed improvements of RPL, (ii) distributed, simple event detection for perimeter surveillance, (iii) distributed, simple event detection for crowd monitoring using the prototype IR camera and the developed video analysis tools, and (iv) a prototype of the knowledge fusion module and an event visualization tool usable on desktops and/or handheld devices.

Next Steps

The next steps in the video processing component will be to optimize the algorithms further to be able to better handle dense crowds. Especially when crowds form very densely packed areas counting of individual people becomes a very challenging task. Therefore detailed sensor data from the camera, such as the applied gain and the observed temperature range in the scene, will be fed into the algorithm in real time. Also, the aggregation and correlation of data from multiple cameras still needs to be tested. Moreover, we are aiming at integrating "human sensors" or "validators" into the knowledge fusion system, by providing airport personnel with mobile devices and easyto-use interfaces.

Further hardware developments are currently in the works such as a rugged version and a low-cost version of the IR camera, as well as an improved prototype of the Smartnode based on an ARM Cortex-A15.

We also plan to conduct a comparative analysis and experiments of different IoT network stack elements, using RIOT as common software platform and cyber-physical systems testbeds (such as Senslab [START_REF] Senslab | Very Large Scale Open Wireless Sensor Network Tesbed[END_REF], DES, FIT [START_REF]FIT: The Future Internet (of Things) Platform[END_REF]). These large-scale testbeds (hundreds of nodes each) will help validate and refine our approach towards tailoring an appropriate network stack for SAFEST. More field tests and fuller software/hardware integration will have to take place in order to prepare for the final demonstration planned in 2015.

In the mean time, we plan to continue our in-depth social study. The results from the problem-centered interviews with flight passengers were used for the construction of the standardized questionnaire. This questionnaire contains items that relate to subjective perception (e.g., cameras and space), acceptance (i.e., privacy in relation to other dimensions of acceptance), security culture at the airport, flight habits (e.g., profession vs. holiday, flights per year), and personal data (i.e., age, gender, education, and occupational field). The survey is planned for early 2014.

Figure 1 :Figure 2 : 3 :

 123 Figure 1: Overall Architecture of SAFEST

Figure 4 :

 4 Figure 4: Processing of IR-Video (a), derived people coordinates and radii (b), derived density (c)

Figure 5 :

 5 Figure 5: Preliminary model of acceptance