Jan Dvorak
email: jan.dvorak@uhk.cz

Ondrej Berger
email: ondrej.berger@uhk.cz

Ondrej Krejcar
email: ondrej.krejcar@uhk.cz

Universal Central Control of Home Appliances as an Expanding Element of the Smart Home Concepts

Keywords: Smart Home, Infrared, Remote Control, Automation, Mobile, Multimedia

With the development of the electronic appliances, there is an increase in the usage of the so called Smart Homes concept. Nowadays, many standards and technologies exist which enable the control of the related home appliances. However, the common devices that support only the handeling via a controller on the basis of infrared transfer are not possible to be operated by these systems. In practical environment, the suggested solution is able to replace the function of such controllers and enable the user to operate the given devices using a website application that is available for example through a smartphone. The part of the solution is the interface for the connection of the application to current Smart Home systems and an intelligent recognition and an automatisation of user's actions with controllers. The result of the implementation is a functional prototype.

Introduction

The recent fast development in the field of electronic appliances also accompanies the development of the systems which contain elements of so called Smart Homes [START_REF] Ricquebourg | The smart home concept: our immediate future[END_REF][START_REF] Vanus | The proposal model of energy savings of lighting systems in the smart home care[END_REF][START_REF] Vanus | Design of a smart building control with view to the senior citizens' needs[END_REF][START_REF] Behan | Modern Smart Device-Based Concept of Sensoric Networks[END_REF][START_REF] Krejcar | Use of Mobile Phone as Intelligent Sensor for Sound Input Analysis and Sleep State Detection[END_REF][START_REF] Krejcar | Smart Intelligent Control of Current Source for High Power LED Diodes[END_REF], the methods for the cooperation with electronic devices enabling the basic functions of a house. Originally, Smart House technologies were developed for the control of energetically advanced systems, such as heating or lighting. However, the current level of electronic equipment enables the connection of almost any components into the Smart Home system. Moreover, this is not only bound to switching on and off devices, it is also possible to create actions which would simplify the inhabitation of such environment and save time on the basis of pre-defined or intelligent algorithms. This is due to the instruments for monitoring of the presence and user's activity [START_REF] Naghiyev | Three unobtrusive domestic occupancy measurement technologies under qualitative review[END_REF]. [START_REF] Haines | Probing user values in the home environment within a technology driven Smart Home project[END_REF] In relation to the intelligent buildings, recently the controlling and monitoring of their energy usage became a popular topic. The reason for the interest is for example the fact of up to 40% of spending on energy is in commercial buildings which can be reduced to 70% [START_REF] Kim | Implementation of a cost-effective home lighting control system on embedded Linux with OpenWrt[END_REF] using automated systems for controlling of lighting. An example of an active operation in this field is the research of wireless control of lighting [START_REF]The value of wireless lighting control.www.daintree[END_REF], technologies of smart transfer networks (Smart Grids [START_REF] Melike | Wireless sensor networks for cost-efficient residential energy management in the smart grid. Smart Grid[END_REF]) or general usage of Smart Homes [START_REF] Dhiren | Energy conser-vation in a smart home[END_REF].

Current systems for intelligent homes usually contain a central controlling unit to which using star or bus line topology there are attached end nodessensors or action members. In relation to the supplier of the given solution, these nodes must support a specific standard of communication. The common representatives of such standards are for example C-Bus which communicated using the Ether network, international standard KNX [START_REF][END_REF] or a wireless system ZigBee [START_REF]ZigBee Specifications[END_REF].

The purpose of the project described in this paper was to suggest and implement a hardware and a software which would contribute to the centralised solution of the home appliances that are operated using the infrared controllers and do not support any of the standards for central controlling. The solution contains a website application which introduces a Human-Machine interface (HMI) and provides API for integration of current Smart Home systems. Moreover, the system continuously monitors the usage of current controllers and on the basis of the user's activities suggests for example the unification of commonly consequential steps into mass actions (macros) or automatic implementation of regular actions. The result of this function should be the time saving and simplifying of everyday activities related to home appliances, which according to [START_REF] Haines | Probing user values in the home environment within a technology driven Smart Home project[END_REF] corresponds to basic requirements of Smart Home technology users.

Problem Definition

The main reason for this project was the absence of the connection of most common multimedia appliances with current Smart Home solution. Some models support the standard HDMI CEC [20] that enables the operating of interconnected appliances using the HDMI cable. However, for the usage of such standard it would be necessary to connect to each device an individual module for its maintenance. This is technically, but also financially a very demanding solution. On the other hand, the traditional infrared remote controls are used by most multimedia home appliances. Moreover, the hardware for the transmitting and receiving is favourably priced. For example, the article [START_REF] Han | More efficient home energy management system based on ZigBee communication and infrared remote controls[END_REF] is concerned with the processing of the signals from IR controllers. This article suggests the system of automatic switch off of unused electric plugs and their repeated awakening using the IR signals.

Currently, there are a few products on the market which enable the functionality of the central controlling of the appliances using the IR signals. The product line Harmony from Logitech contains a wide spectrum of universal remote controls, from which Ultimate Hub, Smart control and Ultimate models have the Hub unit that provides similar functions as the suggested solution. For this solution, the company also offers applications for iOS and Android which are very well built and intuitive. Unfortunately, the Harmony product line is very closed and does not allow the connection with other systems. The prices range from 100 to 300 USD, depending on the specific model.

Moreover, the Open Source Universal Remote application is also worth mentioning when considering the open-source solutions. This application can be installed on any PC which contains IR controller and enables the transmitting of commands to appliances from any web browser in the network, for example even a smartphone. The disadvantage of this application is that it does not offer the possibility to record signals of the current controller and consequently emulate them.

None of these above mentioned solutions has the ability to monitor the user's activities with the current controller and also does not suggest the optimisation of regular activities. The first two are introduced in the in the Fig 1.

New Solution

This chapter introduces the suggested solution for controlling of end devices. The schema (fig 2) shows the architecture of the whole system. It is important to notice the 3 basic communication directions:

• Through the web application, user can operate the end devices using the IR transmitter in the central point of the system.

• Central point captures commands which are sent from the original controller into the end devices.

• On the basis of the manual command, it is possible to duplicate the current controller and use it from the web application.

The requirements for the central point included the ability to receive and transmit IR signals which are compatible with remote controls and communicate with other devices using the HTTP protocol and LAN home network. Most of the remote controls use the IR transfer with coding on the 38 or 36kHZ frequency. Manchester, Pulse distance or Pulse length coding is used for the modulation. In order to decode these signals, it is possible to use a completed three-pin receiver that contains a demodulator and control circuit due to which the generated digital signal is corresponding to the transmitted value. In comparison to the simple IR phototransistor, it is not necessary to manage the demodulation. The inner schema of the used receiver is shown on the Fig. 3.

Fig. 3 The inner connection of the IR receiver TSoP 31238 [START_REF] Machacek | Advanced system for consumption meters with recognition of video camera signal[END_REF].. The suggested transmitting circuit contains two IR diodes and a switching transistor (in order to not overwork the data pin by too much current). Two of the used diodes with the combination of wide-angle transmitting should provide the coverage of all appliances in an average room.

The next task was to ensure the recognition of correct coding of digital signals which come from the receiving module, the subsequent transfer to HEX representation of the pushed button and the possibility to code the value back and send it through the IR transmitter. The LIRC library was used to uncover the coding. It is composed of numerous basic sub-programmes:

• Irrecordan interactive console application which enables to "record" the current remote controlling; the final product is a configuration file liircd.conf which contains for each operation a description of used coding signal and mapping of individual HEX codes of symbols into their aliases • Liircdsystem demon which acts on each defined end device with connected IR transmitter and using the socket enables other applications to send the previously recorded IR commands

• Irsendassociating console application which resends the entered saved command into the liircd socket • Irwsimple utility which reads the lircd aliases of the received codes from the socket that is created by the demon and sends them to the standard output Before the actual implementation, the structure proposal the user's interface of the application was conducted. The frequency of the expected usage of individual tasks which should be implemented was used for the distribution of the most important elements to the best reachable places. The most significant service of the application is the simulation of the push of the buttons for recorded controllers. Therefore, the accomplishing of the mass actions was placed at the original page.

The special page for recording of the controller functions on the basis of interactive navigator with simple commands. It can obtain text data from the users, such as the names of the recorded buttons, show the progress guide or the countdown of the remaining time. Physically, it is an interface which in real time [START_REF] Machacek | Advanced system for consumption meters with recognition of video camera signal[END_REF][START_REF] Benikovsky | Proposal of User Adaptive Modular Localization System for Ubiquitous Positioning 4th Asian Conference on Intelligent Information and Database Systems[END_REF][START_REF] Cerny | Wireless Body Sensor Network in Health Maintenance Systems[END_REF][START_REF] Penhaker | Sensor Network for Measurement and Analysis on Medical Devices Quality Control[END_REF], using the WebSockets communication technology [START_REF] Krawiec | Web System for Electrophysiological Data Management[END_REF]) conveys the commands of the irrecord application to the user and immediately returns to it the obtained data. The structure of the application and possible paths between individual functions are shown on the Fig. 4. The unique function of the suggested solution is the detection of the common actions. It is an implementation of an algorithm for searching of repeated sequences in the recorded time row. There are many papers which cover this problem, for example [START_REF] Ghassempour | Clustering Multivariate Time Series Using Hidden Markov Models[END_REF][START_REF] Wan | A frequent pattern based framework for event detection in sensor network stream data[END_REF][START_REF] Han | Frequent pattern mining: current status and future directions[END_REF][START_REF] Berndt | Using Dynamic Time Warping to Find Patterns in Time Series[END_REF]. Most of them suggest the usage of algorithms based on the Markov chains or decision trees. However, in regards to their intended purpose for continuous data or even multi-dimensional data, they are too complicated for the needs of this paper.

The commonly used model of decision trees was used during the suggestion of a simpler algorithm which enabled the recognition of samples in recorded binary data that represent the buttons' push of the controllers. The recorded time line is processed by the algorithm for the detection of clusters and then each cluster and its metrics (for example the average delay between individual commands) are recorded in the form of a sequence into the graphic database. This obtains a system of sets of trees from which each represents a sequence of commands with identical first node. Therefore, each recorded sequence is connected to the current tree or there is a new tree created. When repeating the sequence, a counting mechanism is incremented only in the database.

From this continuously maintained database, it is possible to simply obtain the sequences which are most commonly used and may be for example suggested to the user for establishing of macros.

Implementation of the Solution

The compact computer Raspberry Pi [28] which functions on the platform ARM was used for the prototyping of the central point of the system (see Fig. 5). This platform is sufficiently powerful in order to carry out all the tasks and it has better dimensions and battery than common x86 PC platforms. It has the dimensions similar to the ones of a credit card and its price is around 25EUR. Moreover, it contains an integrated processor with 700MHz frequency and a 512MB RAM memory. In comparison with the simpler and cheaper device, such as Arduino, it has a significant advantage due to its support of fully-functioning operating systems on the basis of Linux which enable the usage of many software technologies. For the creation of the main application which contains all the functional logistics and offers the possibility of controlling through HTTP, the Node.js platform was chosen [START_REF] Dahl | Node. js: Evented I/O for V8 JavaScript[END_REF]. It is a server solution which functions with the JavaScript running environment V8 and is used in the Google Chrome browser. Its advantage is the orientation on the assynchronic utilisation of events and high power, even with a weaker hardware.

Fig. 6 The proposal PCB of IR module with separated transmitting part.

The existing library was used for calling LIRC sub-programmes from Node.js on which even the competing solution Open Source Universal Remote is built. Unfortunately, it supports only the sending of the commands. Therefore, it was necessary to add the support for irrecord which enables the initiation of the record of the existing remote control from a smartphone or a PC. This was a significantly difficult task due to the fact that the given sub-programme has an interactive interface from which it is necessary to transfer all events from console through Node.js and into the browser, as well as back.

The whole structure of the application, described in the chapter III, was implemented in a way which enables a fully valuable and comfortable control from a web browser, as well as from a smartphone, independently of the resolution. An example of the application run from a smartphone which was adjusted to the resolution of the display is shown on the Fig. 7. The result of this paper is the functional prototype of the device which ensured a centralised control of home multimedia appliances using the IR signals. In this chapter, the suggested solution is compared with competing products. The following table 1 shows the main advantages of the individual solutions which are comparable with the suggested one. The purpose is not to give a detail description of each product, but only to compare the important characteristics. It is clear from the table that the main advantages of the suggested solution are the possibility of the linkage to other services using the public API and the ability to automatically recognise the most commonly used sequences. The commercial products offer more functions, but they are usually dependent on the hardware of the particular manufacturer.

Furthermore, the user testing on 10 sample subjects in the age 15-55 was run. It contained following tasks:

• Connect devices into the multimedia home system and run it,

• record the current controllers,

• try to use web application in a smartphone instead of controllers,

• let it create and name a mass action which is automatically suggested after a certain time period of usage.

After the evaluation of the results, it was found that users were able to fulfil given tasks with the success rate of 80% which shows a correctly designed users' interface and generally the principle of the solution. The created reminders were used for further improvement of the application.

Conclusions

This paper suggests and open-source solution for centralised control of multimedia appliances which use IR controller. The suggested solution can be used as a supplement to the current Smart Home systems. Using the web application which is accessible from even a smartphone, it is possible to record commands of the current controllers, emulate their function using an individual approach and on the basis of long-term activity of the user, it suggests the unification of frequent sequences of commands into macros. This functionality can save time of the user and increase the comfort of working with multimedia appliances. The functional prototype was compared to the competing solutions and tested with users. One of the next directions of the development could be for example the improvement of the algorithm for detection of common usage patterns, independent of the order of carrying out the particular commands in the case when it does not matter.

Fig. 1

 1 Fig. 1 Commercial device enabling the central control of IR appliances.

Fig. 2

 2 Fig. 2 Devices present in the system for the central control of the appliances.

Fig. 4

 4 Fig. 4 Diagram of the transfers between individual pages of the application.

Fig. 5

 5 Fig. 5 The finished prototype of the central point.

Fig. 7

 7 Fig. 7 Web application is responsive and adjusting its settings according to the type of device used.

TABLE 1 .

 1 COMPARISON OF COMPETING SOLUTIONS[START_REF] Bain | Open Source Universal Remote[END_REF][START_REF]Harmony Remotes[END_REF][START_REF] Elko Ep S.R | Převodník LAN na IR[END_REF]

	Name	Characteristics	Suggested solution
	Logitech	Native application for Android	Instead	of	the	native
	Harmony	and iOS. More expensive	application, it is possible to
		versions also have a hardware	use a web application which
		controller. It does not have API	is optimised for mobile
		and does not learn according to	devices.
		the user's behaviour.
	ELCO	The closed solution with a non-	Open API for the linkage of
	eLAN-IR-3	intuitive application. Does not	services of 3 pages and
		read the user's behaviour.	algorithm for the detection of
			common	sequences	of
			commands.
	Open Source	An intuitive web application	An interactive guide for
	Universal	even for mobile devices. But	recording of the controller.
	Remote	non-intuitive recording of the	Princip	web	application
	(OSUR)	controller using command	originated from OSUR.
		lines.
	Traditional	Large selection of expensive	IR signals	enable the
	Smart Home	specialised appliances which	controlling		of	most
	solutions	are able to be centrally	multimedia appliances with
		controlled, but the inability to	low expenses. For controlling
		control common appliances	of more advanced elements, it
			is necessary to link another
			system.

Acknowledgment This work and the contribution were supported by project "SP/2014 -Smart Solutions for Ubiquitous Computing Environments" Faculty of Informatics and Management, University of Hradec Kralove, Czech Republic.


Nhat -Quan

Huynh Nguyen

Thai Hoang Le
email: lhthai@fit.hcmus.edu.vn

Enhanced LBP-based face recognition system using a heuristic approach for searching weight set

Keywords: Face recognition, Local binary patterns, Heuristic search, FERET, Weight set

Local Binary Patterns is one of the most effective approaches for pattern recognition in general and face recognition in particular. There have been many studies on improving this method such as changing the input values or using another kind of histogram. Although weight set is also an important key leading to the success of this method, it does not seem to get much attention. A majority of LBP-based approaches are still using the weight set of Ahonen et al.'s study, one of the first researches applying LBP to face recognition. In this study, we introduce a powerful algorithm named Heuristic Weight Search, which finds a suitable weight set for not only LBP-based approaches but also other methods using weight set to improve performance. Experiments on the FERET database prove an ability of HWS thanks to their higher accuracy than original methods.

Introduction

Face recognition is a highly active area of researchers thanks to its real-world applications. Up to now, a large number of face recognition methods have been developed. Local Binary Patterns (LBP) is widely used for face recognition based on its potential for speed and accuracy. LBP can be applied for not only face recognition, but also many applications such as: texture classification, facial expression recognition and gender recognition. Since Ahonen et al. [START_REF] Ahonen | Face description with local binary patterns: Application to face recognition[END_REF] introduced LBP as a powerful method for describing faces and reported impressive results in experiments on the FERET database in 2004, many LBP-based descriptors have been proposed. In 2012, a research from Ngoc-Son Vu et al. [START_REF] Vu | Face recognition using the poem descriptor[END_REF] introduced the Patterns of Oriented Edge Magnitudes (POEM) descriptor which uses LBP operator as a main part of this powerful descriptor.

According to Ahonen et al.'s research [START_REF] Ahonen | Face recognition with local binary patterns[END_REF], a weight set is computed through a simple procedure. Although this weight set has been used for many researches to improve performance without any changes, it seems to be too general to fix the characteristic of each data set. That makes the problem how to create an efficient weight set for a respective condition of images such as facial expression or illumination changes.

In this paper, we propose a Heuristic Weight Search (HWS) algorithm to solve the problem of generating weight set based on the properties of each image set. Applying this algorithm on any face recognition system using weight set helps to improve performance at all. A sample system integrating POEM descriptor into HWS algorithm is also introduced.

Our study is organized as follows: in section 2 and 3, LBP and POEM are introduced respectively; section 4 represents our algorithm to generate weight set of image sets; the sample system combining POEM descriptor and HWS algorithm is illustrated in section 5; the next section presents our experiments on FERET database; conclusion and references are in the rest of this paper.

Local Binary Patterns

Being introduced by Ojala et al. [START_REF] Ojala | A comparative study of texture measures with classification based on featured distributions[END_REF] in 1996, LBP operator is a powerful tool for texture description. This operator labels the pixels of an image and thresholds each neighbourhood of 3 × 3 pixels by using the central pixel value. The gray value of each pixel g p in the neighborhood is compared to the gray value g c of the central pixel. If g p is greater than g c , then it is assigned 1, and 0 if not. The LBP label for the central pixel of image is obtained as [START_REF] Ahonen | Face recognition with local binary patterns[END_REF]:

LBP = 7 p=0 S(g p -g c )2 p , where S(x) = 1, x ≥ 0 0, x < 0 (1)
The basic operator was extended to use neighbourhood of different sizes to capture dominant features at different scales. Notation LBP (P, R) denotes a neighborhood of P equally spaced sampling points on a circle of radius of R. For example, the original LBP operator can be denoted LBP (8, 1).

According to [START_REF] Ojala | Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[END_REF], there is a small subset of 2 p patterns, called uniform patterns, accounted for the majority of the texture of images, over 90% of all patterns for LBP (8, 1) and about 70% for LBP (16, 2). These patterns contain at most two bitwise transitions from 0 to 1 or vice versa for a circular binary pattern. For instance, '00000000' (0 transition), '01111100' (2 transitions), and '10001111' (2 transitions) patterns are uniform while '11001110' pattern is not uniform due to its four bit transitions. The uniform patterns can be used to find the pixels that belong some texture primitives such as spot, flat area, edge, and corner. It can be denoted as LBP u2 (P,R) , which was mentioned in [START_REF] Ahonen | Face description with local binary patterns: Application to face recognition[END_REF].

Fig. 3: LBP feature extracting procedure in two case of applying weight set.

In the face recognition procedure of Ahonen et al. [START_REF] Ahonen | Face description with local binary patterns: Application to face recognition[END_REF], a facial image is divided into m small same sized non-overlapping regions called patches from which LBP histograms are extracted and concatenated into a single vector, which enhanced the facial representative ability. A patch histogram is defined as [START_REF] Ahonen | Face recognition with local binary patterns[END_REF]:

H i = x,y I{LBP (x, y) = i}, i = 0, 1, ..., n -1 (2)
in which n is the number of different labels produced by the LBP operator and

I(A) = 1, A is true 0, A is false. (3)
In using classifier like k-Nearest Neighbour to decide label for input image, distance computing formula must be used in order to calculate distance between images which is difference between feature vectors of these images. It was also announced by psychophysics studies that some regions on face contain more useful information than others in the process of identifying a person. That means each region on face has its own contribution which can be represented as a number called weight. Difference between applying specific and default weight set is presented in Fig 3 . In this research, we suggest using weighted Chi square statistics, which has been used to build a dissimilarity measure of face with the weight set W .

χ 2 W (S, M ) = i,j w j (S i,j -m i,j ) 2 S i,j + M i,j (4)
where S and M are two LBP histogram, w j is the weight for jth region.

Patterns of Oriented Edge Magnitudes

In their research, Ngoc-Son Vu et al. [START_REF] Vu | Face recognition using the poem descriptor[END_REF] introduce a LBP-based structure on oriented edge magnitudes called Patterns of Oriented Edge Magnitudes (POEM). This descriptor uses the gradient magnitudes instead of intensity value in the computing using LBP operator. POEM feature is extracted through the process shown at Fig. 5. Firstly, the oriented edge magnitude image (oriented EM I) generated from an original input image is divided into m uni-oriented EM Is uEM I θi through gradient orientations of pixels, over 0 -π (unsigned representation) or 0 -2π (signed representation). Then, each pixel on uEM Is is then replaced by the sum of values of its w × w neighbour area called cell. The resulting images are called accumulated EM Is (AEM Is). On the next stage, POEM images are created by applying LBP u2 operator on AEM Is. Finally, the output POEM feature called POEM-HS (POEM histogram sequence), which is the concatenation of LBP histograms of all POEM images, is generated: P OEM = P OEM θ1 , ..., P OEM θm , P OEM θi = LBP H u2 AEM I θi [START_REF] Ojala | A comparative study of texture measures with classification based on featured distributions[END_REF] In their study, Ngoc-Son Vu et al. [START_REF] Vu | Face recognition using the poem descriptor[END_REF] made many experiments on the FERET database and got the optimal POEM parameters for face recognition as: an unsigned representation with three direction bins (m = 3) built on 7 × 7 pixels cell using LBP u2 [START_REF] Thai | On approaching heuristic weight mask to improve face recognition[END_REF][START_REF] Ojala | A comparative study of texture measures with classification based on featured distributions[END_REF] . These experiments were made on face images of 110 × 100 pixels. The similar optimal parameters were obtained when testing with images of around 90 × 90 pixels.

Heuristic Weight Search (HWS)

Based on proposed approach presented on a submitted paper [START_REF] Thai | On approaching heuristic weight mask to improve face recognition[END_REF], we made some changes and show our work here. In this section, we introduce an algorithm called Heuristic Weight Search (HWS) which uses heuristic search to find an optimal weight set. Firstly, we show how to prepare data for HWS. Then, fitness function evaluating goodness of a weight set, a way to prioritize patches and a complete HWS algorithm is presented in order.

Organizing data

A certain database is divided into three image sets for using through weight search and evaluating accuracy process: In process of HWS, only S G and S T r are used to generate an efficient weight set. S T s is needed in evaluating performance to prove a power of proposed algorithm. Detailed content is presented in section 6.

Weight set fitness function (WsFitness)

There are many face recognition systems using weight set to enhance their performance. It cannot be denied that weight set plays an important role on these systems. Therefore, anyone can also easily wonder how to know a suitable weight set for their data, or which is the best weight set from a several ones. To solve this problem, we suppose each weight set has its own fitness value for a certain image set. A basic idea of fitness value of a weight set is a recognition rate of system in case of using that weight set. The higher value is, the better weight set is. We call a function calculating fitness value of a weight set W sF itness.

I G ∈S G χ 2 W (I T r , I G )    |S T r | (6)
where label(X) is identification of image X and argmin x∈S (f (x)) shows the object of x producing the smallest value in computing with function f (x).

Different contributions of patches

Each region on a face has its own role in process of face recognition. Calculating contributions of all patches is needed before processing proposed algorithm. In this subsection, we define a single entry weights matrix denoted sEM (i) whose values are zero except an element in position of i. If we apply this matrix to face recognition process, only nonzero patch will affect recognition rate. Therefore, we call a fitness value of sEM (i) a contribution of ith patch. We also name the matrix containing these values a contribution matrix CM .

Heuristic Weight Search in detail

After calculating a contributions of all patches, heuristic search is applied to find an optimal weight set whose elements are scaled on [0, 1] range. First of all, we initialize a new weight set W n with 0.0 for all weights except that the most important patch, which gains the highest contribution, is assigned a value of 1.0. From now on, all changes in weight values are processed in W n . Then, weight values of other patches are calculated in descending order of the contributions of patches, which use a similar procedure presented as follows: (1) weight of a certain patch decreases steady from 1.0 to 0.0 by small static changing step of 0.05 (read 6.2 for more details); (2) the value maximizing W sF itness(W n ) is an optimal weight of that patch.

Heuristic Weight Search (HWS) algorithm 1. Compute a descending ordered contribution list CM . 2. Initialize new weight set W n = {0, }; set weight for the most important patch W n [index(CM [0])] = 1. 3. Loop for each remaining value in CM , find the best weight value for each patch, which is in range of [0, 1] and maximizes W sF itness(W n ). 4. The optimal weight set W n is ready for application.

In above presentation of HWS, index(X) returns a respective position of patch having fitness value of X.

Combination

In this part, we introduce a face recognition system applying POEM descriptor to HWS in order to show a boosting ability of the proposed algorithm. Fig. 7 presents the system integrating POEM descriptor into HWS algorithm, which is called HWS-POEM system. As can be seen from that diagram, our proposed system follows the procedure of normal face recognition system, except a few changes. That means people can easily apply HWS-POEM combination to their certain system to improve performance. Condition of input images make a huge impact on recognition result. If we apply a powerful descriptor on bad quality images, we might get a worse result than applying a normal descriptor on good pre-processed images. In this work, we use pre-processing tool of CSU Face Identification Evaluation System [START_REF] Bolme | The csu face identification evaluation system: its purpose, features, and structure[END_REF] to ensure that condition of images are better. Fig. 8 shows five steps in converting a PGM FERET image to a normalized image.

Pre-processing

Certainly, there are many better methods of image pre-processing, for instance, that of Ngoc-Son Vu et al. [START_REF] Vu | Face recognition using the poem descriptor[END_REF]. Although applying these methods improves performance, we only use CSU tools in all of our experiments in order to evaluate approaches on FERET database.

Discussion

This system inherits abilities from both POEM descriptor and HWS algorithm, so it has their characteristic.

POEM has many properties which are mentioned in [START_REF] Vu | Face recognition using the poem descriptor[END_REF]; two most important ones are: (1) POEM is an oriented feature so that it has the ability to capture image information with different levels of orientation accuracy, (2) using gradient magnitudes instead of intensity values makes POEM more robust to lighting variations than original methods of LBP.

HWS algorithm brings us a number of benefits. Firstly, HWS algorithm can be applied to any face recognition systems, provided weight set can be used in that system. In other words, if features of facial parts can be calculated separately using a face descriptor, we can use our algorithm there. Secondly, After finding weight for each patch, we could get the best weight in range of [0, 1]. With the first assigned value 1, a new calculated value could be better or equal to the weight of 1. Even in the worst case, we might still get the weight set whose elements are all 1, which means a new recognition rate of system applying weight set from HWS algorithm could not be less than that of original system. Finally, HWS algorithm just generate an efficient weight set without integrating into applications. Therefore, speeds of these applications are not be affected.

Experiments

Database

In our experiments, we use facial images from FERET database [START_REF] Phillips | The feret evaluation methodology for face-recognition algorithms[END_REF]. This database contains 14051 face images which are divided into many subsets based on their condition. According to FERET Tests September 1996, some frontal image subsets were classified as below, all changes of non-Gallery (we can name it Probe) images are in comparison with these ones in Gallery set. First of all, we made some experiments to compare effect on fitness (section 4.2) of weight change value in HWS process. Firstly, it can be easily to aware that a large step makes weight search faster. Secondly, as can be seen from Fig. 9, in the early stages of HWS process, both fitness graphs of 0.05 and 0.01 share the same pattern despite a difference of step value. It means effect of 0.05 step value on fitness is similar to 0.01 one. In addition, Fig. 10 shows the fitness values of a new weight set after searching weight for each patch. Although spending much time on calculating, the experiment using step of 0.01 reach the final fitness slower than that of another one. In another experiment in which a step value of 0.1 is used, it also reaches the final fitness slower than that of 0.05. This problem can be explained as 0.1 is such a large weight change that it pass through optimal weight values of patches while 0.01 is so small value that over fit training data.

Finally, using a small step of weight change like 0.01 is too waste and brings no more benefits than a medium value like 0.05 while a weight change of 0.1 leads to some missing optimal values. Therefore, 0.05 is a suitable weight step for HWS algorithm.

Result

We made experiments using HWS algorithm to find an optimal weight set for some LBP-based methods. Experiments of the same descriptor use the same parameters. In the There are some differences (worse or better) between results of our experiments and those reported in original papers (POEM [START_REF] Vu | Face recognition using the poem descriptor[END_REF]; Multi-scale LBP -MLBP [START_REF] Chan | Multi-scale local binary pattern histograms for face recognition[END_REF]; Gabor+LBP -LGBP [START_REF] Zhang | Local gabor binary pattern histogram sequence (lgbphs): A novel non-statistical model for face representation and recognition[END_REF]) because of different pre-processing methods (mentioned in 5.1) and matching algorithms (presented in section 2). However, in our experiments, each method gains a higher performance than its original when applying weight set found by HWS algorithm, which proves the effectiveness of proposed algorithm. As can be seen from the results of LGBP group, the performances of all experiments when applying our algorithm to find weight set are higher than those of LGBP approach and LGBP using weight set of Ahonen et al. [START_REF] Ahonen | Face description with local binary patterns: Application to face recognition[END_REF], except an experiment on FAFC subset where the recognition rate of LGBP using this weight set is higher than found weight set from HWS algorithm. This can be explained that LGBP is so suitable for illumination changes on images in FAFC subset that it does not boost significantly a final performance when applying HWS algorithm for this experiment. In this case, a hand labeled weight set like that of Ahonen et al. [START_REF] Ahonen | Face description with local binary patterns: Application to face recognition[END_REF] is good enough to use. This paper introduces a new algorithm named Heuristic Weight Search (HWS) to find an optimal weight set for any face recognition system applying it. The combination system of HWS algorithm and POEM descriptor also shows an ease of integrating proposed algorithm to a certain face recognition system. Experiments on FERET database helps to analysing the parameter of weight change and suggests a suitable step value. The performances of systems integrating proposed algorithm proves its ability in comparison with other approaches.

Fig. 1 :

 1 Fig. 1: Architecture of face recognition system in use.

Fig. 2 :

 2 Fig. 2: LBP (8, 1) operator.

Fig. 4 :

 4 Fig. 4: Ahonen et al.'s weight set for the weighted χ 2 dissimilarity measure. Black square indicate weight 0.0, dark gray 1.0, light gray 2.0, and white 4.0.

Fig. 5 :

 5 Fig. 5: POEM descriptor extracting procedure.

-

  Gallery set (S G ) contains images of everyone in database, in which each person has only one image. Both training and testing stage need to use S G to decide labels of images. -Training set (S T r ) includes images captured in different condition such as expression or illumination from those in Gallery set. It does not require containing images from all people in database. Training set is used in weight search process and does not involve with testing stage. -Testing set (S T s ) also has images captured in the similar condition to those in Training set, which is only used in computing accuracy of system.

I

  T r |I T r ∈ S T r , label(I T r ) = label(I), I = argmin

Fig. 6 :

 6 Fig. 6: Contribution matrix in an experiment.

Fig. 7 :

 7 Fig. 7: Architecture of HWS-POEM evaluating system.

1 . 2 .Fig. 8 :

 128 Fig. 8: Image normalization using pre-processing tool of CSU.

Fig. 9 :

 9 Fig. 9: Fitness changes in comparison between steps of 0.05 (left) and 0.01 (right) in weight search of the 2nd (top) and 4th (bottom) patches (based on an importance of patches). Vertical axis is fitness value; horizontal one is weight of a certain moment.

Table 1 :

 1 FERET frontal image subsets.For experiments, Gallery set of FERET database is used as Gallery set S G of HWS algorithm. Training set S T r and testing set S T s is generated by crossvalidation k-fold method with k = 2 for each FERET's Probe set. The final result of a certain Probe set is an average of experiments on that image set.

	Image set Quantity Description	Image list
	Gallery 1196	Single images of all subjects	gallery.names
	FAFB	1195	Images having changes in face ex-pression	probe fafb *.names
	FAFC	194	Images captured under different illu-mination	probe fafc *.names
	DUP1	722	Images taken between 0 to 1031 days after	probe dup 1 *.names
	DUP2	234	Subset of DUP1, contains images captured at least 18 months after	probe dup 2 *.names

Table 2 :

 2 table of results below, we call Ahonen et al.'s weight set by Ahonen and weight set found by Heuristic Weight Search algorithm by HWS. Fitness of new weight set after completely searching weight for each patch in comparison between steps of 0.05 (left) and 0.01 (right). Vertical axis is fitness value; horizontal one is the completion of new weight set (maximum value is 100). Boosting performance ability of HWS algorithm in comparison with original approaches.

	1									1
	0.9									0.9
	0.8									0.8
	0.7									0.7
	0.6									0.6
	0.5									0.5
	0.4									0.4
	1	11	21	31	41	51	61	71	81	91	1	11	21	31	41	51	61	71	81	91
	Fig. 10: Method					FAFB		FAFC			DUP1		DUP2
		LBP [2]					93.0		51.0				61.0			50.0
	LBP + Ahonen [2]			97.0		79.0				66.0			64.0
	LBP + HWS				98.33		84.07			67.88			71.90
		MLBP					92.38		28.28			61.09			49.59
	MLBP + Ahonen			97.24		58.76			65.65			61.11
	MLBP + HWS			98.24		85.07			66.49			71.79
		POEM					96.82		77.85			72.73			71.08
	POEM + Ahonen			98.22		86.88			71.83			70.21
	POEM + HWS			99.08		90.73			75.07			77.35
		LGBP					98.16		96.41			78.95			76.50
	LGBP + Ahonen				98.91		98.45			80.47			80.77
	LGBP + HWS				99.16		97.42			82.41			83.33

Acknowledgement

This research is funded by Vietnam National University of Ho Chi Minh City (VNU-HCMC) under the project "Features descriptor under variation condition for real-time face recognition application", 2014. We are also supported by research funding from Advanced Program in Computer Science, University of Science, Vietnam National University -Ho Chi Minh City.


Maciej Brzozowski
email: m.brzozowski@pb.edu.pl

Andrzej Chmielewski
email: a.chmielewski@pb.edu.pl

Hardware approach for generating b-detectors by immune-based algorithms

Keywords: artificial immune system, anomaly detection, hardware implementation

The most interesting feature of negative selection algorithms is ability for detecting novel, never met anomalies. This is especially important in security systems like intrusion detection, spam, virus detection, etc. However, the main problem is scalability which occurs for both: binary and real-valued representation. This paper describes a hardware implementations of the process of generating b-detectors which allows for a fast generation the receptors as well as a very fast recognition of anomalies in high-dimensional datasets.

Introduction

Natural immune system (NIS) prevents living organism against intruders called pathogens. It consists of a number of cells, tissues, and organs that work together to protect the body. The main agents responsible for the adaptive and learning capabilities of the NIS are white blood cells called lymphocytes. These differentiate into two primary types: B-and T-lymphocytes called also B-and T-cells for brevity. T-lymphocytes are like the body's military intelligence system, seeking out their targets and sending defenses to lock onto them. Next, B-lymphocytes, destroys detected invaders to protected the body. It is only a very short description of NIS; an unacquainted reader is referred e.g. to [START_REF] De Castro | Artificial Immune Systems: A New Computational Intelligence Approach[END_REF] for further details.

The mechanisms and procedures developed within NIS were an inspiration for Artificial Immune Systems (AIS). One of them is negative selection oriented towards fast and efficient discrimination between own cells (called self ) and pathogens (called nonself ). Additionally, it helps the body against self-reactive lymphocytes. A nice feature is that it does not need examples of nonself samples (counterpart of pathogens) to detect them. The information about own cells is sufficient. Hence, every organism has a unique "protection system", capable of detecting even new type of attacks. This is an inspiration to build a computer security systems (e.g. firewall).

A key problem when applying negative selection algorithm (NSA) in real-life application is scalability. To detect spam or intruders at computer networks NSA should be able operate on high-dimensional data. Moreover, in such domains, the classification is performed online, without significant delays, what makes this task much more difficult to solve. This descriptions correspond to NIS, where only fast and effective response on intruders activity can protect organisms against damaging or even die. To overcome this problem, many solution were proposed, including hybrid b-v model [START_REF] Chmielewski | Hybrid negative selection approach for anomaly detection[END_REF], emloying binary (called b-detectors) and real-valued detectors (called v-detectors) to increase the efficiency, especially in case of classification process.

In this paper, we present hardware implementations of NSA for b-detectors, where most of computationally complex operations can be paralellized and done within a few ticks of clock. This is the first step towards hardware firewall embedded in a reprogrammable FPGA (see Section 3). Such solution makes that rules in security systems are not hardcoded and updating the set of detectors is possible. Finally, there are compared two possible approaches: combinational and pipeline.

Negative Selection Algorithm

The NSA, proposed by Forrest et al., [START_REF] Forrest | Self-nonself discrimination in a computer[END_REF], is inspired by the process of thymocytes (i.e. young T-lymphocytes) maturation: only those lymphocytes survive which do not recognize any self molecules.

Formally, let U be a universe, i.e. the set of all possible molecules. The subset S of U represents collection of all self molecules and its complement N in U represents all nonself molecules. Let D ⊂ U stands for a set of detectors and let match(d, u) be a function (or a procedure) specifying if a detector d ∈ D recognizes the molecule u ∈ U. Usually, match(d, u) is modeled by a distance metric or a similarity measure, i.e. we say that match(d, u) = true only if dist(d, u) ≤ δ, where dist is a distance and δ is a pre-specified threshold. Various matching function are discussed in [START_REF] Harmer | Artificial immune system architecture for computer security applications[END_REF], [START_REF] Dasgupta | Revisiting negative selection algorithms[END_REF].

The problem relies upon construction the set D in such a way that

match(d, u) = false if u ∈ S true if u ∈ N (1)
for any detector d ∈ D.

A naive solution to this problem, implied by biological mechanism of negative selection, consists of five steps: So far, there were considered two types of detectors: b-and v -detectors [START_REF] Dasgupta | Real-valued negative selection algorithm with variable-sized detectors[END_REF]. Usually, each of them were used separately, except b-v model [START_REF] Chmielewski | Hybrid negative selection approach for anomaly detection[END_REF]. In this paper, we focus only on binary representation.

In case of binary encoding, the universe U becomes l-dimensional Hamming space, H l = {0, 1} l , consisting of all binary strings of fixed length l:

H l = {000...000 l , 000...001 l , . . . , 111...111 l }
Hence the size of this space is 2 l . The most popular matching rules used in this case are:

(a) r-contiguous bit rule [START_REF] Forrest | A sense of Self for Unix Processes[END_REF], or (b) r-chunks [START_REF] Balthrop | Coverage and generalization in an artificial immune system[END_REF].

Both the rules say that a detector bonds a sample (i.e. data) only when both the strings contain the same substring of length r. To detect a sample in case (a), a window of length r (1 ≤ r ≤ l) is shifted through censored samples of length l. In case (b) the detector t i,s is specified by a substring s of length r and its position i in the string. Below an example of matching a sample by r-detector (left) and r-chunk for affinity threshold r = 3 is given

l 1 0 0 0 1 1 1 0 ← sample → l 1 0 0 0 1 1 1 0 0 1 0 0 1 r 0 0 1 ← r-detector; r-chunk → * * 0 0 1 r * * *
Here it was assumed that irrelevant positions in a string of length l representing the r-chunk t 3,001 are filled in with the star (*) symbol. This way r-chunk can be identified with schemata used in genetic algorithms: its order equals r and its defining length is r-1. Although a single r-detector recognizes much more strings than a single r-chunk, this last type of detector allows more accurate coverage of the N space [START_REF] Balthrop | Coverage and generalization in an artificial immune system[END_REF].

Further, the notion of the ball of recognition allows to define "optimal" repertoire D. Namely it consists of the detectors located in H l in such a way that they cover the space N and their balls of recognition overlap minimally. A solution to such stated problem was given in [START_REF] Wierzchoń | Generating optimal repertoire of antibody strings in an artificial immune system[END_REF]. To construct the r-detectors we split all the self strings into the templates represented identically as the r-chunks and we construct the detectors by gluing these r-chunks that do not belong to the set S. More formally, if t i,s and t j,w are two candidate r-chunks, we can glue them if both the substrings are identical on r-1 positions starting from position i+1.

Using such an optimality criterion we come to the conclusion that shortest detectors are more desirable as they are able to detect more samples. However, Stibor [START_REF] Stibor | Phase transition and the computational complexity of generating rcontiguous detectors[END_REF] showed the coherence between r and l values for various cardinalities of S in terms of the probability of generating detectors, P g . He distinguished three phases:

-Phase 1 (for lower r) -the probability P g is near to 0, -Phase 2 (for middle r) -the probability P g rapidly grows from 0 to 1 (so called Phase Transition Region), -Phase 3 (for higher r) -the probability is very near to 1.

Hence, we should be interested in generating detectors with medium length r (belonging to the second region) and eventually with larger values of r if the coverage of N is not sufficient.

In case of software approach, the detectors can not be too long, due to exponential increase in the duration of learning process, which should be finished in reasonable time. That was a main reason why b-detectors were not used to solve problems in high-dimensional datasets. In litetaure, we find that longest binary samples (l = 49) were used to model the system for monitoring TCP SYN packets to detect network traffic anomalies (called LISYS) [START_REF] Hofmeyr | Architecture for an Artificial Immune System[END_REF]. In Section 3, we show, that detectors even with l = 95 can be generated in just a few ticks of clocks in case of hardware implementation of NSA algorithm.

Hardware implementation

Traditional software firewalls when analyzing and filtering packets flowing through the network use the processing power on which they are installed. This can lead to a significant reduction in responsiveness of the computer during a network attack. An alternative to software solutions are hardware firewalls, however, they are relatively expensive to use.

Another solution are firewalls embedded in a reprogrammable FPGA (Field Programmable Gate Array) architectures characterized by a high degree of flexibility during the design process. Usage of HDL (Hardware Description Language) languages reduces the cost of design and allows to transfer design between different architectures from manufacturers such as Xilinx or Altera. The use of reprogrammable architecture for the construction of a firewall reduces the cost of the final solution and increases the number of classified packets compared to pure software solution.

Construction of FPGA allows for parallelization of calculations and algorithms, so that they have wide range of applications during the HPC process (High Performance Computing). Another advantage of FPGAs is their low power consumption compared to the GPU (Graphics Processing Unit) solutions used in HPC, so that they represent better value performance-per-watt power consumption [START_REF] Vanderbauwhede | High-Performance Computing Using FPGAs[END_REF].

The use of FPGAs significantly shortens time to market (the length of time since the inception of the product concept to placing it on the market) compared to system based on ASIC (Application Specific Integrated Circuit).

The biggest advantage of FPGAs is that they can be reprogramed even after they have been installed in the target system. This allows to correct the errors or complement the design with new functionality. In the case of ASIC the process of design and manufacturing should be repeated and then replace a malfunctioning chip in the target system, which is associated with high costs.

The research are focused on building firewall with high throughput and latency as low as possible through the use of reprogrammable architectures. De-signed in this way, the firewall will work independently without supervision of other (external) devices.

FPGAs contain programmable logic (configurable logic blocks) contains a set number of LUTs (LookUp-Table -small memory generate boolean function), flip-flops and multiplexers connected via programmable interconnects.

As the implementation language of hardware firewall has been chosen VHDL (Very High Speed Integrated Circuits Hardware Description Language) because is the most supported hardware description language by synthesis software and source code simulators.

Combinational component approach

In initial stage of work on the hardware firewall design was considered as a purely combinational circuit. The advantage of this approach is the clarity of its timing behavior (input/output response) analysis in oder to verify its correctness. Another advantage of this approach is the eases of making changes in the design description. Figure 1 shows a simulation of generating detectors for combinational component approach. In order to increase the readability of input/output values, results of operations component the were limited to a l = 16 for each sample and r = 6 for r-chunks. On simulation sample was market as Self -on the input might be inserted signal values of Self/NonSelf; detectors as P attern -r-chunks; and the Result as the output of designed unit. V alue of 1 on output denotes that the detector bonds a sample. For example, if Self = {0000000000101000} and P attern = {000010} then Result = 1 (Self and P attern are matched). The same behavior is observed for larger values l = 64 and r = 32 or l = 128 and r = 64 etc. The only restriction is length of 1 ≤ r ≤ l.

Diagram on Figure 2 is schematic representation of the combinational unit design in terms of logic elements optimized to the target Xilinx Virtex 4 device xc4vfx12-12sf363. Technology schematic was generated for l = 6 and r = 3 (design occupies 10 LUT tables) because generating schema for lengths greater than the specified would lead to difficulties in the analysis. Detailed schematic generated for l = 16 and r = 8 occupies more than four pages and 53 LUT tables.

The proposed solution indicates very regular structure. Therefore, a different approach -pipelined -should be considered.

Pipelined approach

Throughput and delay are two critic performance criteria for designed component. Delay is the time that one task required to be completed, and throughput is the number of tasks that can be completed in one unit time. The major step of adding pipeline to immunologic anomaly detection design into stages. As was mentioned before combinational version indicates very regular structure. Comparison of r-chunk vector witch parts of self/nonself vector is realized on LUT tables which have the same or very close critical propagation delay time. An analogous situation occurs with the logical summation of the results of the comparison which is realized by a cascade connected LUT tables. Therefore when converting the combinational version of immunologic design to pipeline component version the system delay would not change but the system throughput will increase. 1 shows comparison of two approaches, combinational and pipeline, of matching patterns and samples. All measurements were made for a Xilinx xc3s250e-5pq208 device from Spartan3E family. Parameter tc describe maximum combinational path delay for combinational version of matching system, tpmaximum combinational path delay for pipeline version and n how many ticks of the clock is needed to compute output value. It means that for the length l = 47 and r = 32 combinational system version may match 54M of sample/pattern pairs and pipeline -138M -but the system response take 5 clock ticks (36,075ns).

Comparison of hardware implementations

Conclusions

Hardware implementations of negative selection algorithm are the way to overcome the scalability problem. The main advantage of this solution is a very short duration of both processes: generating detetors and censoring, as the most complex operations can be parallelized and aditionally computed without CPU utilization. Hence, in comparing to software approach, b-detectors can be generated very fast even for very long binary representations. Also, the result the overall duration of classification was significantly reduced and makes this solution possible to apply in on-line filtering the network connections.

There were compared two hwardware implementations: combinational and pipeline. Pipeline is commonly used technique to increase the performance of the design. Through a combination of pipelining and parallel processing, which is an advantage of FPGA devices, increases number of classified packets in network traffic using immunological methods.

This article is a first step for building dedicated (unique system security) hardware firewall with abbility to detect new types of attacks. In next step, implementation of V -Detector algorithm on FPGA will be performed to build hardware version of b-v model.

  (a) Initialize D as empty set, D = ∅. (b) Generate randomly a detector d. (c) If math(d, s) = false for all s ∈ S, add d to the set D. (d) Repeat steps (b) and (c) until sufficient number of detectors will be generated.

Fig. 1 .

 1 Fig. 1. Combinational component approach: simulation waveform (timing diagram) of anomaly detection.

Fig. 2 .

 2 Fig. 2. Combinational component schema mapped into technology.

Fig. 3 .

 3 Fig. 3. Pipeline approach: simulation waveform (timing diagram) of anomaly detection.

Figure 3

 3 Figure 3 shows a simulation censoring samples for pipeline component approach. In order to increase the readability of input/output values results of operations component the were limited to a l = 23 for each sample and r = 8 for r-chunks. Clk is a signal used for computing stages synchronization. Test signal value is delayed for 4 ticks of the clock. It means that the system needs 4 ticks of the clock to calculate value of Test signal and its directly depend on length of l and r.

Table 1 .

 1 Comparison of combinational and pipeline versions.Table

	l	r tc [ns] tp [ns] n
	11 8	9.101	7.234 3
	23 8 11.133 6.869 4
	71 8 20.201 7.486 5
	35 32 11.801 7.454 4
	47 32 18.335 7.215 5
	95 32 14.615 7.906 6

Acknowledgment

This work was supported by Bialystok University of Technology grants S/WI/3/13 and MB/WI/1/2014.