Alain Lelu

Martine Cadot

Loria

Winning contribution to Clustering Benchmark Challenge -IFCS 2019

Keywords: Evaluation method, method comparison, text clustering, K-Means, Normalized Matrix Factorization, Latent Dirichlet Allocation, hierarchical clustering, linkage method, spectral clustering, graph partition, kernel clustering

Fair evaluation of text clustering methods needs to clarify the relations between 1)pre-processing, resulting in raw term occurrence vectors, 2)data transformation, and 3)method in the strict sense. We have tried to empirically compare a dozen well-known methods and variants in a protocol crossing three contrasted open-access corpora in a few tens transformed dataspaces. We compared the resulting clusterings to their supposed "ground-truth" classes by means of four usual indices. The results show both a confirmation of well-established implicit combinations, and good performances of unexpected ones, mostly in spectral or kernel dataspaces. The rich material resulting from these some 600 runs includes a wealth of intriguing facts, which needs further research on the specificities of text corpora in relation to methods and dataspaces.

-Introduction : motivations and goals.

Evaluation of text clustering methods is one of the key issues in the problem of bibliometric delineation of scientific fields. As co-authors of [Zitt et al. 2019] we have tried to test seventeen clustering methods with a publicly available real-life test set, the Reuters' test bench [START_REF] Lewis | Rcv1: A new benchmark collection for text categorization research[END_REF]] which adds up several difficulties of text clustering, i.e. mainly strongly unbalanced man-made classes (the targeted "ground-truth"), and texts of unbalanced sizes. Our report is accessible online [Cadot, Lelu 2018] as a supplementary material to the above-mentioned book chapter. An unexpected result was that antique agglomerative methods, especially Ward hierarchical clustering, performed better than many more recent ones. Was it the case for all types of corpora? Above all we realized that for the sake of fair comparisons, as well as conceptual clarity, we should clearly separate the transformations of the raw word-count data (for example into Salton tf-idf vector representation, or Laplacian spectral space, etc.) from the algorithms in the strict sense, instead of using long-time accepted implicit combinations. For example, no rational reasons forbid using e.g. Non-negative Matrix Factorization in a spectral space. This consideration is in line with the conceptual clarifications operated in [Van Mechelen et al. 2018]; last, but not least, unexpected recommendations may proceed from nonclassic combinations. This clarification is one of our guiding threads in the present research.

Though restricting our scope to text clustering, it is clear that many types of texts need now to be processed: abstracts or plain texts of scientific papers, which are our primary scientific interest, or journal, litterary or legal texts, or texts originating in the social nature of Internet communications, such as contributions to forum discussions, or social networks. We decided to base our present survey on three typical and contrasted test sets: a full-text scientific database, a wire of press agency, and an Internet discussion forum. It is clear that the complete text preprocessing chain is out of the goal of our research, so we have to rest on one same linguistic -or weakly linguistic -term, lemma or stem extraction scheme, and same elimination of infrequent or too frequent words. This point must not keep us from exploring the influence of truncating the remaining vocabulary in chosen distribution quantiles. Usual benchmark studies mention an absolute occurrence threshold, period... All these specifications led us to the choices we expose in the methodology section.

Of course the options on methods and types of dataspaces to be considered are inevitably somehow arbitrary: we tried to take account of the most usual algorithms, or method families, such as K-means, hierarchical agglomerative clustering, spectral clustering, graph clustering, kernel clustering, and we added two more specific methods, i.e. Nonnegative Matrix Factorization (NMF) and Latent Dirichlet Allocation (LDA), which amounts to a dozen methods and variants.

Concerning dataspaces, we chose to add to the plain term occurrence vector space the transformed spaces by Salton's and Okapi's tf-idf weighting schemes, by chi-square metrics, by Laplacian spectral decomposition, by Correspondence Analysis, and last by order-2 polynomial kernel expansion. These transformations are referenced and formally presented in the next section.

Given the combinatorics of the three main elements -text types, dataspaces and algorithms -our research study could be nothing but an exploration, strongly constrained by available resources. However, some interesting conclusions will be drawn out this exploration. In the final conclusion, we will deal with what may be continued and deepened in our perspective, given the results.

Let us close this introduction saying that we are indebted to the remarkable initiative of the Brazilian LABIC team [START_REF] Rossi | Benchmarking Text Collections for Classication and Clustering Tasks[END_REF]] who homogeneously pre-processed [LABIC stemmer] some forty text collections and made the document by term matrices available online on their site [LABIC data].

-Methodology

Drawn from our experience of clustering methods, our first claim is that using the supervised learning methodology for comparing non-supervised methods is in any case better than relying on unfounded claims or comparisons to be suspected of author's bias. In relation to the issue of author's bias, let us state that, though being authors of a few clustering algorithms (Axial K-means, Local Component Analysis, Germen), we have excluded these algorithms of our survey. This supervised methodology may at least result, in default of a universal ranking of methods, in fruitful reflections on the typology of texts, or the nature of the human categorization and abstraction process and its similarities and differences with methods mostly optimizing an intrinsic objective function.

Another core imperative we have set is transparency and reproducibility: in addition to the direct link to the documentby-term matrices we have provided above, a complementary material HAL site [HAL CNRS] will gather the data and code we used. Though most algorithms are theoretically insensitive to the ordering of input vectors, in practice we experienced that tied effects, among others, could affect the results. This is why we have randomly scrambled the data vectors, and will drop our scrambled data and label files on the above-cited site.

-Choice of test corpora

The three "prototype" test corpora mentioned in the introduction are, first, Reuters' "ModApté Split" [START_REF] Apté | Automated learning of decision rules for text categorization[END_REF]] limited to the eight most important classes ("Re8" in the present study, 7674 documents,8901 terms), second, the ACM collection made of the proceedings of forty conferences in different computer science areas (3493 papers, 60 768 terms), third, the "20 Newsgroups" collection ("Ng20") composed of 18 808 messages posted in twenty Usenet groups (45 434 terms). The size of the man-made reference classes is strongly unbalanced in the case of Re8 (two of them constitute 81% of the documents), roughly equal in the case of ACM and Ng20.

It is to be noted that the sole Reuters' class labels are issued from a direct manual indexing. The two others origin in the concatenation of sub-corpora of comparable size. They could therefore be considered as "semi-real-world data", not really representative of real-life non-annotated corpora. Note also that the mean "Silhouette" coefficient [Kogan, 2007], which measures the inter-class contrasts, is higher (.89) in the case of Re8 than in Ng20 (.80) and ACM (.75) ones.

-Truncating the vocabularies

As the size of the vocabularies are unbalanced (Re8: about 8900 terms, ACM: 60 800 terms, Ng20: 45 000 terms) but extensive (the hapaxes, i.e. terms of total occurrence one, are included in this count), we decided a common scheme for a vocabulary-independent truncation by thresholds: in addition to the basic option of retaining the whole vocabulary, we built two sub-corpora per test corpus retaining the third quartile of the term distribution (25% of the total occurrences), and the seventh "octile" (12.5%).

-Choice of clustering methods

We have affected a lower priority to algorithms with two parameters (DbScan, Affinity Propagation, Smart Local Moving Algorithm) or one parameter with deceptive results on Reuters' corpus (Density Peaks, Independent Component Analysis, Fuzzy c-means, K-Means++). We selected:

-First, plain K-means clustering ("KM"), see [START_REF] Mac Queen | Some methods for Classification and Analysis of Multivariate Observations[END_REF] algorithm (in the adaptive case) and [Forgy 1965] (in the usual iterative case), initialized by randomly drawing data vectors -we had an unconvincing experience of K-means++ initialization in the context of text clustering. We implemented 20 elementary runs, or "passes", per run, selecting the "best" one in terms of the local optimum of the K-means objective function (sum of squared intra-cluster Euclidean distances).

-Hierarchical agglomerative clustering with two linkage variants: average link ("HCa"), and Ward ("HCw") [Ward 1963]. Originally in O(#documents 3 ) time complexity, the more recent contributions [Murtagh 1984] and [START_REF] Müllner | fastcluster: Fast Hierarchical, Agglomerative Clustering Routines for R and Python[END_REF]] have lowered this constraint to O(#documents 2 ).

-Spectral clustering [START_REF] Meila | [END_REF]: we used the "standard" combination K-means/Laplacian spectral dataspace, but also explored (with success, see below) many other combinations.

-Graph clustering methods: we chose the two most broadly recognized ones, i.e. Louvain [Blondel et al. 2008] and InfoMap [START_REF] Rosvall | [END_REF]. Note that these methods, in contrast to all the other tested ones, do not need fixing a desired number of clusters, hence a major operational advantage when no idea of the "true number of clusters" is known beforehand -hierarchical clustering being in an intermediate position, as in one run it leaves the choice of the cluster number to the user.

-Non-negative Matrix Factorization ("NMF") [START_REF] Lee | [END_REF]: this decomposition is akin to be used as a clustering method, when the label of a document is attributed depending on the axis of its maximum projection. As this method converges to local optima of its objective function, we implemented the same "20 runs" strategy as for K-means. Note that the resulting gradual representation of documents in the clusters is also interesting in that outliers, or even additional cluster seeds, may be identified -one kind of "possibilistic" clustering.

-Latent Dirichlet Allocation ("LDA") [Blei et al. 2003] is well-known and much respected as deeply founded in theoretic grounds.

-Kernel clustering [Girolami 2002]: thanks to the "kernel trick", a document by document similarity matrix ("Gram matrix") is built without explicit expansion of the raw dataspace by a kernel function. Here we used an order-2 polynomial kernel, which amounts to take into account the wholeness of the 2-term itemsets in each document when comparing one to another (in addition to standard "1-term" itemsets). In this case the raw dataspace is not made of numeric occurrence vectors, but of binary existence ones.

-Choice of dataspaces:

In addition to the plain term-occurrence vector space, we have considered and built: -Salton's vector space, weighted by the classic tf-idf scheme: xij = tf idf where tf = nij idf = log n -log e.j Notations: N is the document by-term-matrix, n ij is the occurrence number of term j in document i, n i. is the total occurrences of terms in document i, n .j is the total occurrences of term j, n.. is the grand total of occurrences in the corpus, n is the number of documents.

E is the binary document by term matrix, e ij is 1 if term j exists in document i, 0 else, e i. is the total number of terms which appear in document i, e .j is the number of documents in which term j appears, e.. is the grand total of term appearances in the corpus. k* is the number of manually attributed classes in each corpus.

-Okapi (also coined BM25) vector space [START_REF] Robertson | Okapi and TREC-2[END_REF], with a more cryptic, but statistically grounded, weighting scheme [START_REF] Robertson | Okapi and TREC-2[END_REF]]: xij = tf idf tf = nij (a+1) / (nij + a(1-b+b(ni.n/n..)) ; recommended values: a=1.2 b=.75 idf = log(n -e.j + .5) -log(ni. + .5) For enabling idf to be always positive or zero, we have ceiled the value (e.j+ni.) to n.

-Chi-square metrics, which amounts to a Euclidean vector space with vectors transformed as suggested in [Legendre, Gallagher 2001

]: xij = n.. 1/2 nij / (ni. n.j 1/2 ) -Laplacian spectral space [Von Luxburg 2007]:
Given Cos, the (truncated or not) cosine matrix between the documents, which defines a non-oriented graph, and s the sum vector of its rows or columns, the transformed matrix Qlapl = diag(s -1/2 ) Cos diag(s -1/2 ) gives rise to a Singular Value Decomposition (SVD):

Qlapl = Ulapl D Vlapl
The first rank factors concentrate the relevant information. When aiming at k* classes, we chose to take into account three configurations of Ulapl, with respectively integer(k*/2), k* and 2k* top-ranking factors.

-Correspondence Analysis spectral space [Benzecri 1973] [Greenacre 1984] [Lebart et al. 1998]:

The transformed matrix Qca = diag{ni. -1/2 } N diag{n.j -1/2 } gives rise to a SVD decomposition Qca = Uca L Vca resulting in the CA factors:

F = n.. 1/2 diag{ni. -1/2 } Uca L
As the first factor is trivial, we chose to take into account three configurations, with respectively 1+integer(k*/2), 1+k* and 1+2k* top-ranking factors.

Note that Euclidean distances in the complete factor space equal chi-square distances [Benzécri 1973]. Therefore, truncating this space in this manner amounts to consider "partial chi-square" distances, a priori more relevant than chi-square distances. We will check this point below.

These six transformations of a document-by-term matrix are convenient for the KM, NMF, LDA and Spectral Clustering methods. Other methods, such as Hierarchical Clustering, Graph methods and Kernel methods, need a document-by-document similarity (or dissimilarity) matrix. We have derived the distance or cosine matrices for all the above-mentioned dataspaces. Depending on each dataspace-method combination, we have used Euclidean distance or "cosine" distance (i.e. 1-cosine, which weights half of the squared chord distance [Legendre, Gallagher 2001]).

-Kernel space [Girolami 2002]:

In matrix notations and in the case of order-2 polynomial kernel, the Gram similarity matrix writes:

Xk2 = (1 + E E') .^ 2
where X .^ 2 means squaring each value of matrix X Given the much contrasted values in this matrix, the cosine distance is well-fit to this dataspace.

-Choice of evaluation measures

We chose the four most usual indices encountered in the evaluation literature, i.e. first, Normalized Mutual Information (NMI) [Cover, Thomas 1991] and Adjusted Rand Index (ARI) [Rand 1971], which compute independently from the number and labels of clusters; and second, mean local class-vs.-cluster F-scores (F) [Van Rijsbergen 1979] and global Purity score (i.e. 1-global error rate) which need the same number of clusters and classes, and same labels. Even when this was not the case (graph methods), we have aligned the k classes and the k most "analogue" clusters, in the sense of local F-scores, by means of the ranking issued from the first non-trivial factor in the Correspondence Analysis of the classes by clusters F-score matrix.

-Code implementation and computer efficiency

As computer efficiency is out of our goals, we implemented the data transformations, method code, and postprocessing code in an Octave environment, on an Intel 6-core I7, 3.33GHz, 48Go RAM computer. Method codes were derived from existing Matlab® codes (links to the original pieces of code are available in the supplementary material). Their degree of computing time optimization varies considerably: e.g. in the case of the 19 000 documents Ng20 test set, from 2 minutes for twenty elementary runs of the standard "litekmeans.m" code, to 6 hours for one run of Louvain method.

-Evaluation results for each method

We have tried as much as possible to cross-combine corpora × data transformations × methods. This was not always possible, due to constraints such as computing time or resources devoted to systematically poor results.

-K-means

Altogether with hierarchical agglomerative clustering, this method is an inescapable classic of exploratory data analysis. Our results confirm why it is so widely used till now: good performances (nmi>.60 whatever the corpus), low computing complexity O(#documents × #terms), fast runs as noted above. The vocabulary truncation does not seem to be quite influential, except in the case of the Chi-square space. So one recommendation may be drawn: a 12,5% vocabulary threshold seems good enough for acceptable K-means results, resulting in a lighter processing task.

Examining performances, the results are surprising: Okapi dataspace performs best for ACM, Salton's space for Ng20, and both perform equally for Re8, as well as raw occurrence dataspace. Chi-square space performs poorly, but we will see that partial Chi-square space performs much better, when limited to the main Correspondence Analysis factors.

Conclusions would need a specific study of the peculiarities of text corpora with regard to these disorientating kinds of observations. This more recent method confirms the surprising observation above: ACM and Ng20 yield good performances in Okapi space, whereas Re8 does in Salton's space. The somehow chaotic performances and the unpredictable computing time (10 to 40 minutes) led us to not process the 3-times bigger Ng20 corpus.

-NMF

-LDA

However, it seems that LDA is more fit to the raw Euclidean dataspace, perhaps more in line with its underlying statistical hypotheses. Another observation: the more the vocabulary is truncated, the better the result. At first glance, it seems that the average link variant performs correctly with Reuters'8, but not with ACM -another intriguing fact. Ward linkage variant on raw, non vocabulary-truncated corpus, seems also a clear winner.

-Linkage methods

Figure11 -Hierarchical Clustering on Ng20 corpus

Using Okapi dataspace outperforms Salton's and Raw ones for ACM and Ng20 corpora, in line with the observations above. An unexpected result is that spectral HC family seems to outperform all other methods tested yet on the ACM corpus. Not so good, purity is privileged, seemingly due to its good performance in the two major clusters. Spectral Hierarchical Clustering with Ward linkage option clearly outperforms average linkage one, all the more so as average linkage option is far more time-consuming.

-Spectral clustering

ACM corpus

Re8 corpus

-Kernel clustering

When using a second-order polynomial kernel, this more recent method is not based on term counts, unlike all other methods, but on binary presence/absence vectors, which makes it possible to take into account term 2-itemsets, i.e. supersets of term bigrams. As semantic units seem to be embedded in word bigrams or trigrams more than unigrams, at least in technical texts, this dataspace should result in another meaningful view of the data. It is indeed the case, the surprise being that usual Kernel K-means is clearly outperformed by Ward-linkage clustering in the kernel space. We have not heard of this result elsewhere in the literature.

Given the long computation time on Ng20 corpus (3 hours), we performed only two runs with a disappointing outcome: at best, NMI=.404, ARI=.202, Fscore=.313, Purity=.336 on a 25%-truncated vocabulary. The performance is not so good as using spectral HC-Ward, but shows a slight indifference to thresholds, applied to vocabulary as well as cosines. Thus a "graph-like" sparse cosine matrix does not seem to be a clear advantage, contrary to our initial guess.

-Graph clustering -Louvain

This guess led us to test a single run of Louvain (6 hours) on a doubly truncated Ng20 space (25% truncated vocabulary, 25% truncated cosines) but the performances were poor: NMI=.302, ARI=.116, Fscore=.114, Purity=.211, which seems to confirm the above statement.

-InfoMap The performance is good, especially in the Okapi and Salton's dataspaces, with no sensible influence of the thresholds.

We tested two InfoMap runs on two Ng20 spaces (3 hours each), both 25% cosine-truncated, the first with 100% vocabulary, the second with 25%, resulting in deceptive scores, at best: NMI=.356, ARI=.162, Fscore=.213, Purity=.290.

-Synthesis

Let us focus first on the measurement tools: we can observe that in the case of the two "balanced" corpora, the four evaluation indices behave in a much parallel and orderly manner -see fig. 1 , 3, 4, 6, 25. In contrast, this parallelism and regular ranking deteriorate in the Reuters'8 unbalanced corpus, and to a lesser extent when hierarchical methods are used. F-score and Purity indices may (see fig. 17, 28) or may not present (see fig. 16, 20, 21, 25) a somehow contradictory or non-monotonic behavior. A thorough investigation could perhaps explain these interesting discrepancies, but is clearly out of our present goals. We have thus chosen the more stable NMI index as a reference measure for ranking each corpus' runs (ACM: 246 runs, Re8: 237 runs, Ng20: 109 runs, summing up to 592 runs).

4 -1 -"Top three" methods (NMI criterion):

ACM corpus:

1) Spectral Hierarchical Clustering: HC-Ward on vocabulary-truncated or not, Salton-weighted corpus, in Laplacian space with or without cosine truncation (50% or 25%), with k* (i.e.40) or 2k* (i.e.80) factors. NMI: .6980 to .6714. Elapsed time: 77".

2) Standard Hierarchical Clustering: HC-Ward on non-vocabulary-truncated , Okapi-weighted corpus. NMI: 6739. Elapsed time 75"

3) Spectral K-Means: cosine ("spherical") K-means on 12.5% vocabulary-truncated , Okapi-weighted corpus, in Laplacian space without cosine truncation and 2k* factors. NMI: .6712. Elapsed time: 6".

Reuters'8 corpus:

1) Standard K-Means : cosine ("spherical") K-means on non-vocabulary-truncated , Okapi-weighted or Salton-weighted with or without 12.5% vocabulary truncation corpus. NMI: .6461 to .6314. Elapsed time ~10".

2) Spectral Hierarchical Clustering: HC-average on 12.5 %-truncated vocabulary, in k* (i.e. 8 dimensions) CA space with cosine measure. NMI: .629. Elapsed time ~24h.

3) Kernel Hierarchical Clustering: Hc-Ward on order-2 kernel Gram matrix, issued from 12.5% vocabulary-truncated corpus. Elapsed time ~800".

NewsGroups corpus:

1) Standard Non-negative Matrix factorization: NMF on non-vocabulary-truncated, Okapi-weighted corpus; NMI:.6252. Elapsed time: 107".

2) Spectral Hierarchical Clustering: HC-Ward in 2k* (i.e. 40 dimensions) CA space with cosine measure; vocabulary is not truncated; NMI:.6220. Elapsed time ~4h.

3) Standard K-Means : cosine ("spherical") K-means in Salton's dataspace, with or without 12.5% or 25% vocabulary truncation; NMI:.621. Elapsed time: ~150".

These optimal runs clearly depend on the corpora. A large variety of dataspace transformations (truncated or not vocabulary, Salton's, 0kapi, or raw dataspace, kernel or Laplacian spectral space, ...) and methods (HC-Ward, K-Means, NMF) are present. It can be noted that four methods upon nine may be considered as "classic", i.e. K-Means on Salton's or Okapi dataspace, Standard Hierarchical Clustering, and Non-negative Matrix factorization.

-2 -Limits

Now let us investigate what the maximum index values mean in the real world, coming back to the data. The correspondence table between classes and best cluster partition for Ng20 corpus (see figure 29) shows off a "snake" structure, more than a diagonal one: some clusters totally or partially correspond to several classes, and conversely. The #3 and #9 classes are dispersed through many clusters. The global 0.62 NMI and 0.57 Purity are not so good, after all, and this is confirmed by the 0.53 mean F-score. The reasons of the divergence between man-made categories and clusters should be thoroughly investigated in relation to experts of the application field, by examining for example the case of classes 3 and 9. This process may perhaps converge to a consensual "ground truth", or diverge, showing off the limits of the sole textual information -or the limits of a weakly linguistic term extraction, in the present case ? To come back to figures 1 to 28, the evenness of many visually eye-catching weird observations shows that nonclearly elucidated phenomena and interactions are at work: e.g. the role that could be played by the size unevenness and number of the clusters, by the size of the vocabulary, by the type and style of texts, when matched against multiple potential classificatory points of view, by density contrasts in dataspaces, and so on. This opens up a whole unexplored research field.

-3 -Commonalities

Aside from that, let us search for commonalities. Examining the "top 50" runs of each corpus (ranked by decreasing NMI values), a few common behaviors emerge.

-Partial commonalities:

In ACM corpus: the HC-Ward method combinated with all the variants of the Laplacian Okapi-weighted space dominates massively; the spectral KMeans in the same spaces joins the list nine times, HC Ward standard and NMF standard join two times and three times respectively.

In Re8 corpus: Okapi-weighted or Salton-weighted standard K-Means dominates, followed by spectral HC-average in the CA factor space, then by spectral K-Means in the CA (sometimes Laplacian) factor space; three kernel HC-Ward join the list, as well as six standard NMF and three Louvain methods in the Salton space.

-In Ng20 corpus: standard K-Means comes out on top, together with NMF Okapi in the CA factor space and spectral HC Ward in the same space. Next come Salton-weighted (sometimes Okapi-weighted) spectral K-Means, and spectral HC Ward in CA space. Kernel HC Ward ranks last.

-Global commonalities

To get an idea of inter-corpus comparisons, we constructed a relative performance indicator by relating the NMI of a given combination "data space + method" for a certain corpus to the maximum NMI observed on this corpus. Having 3 values per combination, we can situate -for heuristic rather than rigorous -the overall performance of a combination by calculating the average and the maximum difference between these 3 values. The following lines summarize this process for the three combinations which seemed to us to achieve the best compromise between performance and independence from the corpus (i.e. low deviation):

1) Standard NMF on Okapi weighted data, with non-truncated vocabulary:

relative NMI: 95.4%, difference: 7.9%

2) Spectral hierarchical clustering (Ward link) in the space of the 2k * first AFC factors, with a strong truncation of the vocabulary (12.5% of the original vocabulary):

Relative NMI: 92.0%, difference: 9.1%

3) Standard K-Means on Okapi weighted data, and vocabulary also truncated at 12.5%: relative NMI: 91.1%, difference: 18.2%

The main problem for one to follow recommendation 2 is to build the spectral space for big real-life data. In many computer languages indeed, efficient sparse Singular Value Decomposition procedures exist, appropriate when the problem is to draw a limited number of main eigenvalues and eigenvectors from huge datatables, which is the case in the present study. Otherwise parallel graphics co-processors may be dedicated to this task.

-Conclusions and perspectives

We hope we have brought some clarification to the problem of evaluating text clustering procedures, by considering separately the algorithms and the dataspaces in which they operate. We have achieved some 600 runs of a dozen algorithms and variants, in a few tens various dataspaces, on three prototypical and public access test corpora. We have brought to light an unexpected variety of optimal combinations of methods and dataspaces, from which we have derived three cautious recommendations. The variety of possible transformations and parameters requires a considerable continuation effort for improving our understanding and mastery of artificial vs. human categorization processes. We hope that this empirical survey will contribute to such an issue. In a modest first step, we will explore the influence of linguistic pre-processing: choice or elimination of word categories, comparison between taking into account multi-word expressions and kernel expansion of uniterms.

FigureFigure 2

 2 Figure 1: K-means on ACM corpus

Figure

  Figure 4: NMF on ACM corpus

Figure 7 :

 7 Figure 7: LDA on ACM corpus

Figure 9 :

 9 Figure 9: Hierarchical Clustering on ACM corpus

Figure 12 :

 12 Figure 12: Spectral K-means Clustering on ACM corpus

Figure 13 :

 13 Figure 13: Spectral Non-negative Matrix Factorization clustering on (un-)weighted ACM corpus NMF is good, but not as competitive as K-Means in the same spectral context.

Figure 14 :

 14 Figure 14: Spectral Hierarchical-Ward Clustering on ACM corpus in Correspondence Analysis factor space (cosine distance)

Figure 16 :

 16 Figure 16: Spectral K-means Clustering on Reuters'8 corpus

Figure 17 :

 17 Figure 17: Spectral NMF Clustering on Reuters'8 corpus

Figure 18 :

 18 Figure 18: Spectral Hierarchical Clustering on Re8 corpus in Correspondence Analysis factor space (cosine distance)HC-average is clearly a bad option in raw CA space, but a good one using cosine distance in this dataspace.

Figure 19 :

 19 Figure 19: Spectral Hierarchical-Ward Clustering on Re8 corpus in Okapi/Salton-weighted Laplacian factor space (quantile-truncated -or not-cosine distances) Ng20 corpus

Figure 21 :

 21 Figure 21: Spectral NMF Clustering on Ng20 corpus

Figure 22 :

 22 Figure 22: Spectral Hierarchical Clustering on Ng20 corpus in various factor spaces (cosine distances)

Figure 23 :

 23 Figure 23: Kernel Clustering on ACM corpus

Figure 25 :

 25 Figure 25: Louvain graph clustering on ACM corpus

Figure 27 :

 27 Figure 27: InfoMap graph clustering on ACM corpus

Figure 29 -

 29 Figure 29 -Cross tabulation of clusters vs. classes on best-NMI run of 20 NewsGroups corpus. At left hand: classes; at right: clusters

Alelu@orange.fr

Martine.Cadot@loria.fr -corresponding author


Omer Rana

Keywords:

The research group is structured around five research themes related to the Internet of Things (IoT) with a significant emphasis on build-driven research method: (1) Design-Time Privacy and Security, (2) Run-Time Privacy and Security, (3) Data Marketplaces, (4) Sensing Infrastructure, (5) Learning Technology.

• Six PhD students have started working on these themes, including two affiliated PhD students.

Run Time (Privacy and Security):

This theme focuses on runtime privacy and security aspects. We develop techniques to detect in-vehicle cyber attacks by examining CAN data. We also study how to optimally place and schedule of service function chaining under security constraints. We aim to understand vulnerabilities in IoT devices better when integrated into future digital ecosystems, particularly safety-critical systems.

Data Marketplaces: This theme focuses on understanding future data marketplace better, from different perspectives such as data management, end-to-end infrastructure, and business model. In this work, we look at smart city data, smart home data, and personal data as well. As a consequence, we are also interested in looking at privacy challenges (ranging from privacy-preserving data trading to informed consent) around IoT data marketplaces.

Sensing Infrastructure: This theme explores how to develop and deploy IoT infrastructure to support various applications and in different domains such as agriculture, building sustainability, human behaviour, independent living, smart homes and offices occupancy patterns and usage, rural and wildlife, etc. We aim to identify different trade-offs and their applicability in different domains and finally to extract best practices.

Learning Technologies: This theme aims to enhance teaching activities. We aim to understand how to teach IoT for different types of audiences (from high school to university students and beyond) with different skills levels with the help of innovative tools and techniques.

• Three MSc students completed their dissertations.

• Developed an IoT Edge network with 100 sensor nodes with the funding received from EPSRC ECR Capital award. o A summer student supported by Cardiff Undergraduate Research Opportunities Programme (CUROP) received research training as part of this project.

• A postdoctoral researcher started working on a PETRAS Catalyst project (Integrity Checking at the Edge).

• Developed Internet of Things teaching materials to be delivered in 19/20 spring semester via Network Communications and Emerging Technology modules.

• Started the RCUK Catapult Researchers in Residence (RiR) project (Quarriable Smart City Data Markets) in collaboration with Digital Catapult, which will span across the next two years.

• Organised a Global Challenges Research Fund (GCRF) workshop with the help of a facilitation funding in order to develop a long-term research program in collaboration with the School of Bioscience and Danau Girang Field Centre (DGFC).

Looking forward to 2020…

Teaching Vision

At the undergraduate level, Internet of Things related content is delivered (to second-year students) through a module titled CM2306 Communication Networks (since 19/20). At postgraduate level, IoT is delivered through a dedicated module titled Internet of Things: Systems Design (planning for 20/21). Both modules are (mostly) identical in terms of delivery and content. However, expectations (from assessment perspective) are higher at postgraduate level.

Content:

The IoT content is structured under eight theme, namely, (1) Applications and Use cases, (2) Architectures, (3) Sensing and Actuation, (4) Networking and Communications, (5) Data management and analytics, (6) Privacy and Security, (7) Human Factors and Interactions, and (8) Design Strategies and Prototyping. Each of these sections gets delivered through one or more lectures (which includes dedicated slide decks).

Modularity and Complexity:

The content under each theme is developed in a modular and layered fashion based on the complexity of the content. This means that each topic has a certain amount of content that delivers the basic information to the students, which are sufficient enough to complete both undergraduate and postgraduate modules. However, if a student interested in learning more they can follow the advance material and learn by themselves. Advanced materials are structured and delivered in a similar fashion to the basic material (at times embedded within basic material but are clearly marked) and also provide close guidance on how to follow up and self-study the material. Specialist materials are less structured and less organised. they are delivered through either seminars or tutorials (pre-recorded or in-class). Advance and specialised material may help the students to complete the assignments in a much higher quality but by no means mandatory to follow. Specialised material may be useful for new research students to advance their knowledge.

Labs and Practical: As a result of being an applied module, students are expected to complete at least six lab sessions. Students are provided with the lab book that explains each practical session steps by step.

Research with BSc and MSc students: Most of the dissertation projects we offer are research-oriented. These projects are usually aligned with existing projects we are working on, at a given point of time, through either PhD students or research associates. However, we use these dissertation projects to initiate some high-risk projects or new research directions as well. All of our students are encouraged (and supported) to produce research output (such as conference, workshop paper, poster).

Dissemination and Community Engagement

IOT Garage TV (bit.ly/2Md8vJE) YouTube (and similar platforms) has increasingly become a mainstream content distribution stream that provides access to a large audience. As a build-driven research group, demonstrations are a key part of our strategy towards dissemination and increase awareness. Therefore, we have created a dedicated YouTube channel to disseminate our work. We believe visual medium can efficiently and effectively motivate our students to complete their project work with high quality. YouTube videos on our channel also act as a gauge for prospective students. For example, video help students to decide what kind of project they would want to do and the quality of the output they may want to produce. We also use the YouTube channel as a part of our reproducibility and knowledge transfer strategy. We strongly encourage students to create screencasts in such a way that another student could understand what has been done and how. This allows next year students to take the projects forward. Screencasts also help students to provide valuable insights about their projects to their fellow students which might not be feasible to do in traditional documentation approaches.

IOT Garage News (@IOTGarageNews)

As a complementary to the YouTube channel, Twitter has increasingly become one of the primary ways people consume news updates. We maintain a Twitter account to broadcast updates about our group activities. This includes research updates, students successes, public engagement, and so on.

IOT Garage Code (@IOTGarage)

We take reproducibility and 'building on top of previous work' very seriously.

As a supplement to the screencast, we also encourage to organise and share their code through Gitlab (or similar). We actively maintain code repositories produce by each student related to each project.

Group Website (iotgarage.net)

We maintain a group website to disseminate outcomes of different type of projects to the wider audience. Projects can be varied from BSC, MSc, PhD, to funded projects. Under each project, we provide all the relevant information, including, team members, funder, partners, project demos, linked to publications, links to code repositories.

Funding Support

(Co-Investigator)

EP/S035362/1 Total: 13,850,000 GBP Cardiff: 290,920 GBP

PETRAS 2 (National Centre of Excellence for IoT Systems Cybersecurity)

The PETRAS National Centre of Excellence for IoT Systems Cybersecurity is a consortium that connects 12 research institutions with outstanding expertise in securing the connected world. This Research program has funded Integrity Checking at the Edge project. ICE project studies the factories and water treatment systems of the future, undertaking composite vulnerability analysis of interactions between edge devices, cloud and legacy systems.

(Principle-Investigator) EP/T517203/1 Total: 50,000 GBP Cardiff: 50,000 GBP

RCUK Catapult Researchers in Residence award (Digital) -Quarriable Smart City Data Markets

The funding was given to initiate a project affiliated to H2020 funded SynchroniCity project. SynchroniCity project aims to create a data marketplace that facilitates businesses to develop IoT-and AI-enabled services to improve the lives of citizens and to grow local economies. However, at the moment, the data offers are searched, modelled, and sold syntactically. This project aims to enrich data with semantic capabilities using ontologies and reasoning techniques by allowing data consumers to query data semantically. Such semantic technology-driven data marketplaces allow data consumer to acquire very specific data instead of asking large volumes of less relevant data.

GCRF Facilitation Funding (Cardiff University Internal)

Funding provided for University Research Institutes, established Networks and Centres at Cardiff University to bring together potential GCRF partners and stakeholders through the organisation of GCRF focussed workshops and networking events, in the DAC listed Least Developed and Other Low-Income Countries. We received funding to conduct two workshops in Sabah, Malaysia and Cardiff, UK in order to explore challenges faced by Sabah Wildlife.

Partners Awen Collective

Awen Collective develops software for critical infrastructure (water, energy, transport, etc.) and manufacturers reduce the costs of cyber-attacks and cyber-threat.

Airbus Group

Airbus Group Innovations are industry leaders in industrial control system (SCADA) security and have a well-equipped testbed at their Newport site.

Digital Catapult

Digital Catapult drives early adoption of artificial intelligence, immersive and future networks technologies to make UK businesses more competitive and productive, and to grow the country's economy.

The Things Network

The Things Network provide a set of open tools and a global, open network to build your next IoT application at low cost, featuring maximum security and ready to scale.

PETRAS National Centre for Cyber Security

The PETRAS National Centre of Excellence for IoT Systems Cybersecurity is a consortium that connects twelve research institutions with outstanding expertise in securing the connected world.

Vortex IoT

Vortex IoT builds sensors and networks for harsh environments where conditions are hostile, and power supply is limited, AI is needed & data security is critical. iPoint iPoint aims to simplify fleet and data management across the transport industry by unlocking and correlating information from multiple platforms and networks by developing a single transport management platform.

CloudTexo

Cloudtexo is the leading Distribution Service Provider for IIoT, Mobility & Cloud. We deliver affordable disruptive technologies consumed by enterprises.

Danu Gurang Field Center

Danau Girang is a collaborative research and training facility managed by Sabah Wildlife Department and Cardiff University.

Interactive Design Method for Augmenting Software Design Process Toward Privacy-Aware Internet of Things Application Designs

Researcher: Nada Alhirabi (PhD Student-2018-2022) Internet of Things (IoT) applications development and design process is more complicated than others, such as the one for desktop, web, or mobile. That's because IoT applications need both software and hardware to cooperate across multiple nodes with different capabilities. Moreover, it requires different software engineers with different expertise to cooperate (e.g., frontend, backend, database). Due to the above complications, non-functional requirements, such as security and privacy, tend to be overlooked.

Yearly, a significant number of devices and applications are connecting to the internet, which raises potential privacy risks. Typically, IoT applications collect and analyse personal data that can be used to derive sensitive information about individuals. However, thus far, privacy concerns have not been explicitly considered (i.e., as united way), despite isolated solutions (i.e., specific out technique that address specific problem), in software engineering processes when designing and developing IoT applications, partly due to a lack of Privacy-by-Design (PbD) methods for the IoT.

The primary objective of this project is to develop an interactive design method (facilitate through a tool) that incorporate privacy-preserving techniques into the early phases of the software development lifecycle efficiently, effectively and collaboratively. We envision our tool to be collaboratively used by business analysists, requirement engineers, user experience designers, and software engineers together during the process of creating privacy by design IoT application designs. Our secondary objective is to explore whether such a tool (with minor alteration) could also be used to enhance privacy education of high school and university students. We aim to develop our prototype by extending the OWASPS's Threat Dragon Tool. This project composed of three main objectives:

• Review the existing of design notations, models, languages and tools that facilitate capturing non-functional requirements (i.e., security and privacy). • Co-Creating visual language, notations, and set of interaction patterns toolkit for Privacy by Design Representation for the IoT.

• Evaluate the efficiency and effectiveness of PARROT (Privacy by

Design for the Internet of Things) as a tool for augmenting software engineer's capabilities and enhance privacy education.

Partners and Relevant Projects

Outcomes Yearly, a significant number of devices and applications are connecting to the internet, which raises potential privacy risks. Typically, IoT applications collect and analyse personal data that can be used to derive sensitive information about individuals. However, thus far, privacy concerns have not been explicitly considered (i.e., as united way), despite isolated solutions (i.e., specific privacy-preserving technique that address specific problem), in software engineering processes when designing and developing IoT applications, partly due to a lack of Privacy-by-Design (PbD) methods for the IoT.

The primary objective of this project is to develop a method (using Knowledge-based AI) that assist software engineers by automatically incorporating Privacy by Design (PbD) techniques into a given IoT application design. We aim to develop our prototype by extending the OWASPS's Threat Dragon Tool. This project composed of three main objectives:

• Review and synthesise privacy by design knowledge through curating and systematically analysing existing privacy strategies, guidelines, principles, and patterns in the context of IoT.

• Semantically model privacy patterns and IoT systems using knowledge base AI techniques towards automated privacy pattern placement.

• Evaluate the efficiency and effectiveness of PRIVACY CAPTAIN (Context-Aware Privacy Assistant for the Internet of Things) as a tool for augmenting software engineer's capabilities and enhance privacy education. PRIVACY CAPTAIN is an AI assistant that is capable of reviewing a given IoT system design and provide advice on how optimally apply privacy patterns.

Partners and Relevant Projects

Outcomes In order to address this issue, we need to find a way to support and motivate software developers. In this project, we primarily focus on privacy. We aim to address this problem using two methods. First, we need to develop easy to use privacy-preserving software components (some form of modules) that developers can incorporate into their IoT application development process. These privacy-preserving components should be reusable and generic enough to be used across multiple domains and applications. Furthermore, these privacy-preserving techniques should be integrated into existing IoT software development tools (i.e., popular IDEs and software frameworks). Secondly, we will use gamification techniques to motivate the software developers to incorporate more and more reusable privacy-preserving components within their IoT applications. This gamification framework will also be integrated into popular IoT software development tools. This project composed of three main objectives:

• Systematically analyse privacy by design schemes to find out how they can be used to satisfy and comply with privacy laws around the world in the context of IoT.

• Explore how different types of privacy by design schemes and elements within them (such as privacy strategies, principles, guidelines, and patterns) can be transformed into reusable privacy-preserving components.

• Based on the above findings, we aim to develop a series of reusable privacy-preserving components that can be easily adapted into IoT application development process.

• Develop a framework to examine, classify and operationalise each of these privacy-preserving components in order to quantify them towards developing a gamification method.

Partners and Relevant Projects

Outcomes In this project, we investigate Controller Area Networks (CAN) bus and its security vulnerabilities and countermeasures in the context of different vehicle networks (e.g., (i) In-vehicle network, (ii) vehicleto-vehicle and (iii) vehicle-to-infrastructure. Today, vehicles are fitted with a large number of sensors and actuators. These sensors and actuators produce and consume large volumes of data. These data items are managed through a system called CAN bus. At the same time, more and more vehicles are getting connected to the Internet. Furthermore, vehicles also provide a wide range of interfaces that can be used to connect external devices to them, such as mobile phones. Both Internet connectivity and external interfaces create more vulnerabilities. These external connections could be manipulated to conduct cyber-attacks against vehicles. In this project, we are aiming to develop algorithms that can be used to detect malicious activities and cyberattacks by analysing the CAN Bus data. Further, we will explore how the CAN Controller can be improved to be more secure from cyberattacks from inside out and vice versa.

•

Partners and Relevant Projects

Run Time ICE project will study the factories and water treatment systems of the future, undertaking composite vulnerability analysis of interactions between edge devices, cloud platforms and legacy systems. This vulnerability analysis will be complemented by the use of AI at the edge of such systems to create and improve methods to demonstrate transparent processes mapping of data flows to expected activity at the periphery of integrated systems with advanced visualisation methods, and provide security and resilience assurances for critical infrastructures of the future. Inspiration will be taken from explainable and interpretable AI, considering human-machine interaction in the context of questionasking around pathways and interactions involving data, creating links to the ICE-AI project within the usability lens. The Bristol Critical Infrastructures Testbed (which underpinned a linked PETRAS 1 demonstrator) will form a core source for the technical analysis. The project aims to understand vulnerabilities in IoT devices better when integrated into future digital ecosystems -particularly safety-critical systems. There are three main objectives to achieve this aim:

• Using AI to map actual activity to expected outcomes for data flows and behaviour in safetycritical IoT-based systems. • To improve the visualisation of actual and expected outcomes such that anomalies and deviations in behaviour can be human-interpretable. • To enable human-machine interaction with improved visualisations to 'dig in' to anomalous or suspicious behaviour and undertake exploratory analysis to understand malicious behaviour further.

Partners and Relevant Projects Interaction Methods for Privacy Preferences Management in Shared Spaces in Internet of Things Era

Researcher: Bayan Almuhander (PhD Student-2019-2023)

Smart spaces (i.e., smart homes and offices) could comprise many different IoT devices. Most of these devices comprise of different types of sensors that measure a wide range of phenomenon (e.g., temperature, presences, activities, etc.). At the moment, these devices do not have a mechanism to communicate with the user to inform them about how they manage data. Sometimes, these devices may provide very limited information about how these devices manage data during the initial configuration (setting up) process. However, no IoT product today informs the users (i.e., owner as well as other occupants who share the same space) about the data it collects and manages on an ongoing basis. In this project, we aim to develop an interaction method that allows users to manage their privacy preferences on an ongoing basis in shared spaces in Internet of Things era. Cities are increasingly get augmented with sensors through both public, private, academic sector initiatives. Most of the time, these sensors are deployed by having a primary reason in mind (e.g., noise sensors) by a sensor owner (e.g., city council). However, over the last few years, the community has understood the importance of making the data, captured by these sensors, available for a wider community beyond their primary usage. Different business models have been proposed to achieve this, including the creation of data marketplaces(e.g., iot-data-marketplace.com). The vision is to encourage new start-up and small businesses to create novel products and services by utilising the datasets to generate additional value to the economy. At the moment, in data marketplaces, data are sold as pre-defined independent datasets (e.g., noise level and car park status datasets may be sold separately). This approach creates a number of challenges such as;

Run Time

(i) Difficulties in pricing which leads to higher prices (per dataset), (ii)

Higher network communication and bandwidth requirement and (iii)

Information overload for data consumers (i.e., parties who buy data). In this project, we propose a semantic-driven technique to create on-demand data offering for data marketplaces. Our approach goes beyond predefined data offering to on-demand custom data offering. More importantly, we solve the three challenges mentioned above. The project composed of serval objectives:

• Develop an ontology by combining a few different well-known ontologies that have the capability of modelling any type of sensor data in the context of IoT data marketplaces.

Partners and Relevant Projects

Data Marketplaces

Current view of the Data Marketplace: iot-data-marketplace.com

Sustainable Internet of Things Infrastructure Towards Efficient Wildlife Conservation

Researchers: Charith Perera, Omer Rana, Benoît Goossens, Pablo Orozco-ter Wengel ( 2019)

Danau Girang Field Centre (DGFC) is located in Sabah Malaysia (marked in Figure 2). The only access to the field centre is through the river. As a result, all the research activities are conducted by going through the river. It is worth noting that Sabah Malaysia is a high humidity region where electronic components could get damaged quite quickly due to environmental factors such as moisture. Further, in jungle terrains, insects could also get attracted to copper within electronic components. We need to keep in mind this context when we are addressing the challenges. We conducted a two full-day workshop to explore and identify research challenges that could potentially be addressed using the Internet of Things (IoT) technologies. During this workshop, we identified two major areas to focus on:

(1) Sensing Infrastructure, and (2) Data Science. Additionally, we also discussed citizen engagement research, where we could work with local schools and universities to share our technical expertise with the local community. In the long term, such activities will help local communities to develop technologies to solve their problems.

Low-Cost SMS Driven Location Tracking System for Anti-Poaching Investigations

Researcher: Jack Burkett (BSc Student)

Throughout the world, poaching has been an ever-present threat to a vast array of species for over many decades. Traditional antipoaching initiatives target on catching the poachers. However, the challenge is far more complicated than catching individual poachers. Poaching is an industry which needs to be fully investigated. There are many stakeholders, directly and indirectly, involved in poaching activities (e.g., some local restaurants illegally providing meat to tourists). Therefore, to stop or severely decapitate the poaching industry requires a unified understanding of all stakeholders.

The best ways to uncover these geographical and social relationships is to track the movements of poachers. However, location tracking is challenging in most of the rural areas where wildlife sanctuaries are typically located in. Internet-connected communication (e.g. 3G) technologies typically used in urban cities are not feasible in these rural areas. Therefore, we decided to develop an SMS (short message service) base low-cost tracking system (SMS-TRACCAR) to track poachers. We evaluated the proposed system in Kinabatangan Wildlife Sanctuary, Sabah, Malaysia and nearby villages and cities where poachers are typically moving around.

Our evaluations demonstrated that SMS based tracking could provide sufficient quality (granular) data (with minimum energy consumption) that enable us to monitor poacher vehicle movements within rural areas where no other modern communication technologies are feasible to use. However, it important to note that our system can be used in any domain that requires SMS based geo-location tracking. SMS-TRACCAR can be configured to track individuals as well as groups. Therefore, SMS-TRACCAR makes contributions not only to wildlife domain but in a wider context as well.

Exploring the Suitability of BLE Beacons to Track Poacher Vehicles in Harsh Jungle Terrains

Researcher: Karan Juj (BSc Student-2019)

Our overall aim is to explore whether we could use Bluetooth Low Energy (BLE) technology to track poacher vehicles in remote and rural areas such as Sabah, in Malaysia, especially deep inside the jungle terrain with little or no communication technologies exists. Tracking technologies are currently limited to relying on satellites or cellular towers, for environments that do not permit access to these signals, very few viable alternatives exist. This project explores the use of BLE as a method to track vehicles. It works by mounting Bluetooth beacons beside a road and placing a receiver concealed somewhere inside the vehicle. As the vehicle drives past the beacon, the receiver and beacon are momentarily in range, the receiver then stores a unique ID from the beacon, and when the vehicle is then in an area with GSM signal, an SMS is sent containing the unique IDs of the beacons that have been detected. This project is prototyped and tested in collaboration with the Danau Girang Field Centre in Sabah, Malaysia. The results offer insights for how effective BLE beacons are in a tracking situation for where the beacon and receiver are in range for a short period as well as how different obstructions will affect the range and strength of the signal. It is important to note that our objective is not to catch the poacher, instead to understand how they move around within jungle terrain, as we can use such information to develop a comprehensive plan against poaching activities. We placed the BLE beacons, as shown in the figure.

During the evaluation, we tested various types of obstructions that the signal would face in deployment. As illustrated in the following Figure, we tested plastic and cardboard cases which were intended to simulate the effect that adding a case would have on the signal.

We found that different types of obstructions reduce the distance the BLE would penetrate. However, signals could easily research 50m under any obstructions. Usually, this distance is sufficient enough to be useful in jungle roads as the only requirement is to reach an ongoing vehicle. If obstructed by water (e.g. raining), BLE signal penetration could be reduced to 30m, which is still sufficient to be useful.

• Computing devices such as laptops, tablets and mobile phones have become part of our daily lives. End-users increasingly know more and more information about these devices. Further, more technically savvy end-users know how such devices are being built and know how to choose one over the others. However, we cannot say the same about the Internet of Things (IoT) products. Due to its infancy nature of the marketplace, end-users have a very limited idea about IoT products. To address this issue, we developed a method, a crowdsourced peer learning activity, supported by an online platform (OLYMPUS) to enable a group of learners to learn IoT products space better. We conducted two different user studies to validate our tool enables better IoT education. We structured the learning activities into three stages, as illustrated below. Our method guide learners to think more deeply about IoT products and their design decisions. The learning platform we developed is open source and available for the community. In this project, we explored the potential impact of the Internet of Things (IoT) technology may have on the cosplay community. We developed a costume (an IoT Skullfort) and embedded IoT technology to enhance its capabilities and user interactions. Sensing technologies are widely used in many different wearable domains, including cosplay scenarios. However, in most of these scenarios, typical interaction pattern is that the costume responds to its environment or the player's behaviour (e.g., colour of lights may get changed when the player moves hands). In contrast, our research focused on exploring scenarios where the audience (third party) get to manipulate the costume behaviour (e.g., the audience get to change the colour of the Skullfort using a mobile application). We believe such an audience influenced cosplay brings new opportunities for enhanced entertainment. However, it also creates significant challenges. By using the IoT Skullfort prototype as a concrete example, we conducted a focus group and extracted a few interesting views, including the following.

Health and Safety: It was highlighted, that if tailored, this technology could be utilised to mitigate specific disabilities. For example, an individual who has partial deafness could utilise a series of microphones scattered across a costume. Inappropriate, abusive comments, and hate speech: If the LEDs were utilised as a matrix that allowed scrolling text, then allowing open access to the public could cause trouble. Experts agreed that the individuals wearing the device is just as liable for the content displayed, as those that wrote it. Meaning, they would be punished for any problematic content; as they facilitated the public distribution of the message.

BSc Family Dynamics-based Access Control for the Internet of Things

Researcher: Yasar Majib (MSc Student-2019) Today, children are increasingly connected to the Internet and consume content and services through various means. It has been a challenge for less tech-savvy parents to protect children from harmful content and services. Internet of Things (IoT) has made the situation much worse as IoT devices allow children to connect to the Internet in novel ways (e.g., connected refrigerators, TVs, and so on). In this we propose to utilise family dynamics to provide a more natural, and intuitive access control mechanism to protect children from harmful content and services. In our proposed approach, access control dynamically adapts based on the physical distance between family members. For example, a particular type of content can only be consumed, through TV, by children if the parents are in the same room (or hearing distance). This approach allows parents to assess a given content by themselves. This is a contrasting view to AI-based decision making where AI decide on behalf of the parents. We believe that parents are the best person to make the decision, given that they get the right opportunity. Our access control mechanism aims to guarantee that opportunity (i.e., block certain content, services, and devices when the parents are not in the vicinity). We developed a prototype using OpenHAB and several smart home devices to demonstrate the proposed approach. We also conducted a focus group to identify how family dynamics can be further used to support better security and safety for children. We believe that our approach also facilitates the creation of better relationships between family members. Over the last few years, many different Internet of Things devices has been made their way into the market. As a result, many smart home solutions have been introduced by different companies. However, at this point, most of the smart home solutions are engineering products built with limited attention to the end-user need. Many researchers are working towards developing more meaningful smart home solutions and looking at the problem from different perspectives such as human-computer interactions, edge computing, psychology. Another aspect is to explore how these smart home solutions could be useful for independent living, disable and vulnerable people. Further, these smart homes systems need to be privacy-aware and secure. These research activities, at some point, need some kind of human activity simulation tool.

Recognising this problem, this project developed a tool capable of simulating human activities in a smart home.

A set of pre-created scenarios can be made through the simulator tool interface, which will then be published into OpenHAB interface. The OpenHAB platform is used to enable the researcher to display IoT devices of the smart home in its user interface and at the end, obtaining a synthetic dataset ready to research. This was done by setting up the tool with the GUI interface and writing python code to publish the data using the MQTT protocol.

A tool interface has been designed to help simulate various human events, enabling the creation of data to be played out so that research can be conducted. Furthermore, it is allowing the creation of multiple files. This will help to set-up devices on OpenHAB the same as with real devices. The goal is to create a simulation tool that allows researchers to create realistic synthetic smart home data sequences. Such data sequences could be used for research and evaluation purposes. Furthermore, it offers a list of smart sensors and devices that can be expanded to include future emerging technologies.

•

•

  Five BSc students completed their final projects -including two publications o Rhys Beckett (2019) published a poster paper in Ubicomp 2019 o Ahmed Hussein (2019) published a research paper in IEEE Internet of Things Magazine

  This project is focused on establishing a laboratory at Cardiff University to support 'Edge Computing' research. It enables us to develop a new class of applications (and underlying computational analysis algorithms) that are 'latency-sensitive'. The laboratory will enable us to carry out internationally leading research in cybersecurity, real-time data processing and communication networks. The laboratory will be realised by a combination of Raspberry Pi and Arduino boards, that are distributed across open spaces and existing laboratories.

  [Technical Report] Atheer Jeraisy, Omer Rana, Charith Perera, A Systematic Analysis of Privacy by Design Schemes towards Complying with Privacy Laws in the Internet of Things Era, Technical Report, 2020 (will be available soon)

Figure 2 :

 2 Figure 2: A map the region of the Kinabatangan Wildlife Sanctuary

Figure 3 :

 3 Figure 3: (a) GPS Tracker Locator Device [left] and (b) the message return if device is unable to connect to GPS [right]

  Figure 4: Different types of obstructions: (Left) plastic, cardboard, wood, water (right)

  Stages in the proposed crowdsourced peer learning activity for IoT These series of screens demonstrate how a to crowdsource worker add a feature of water-resistant to given IOT device and provide video-based evidence • [Journal] Ahmed Hussein, Mahmoud Barhamgi, Massimo Vecchio, Charith Perera, Crowdsourced Peer Learning Activity for Internet of Things Education: A Case Study, IEEE Internet of Things Magazine (IOTM), Volume x, Issue x, 2020, Pages xx-xx (xx)

•

  [Poster] Rhys Beckett and Charith Perera, IoT Skullfort: Exploring The Impact of Internet-Connected Cosplay, In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers (UbiComp/ISWC '19 Adjunct), ACM, New York, USA, 5-8.

•

  [Conference] Yasar Majib, Charith Perera, Context-Aware Family Dynamics based Internet of Things Access Control Towards Better Child Safety, IEEE 6th World Forum on Internet of Things (WF-IoT), New Oreleans, Louisiana, USA, April, 2020MSc Smart Home Human Activity Simulation Tool for ResearchResearcher: Asma Alotaibi(MSc Student-2019)

  [Technical Report] Asma Alotaibi, Charith Perera, Smart Home Human Activity Simulation Tool for OpenHAB-based Research, Technical Report, 2019 MSc

Development Time (Privacy and Security) Augmenting Software Design Processes using Automated Privacy- aware Internet of Things Application Design Techniques

  Internet of Things (IoT) applications development and design process is more complicated than others, such as the one for desktop, web, or mobile. That's because IoT applications need both software and hardware to cooperate across multiple nodes with different capabilities. Moreover, it requires different software engineers with different expertise to cooperate (e.g., frontend, backend, database). Due to the above complications, non-functional requirements, such as security and privacy, tend to be overlooked.

	Researcher: Lamya Alkhariji (PhD Student-2018-2022)

• [Technical Report] Nada Alhirabi, Omer Rana, Charith Perera, Designing Security and Privacy Requirements in the Internet of Things: A Survey, Technical Report, 2019 Design &

Development Time (Privacy and Security) Motivating Software Engineers to Develop Privacy-aware Internet of Things (IoT) Applications through Reusable Privacy Components and Gamification Techniques

  Internet of Things (IoT) applications development and design process is more complicated than others, such as the one for desktop, web, or mobile. That's because IoT applications need both software and hardware to cooperate across multiple nodes with different capabilities. Moreover, it requires different software engineers with different expertise to cooperate (e.g., frontend, backend, database). Due to the above complications, non-functional requirements, such as security and privacy, tend to be overlooked.

	Researcher: Atheer Jeraisy (PhD Student-2019-2023)

• [Technical Report] Lamya Alkhariji, Omer Rana, Charith Perera, Synthesising Privacy by Design Knowledge: A Systematic Analysis, Technical Report, 2020 (will be available soon)

Design &

(Privacy and Security) Run Time (Privacy and Security) Integrity Checking at the Edge

	Researcher: Matthew Nunes (Research Associate-2019-2021)

(Privacy and Security) Run Time (Privacy and Security) Semantic Knowledge-Driven On-demand Data Offerings for Quarriable Smart City Data Marketplaces

	Researcher: Charith Perera (Principle Investigator-2019-2021)

Research Vision

Research Interests: My research primarily focuses on three research questions:

1. How can we build an efficient and effective sensing infrastructure to acquire and use sensor data in order to understand and improve ourselves (individuals), surroundings (homes), communities, and the world better? 2. How can we encourage sensor data sharing in order to achieve (1)? 3. How can we achieve (1) and (2) without compromising safety, privacy or security? My research group is formulated around research themes as follows:

Design and Development Time (Privacy and Security):

In this theme, we explore how to develop efficient and effective tools to help software engineers to design privacy and securityaware application better. We also extend development time tools to support and motivate software developers too better incorporate privacy and security by design approaches.

Optimal Placement and Scheduling of Service Function Chaining (SFCs) under Security Constraints

Researcher: Areej Alabbas (PhD Student-2019-2022) Virtualized Network Functions (VNFs) is a technique used to replace hardware-based functions by a set of software-based functions. These functions are dynamically deployed across multiple clouds based on the cost of deployment, availability of required resources and proximate to the end-user. The process of chaining these VNFs to form end to end service is called Service Function Chaining (SFC). SFC is a set of service functions (SFs) that are ordered to provide a specific service. This study is aimed to propose an algorithm which is used to optimal placement of SFCs in multi-cloud architecture based on security constraints. In addition, this study will build analytics model with taking into account two factors: security constraints and performance.

PizzaBox: Studying Internet Connected Physical Object Manipulation based Food Ordering

Researcher: Luke Jones (BSc Student-2019) This project focused on designing and testing of PizzaBox, a 3D printed, interactive food ordering system that aims to differ from conventional food ordering systems and provide an entertaining and unique experience when ordering a pizza by incorporating underlying technologies that support ubiquitous computing. The PizzaBox has gone through both low and medium fidelity testing while working collaboratively with participants to co-design and refine a product that is approachable to all age groups while maintaining a simple process for ordering food from start to finish. The final evaluation was conducted at an independent pizzeria where interviews with participants lead us to develop three discussion themes: 1) end-user engagement (from entertainment to education), 2) towards connected real-time products and services, and 3) healthy eating and living.

We believe that each of the themes identified during the user study has there own merit to be investigated more in-depth independently. For example, individual IoT device base advertising is a completely new paradigm that needs to be investigated separately to examine opportunities and potential issues. Such investigation may be carried out from a different point of views such as business, psychology, and law. Other opportunities are to use PizzaBox (or similar IoT devices), as a tool for education and lifestyle changes. Such investigation may be carried out from a different point of views such as education, healthy eating, and medical (e.g., preventing risks related to allergies). PizzaBox type device could also help people who have challenges in using computational devices such as computers or mobile phones (e.g., dementia, Parkinson). Education and healthy eating would be our research focus in the future.