Michael O'byrne

Franck Schoefs

Bidisha Ghosh

Vikram Pakrashi

Texture Analysis Based Damage Detection of Ageing Infrastructural Elements

To make visual data a part of quantitative assessment for infrastructure maintenance management, it is important to develop computer-aided methods that demonstrate efficient performance in the presence of variability in damage forms, lighting conditions, viewing angles, and image resolutions taking into account the luminous and chromatic complexities of visual data. This article presents a semi-automatic, enhanced texture segmentation approach to detect and classify surface damage on infrastructure elements and successfully applies them to a range of images of surface damage. The approach involves statistical analysis of spatially neighboring pixels in various color spaces by defining a feature vector that includes measures related to pixel intensity values over a specified color range and statistics derived from the Grey Level Co-occurrence Matrix calculated on a quantized grey-level scale. Parameter optimized nonlinear Support Vector Machines are used to classify the feature vector. A Custom-Weighted Iterative model and a 4-Dimensional Input Space model are introduced. Receiver Operating Characteristics are employed to assess and enhance the detection efficiency under various damage conditions.

INTRODUCTION

The deteriorating condition of infrastructure worldwide and the excessive costs required for reparative work necessitate the invention of efficient and effective detection techniques. Recent estimates suggest that $1.6 trillion dollars will be needed for rehabilitation, replacement, and maintenance of current infrastructure systems with the next 20 years for the United States alone [START_REF] Adeli | Intelligent Infrastructure-Neural Networks, Wavelets, and Chaos Theory for Intelligent Transportation Systems and Smart Structures[END_REF]. With this in mind, it is vital that a comprehensive strategy for the periodic inspection and monitoring of structures [START_REF] Schoefs | Polynomial chaos representation for identification of mechanical characteristics of instrumented structures[END_REF][START_REF] Lajnef | Toward a smart, long-term pavement monitoring system[END_REF][START_REF] Gangone | Wireless monitoring of a multispan bridge superstructure for diagnostic load testing and system identification[END_REF] is developed beginning from the construction phase. This is even more relevant given the increased loads and ever challenging environmental conditions that structures are faced with [START_REF] Cusson | Durability monitoring for improved service life predictions of concrete bridge decks in corrosive environments[END_REF][START_REF] Xia | Stress development of a supertall structure during construction: field monitoring and numerical analysis[END_REF]. Non-Destructive Testing (NDT) techniques are frequently used for the inspection process as they often offer the only practical means for detecting the presence of damage and quantifying its severity. The information obtained from NDT techniques may be used to better understand damage mechanisms such as initiation and propagation. Identifying these factors allows for improved estimates of the remaining service life which leads to a more reliable Infrastructure Management System (IMS) [START_REF] Rong-Yau | Exploring the deterioration factors of RC bridge decks: a rough set ap-proach[END_REF]. A reliable IMS can help the decision makers to optimize intervention strategies and enable them to make informed decisions regarding the future course of action that would maximize the potential of their investments. This aspect has attracted a growing interest in recent years as the importance of life cycle optimization and the related financial benefits continue to be recognized [START_REF] Sarma | Cost optimization of concrete structures[END_REF][START_REF] Sirca | Cost optimization of prestressed concrete bridges[END_REF]. For a well calibrated IMS, it is important that the input information from an NDT technique is accurate and comprehensive. The measure of the onsite performance of a NDT tool remains a pertinent question in the majority of cases (Schoefs et al., 2012a). The most suitable NDT for a given application will largely depend on the damage to be detected and will require an in-depth knowledge of the advantages and limitations associated with each option.

There exists a broad range of NDT techniques to choose from. NDT techniques may be partitioned into two categories: non-visual and visual based techniques. Among the non-visual NDT techniques are ultrasonic scanning [START_REF] Iyer | Ultrasonic C-Scan imaging of post-tensioned concrete bridge structures for detection of corrosion and voids[END_REF], surface wave simulation [START_REF] Kim | Nondestructive evaluation of elastic properties of concrete using simulation of surface waves[END_REF], acoustic emission techniques [START_REF] Sohn | Reference-free damage classification based on cluster analysis[END_REF]Li et al., 2011a), ground penetrating radar [START_REF] Belli | Model based evaluation of bridge decks using ground penetrating radar[END_REF], eddy current testing [START_REF] Yusa | Detection of embedded fatigue cracks in Inconel weld overlay and the evaluation of the minimum thickness of the weld overlay using eddy current testing[END_REF], as well as a lot of recent interest in vibration based techniques [START_REF] Cruz | Performance of vibrationbased damage detection methods in bridges[END_REF][START_REF] Adewuyi | Vibration-based damage localization in flexural structures using normalized modal macrostrain techniques from limited measurements[END_REF][START_REF] Jafarkhani | Finite element model updating using evolutionary strategy for damage detection[END_REF][START_REF] Li | Fractal dimension-based damage detection method for beams with a uniform cross-section[END_REF][START_REF] Talebinejad | Numerical evaluation of vibration-based methods for damage assessment of cable-stayed bridges[END_REF]. There are several specialist visual techniques such as remote visual inspection [START_REF] Nugent | Remote visual testing (RVT) for the diagnostic inspection of feedwater heaters[END_REF] and laser based scanners [START_REF] Mei | Neural network for rapid depth evaluation of shallow cracks in asphalt pavements[END_REF]Park et al., 2007), etc., yet the most common visual based approach is standard visual inspections carried out by trained engineers.

Visual inspections performed by trained engineers involve significant qualitative and some limited quantitative assessments. The quality of the assessment largely depends on the ability of the inspectors to observe and objectively record details of defects. However, this approach is prone to considerations such as operator boredom, lapses in concentration, subjectivity, and fatigue. These aspects contribute to the variability and reduced accuracy of visual inspections [START_REF] Agin | Computer vision systems for industrial inspection and assembly[END_REF][START_REF] Komorowski | Role of enhanced visual inspections in the new strategy for corrosion management[END_REF]. Furthermore, there is no agreed protocol of collection and subsequent interpretation of visual information despite playing a central role in any infrastructure maintenance manage-ment framework. One feature of visual inspections that does remain constant is that the inspections are almost always accompanied with the creation of an image archive. However, there exist very few techniques that fully exploit these images in either a qualitative or quantitative fashion. Employing an image processing based approach in conjunction with traditional visual inspection techniques seems like a natural partnership given that photographing damaged regions is already a widely embraced practice for visual inspections. The primary advantages associated with image processing offset some of the limitations of visual inspections. Image processing based detection of damages offers a far greater reproducible and measurable performance over visual inspection techniques [START_REF] Gallwey | Task complexity in visual inspection[END_REF]. Additionally, it enables a quantitative assessment of the current extent of damage, as well as offering an accurate assessment of degradation over time by analyzing archived images.

Some examples of application of image processing algorithm in visual detection of damages have been developed by researchers in the field of pipe engineering [START_REF] Iyer | Segmentation of pipe images for crack detection in buried sewers[END_REF][START_REF] Tsai | Automatic detection of deficient video Log images using a histogram equity index and an adaptive gaussian mixture model[END_REF], concrete crack detection [START_REF] Nishikawa | Concrete crack detection by multiple sequential image filtering[END_REF], and road materials [START_REF] Cord | Automatic road defect detection by textural pattern recognition based on Ad-aBoost[END_REF]. However, the image based segmentation techniques are far from being fully exploited in the field of NDT. This article presents a semi-automatic image based technique for detection and classification of damaged regions in images of infrastructural elements using texture as the basis for segmentation. The image segmentation algorithm is unique in including information derived from Grey Level Co-occurrence Matrix (GLCM) based on a quantized grey-level scale along with statistical and energy information from the pixel intensity values based on a predefined range [0,255] to form a feature vector representing the texture characteristics of the image and consequent classification of such feature vectors from all parts of the image to identify damaged region through non-linear Support Vector Machines (SVM) models.

Texture is an innate property of surfaces [START_REF] Haralick | Textural features for image classification[END_REF] which can be utilized to identify damaged regions on infrastructural elements which typically have differing textures compared to the undamaged surfaces. For human observers, texture may be qualified by terms, such as fine, coarse, smooth, rippled, molled, irregular, or lineated [START_REF] Haralick | Textural features for image classification[END_REF]. From a computational perspective, quantifying the perceived texture in an image is significantly more challenging. There are numerous texture based techniques which attempt to characterize texture: wavelet analysis [START_REF] Lu | Unsupervised texture segmentation via wavelet transform[END_REF], Laws' texture energy [START_REF] Choi | Human body parts candidate segmentation using laws texture energy measures with skin color[END_REF], First-Order Statistics (FOS) [START_REF] Gill | Ice cover discrimination in the Greenland waters using first-order texture parameters of ERS SAR images[END_REF], and GLCM [START_REF] Gadelmawla | A vision system for surface roughness characterization using the gray level co-occurrence matrix[END_REF]. This article adopts a GLCM approach in conjunction with FOS. The use of GLCM statistics in conjunction with first order descriptive statistics has been employed before by [START_REF] Abbiramy | A comparative study on human spermatozoa images classification with artificial neural network based on FOS, GLCM and morphological features[END_REF] in the field of medical imaging to detect abnormalities in human spermatozoa. Their approach entailed a pre-processing stage involving noise reduction, followed by the calculation of 15 GLCM statistics coupled with four first-order descriptive statistics from converted grey-scale images. GLCM and FOS statistics formed part of a larger feature vector that was also populated with nine additional morphological features. The authors used a single feature vector assigned to the overall image which was later classified through a neural network structure. The proposed technique in this article extends this image classification study to a color image segmentation algorithm incorporating a number of original aspects that are particularly advantageous to the chosen application.

The proposed technique involves a unique set of texture measures that are calculated at every pixel in an image using a sliding window approach. Four GLCM statistics are calculated based on a quantized grey-level scale with 8 levels. These are: Angular Second Moment (ASM), homogeneity, contrast, and correlation. Six FOS and energy information features are calculated directly from the pixel intensity values which are based on a predefined range [0,255]. These are: Shannon entropy, mean, variance, range, skewness, and kurtosis. The quantized range adopted for the GLCM statistics avoids formation of sparse matrices during computation, leading to a faster detection. The wider range adopted for the FOS and Shannon entropy offers more sensitivity allowing for more accurate and representative features to be extracted.

Non-linear SVM models are used to classify pixels as either damaged or undamaged based on the feature vector. Although GLCM had been used previously in conjunction with SVM classification (Ben [START_REF] Salem | Automatic recognition of woven fabrics based on texture and using SVM[END_REF][START_REF] Xian | An identification method of malignant and benign liver tumors from ultrasonography based on GLCM texture features and fuzzy SVM[END_REF], the approach in this article introduces two new SVM classification models; a Custom-Weighted Iterative (CWI) model and a fourdimensional input space (4DIS) model in which the feature vectors were mapped to a 4DIS. This article also presents a Receiver Operating Characteristics (ROC) based framework for parameter optimization. The parameter optimization procedure involved measuring the performance of an SVM classifier for a given pair of parameters using the α-δ method [START_REF] Schoefs | The αδ method for modelling expert judgment and combination of NDT tools in RBI context: application to marine structures, structure and infrastructure engineering: maintenance, management, life-cycle design and performance (NSIE)[END_REF] and adjusting the parameters accordingly until suitably optimized. The proposed technique was evaluated in various color spaces (Red-Green-Blue (RGB), Hue-Saturation-Value (HSV), and L * a * b * ) to determine the best segmentation space.

The following section details the methodology of the proposed technique. The proposed methodology is evaluated through the identification of six disparate damage types from six different images of ageing infrastructural elements under different lighting and environmental conditions. Section 3 evaluates the performance of the SVM classification models applied to each image in each color space. Section 4 concludes the article.

METHODOLOGY

An image based damage detection algorithm has been proposed in this article. The algorithm involves two steps; the first step is to develop a texture characteristics map of a color image. The map has been developed using feature vector for each pixel following the method described in Section 2.1. The second step is to classify the damaged regions in the image using SVM as described in Section 2.2. The methodology is illustrated in the following flowchart (Figure 1).

Texture characteristics map

A texture characteristics feature vector {v f } a,b,c has to be generated for each pixel within the original image, I, for each color channel, c, where f indicates the index of the vector element and (a,b) indicates the spatial coordinates of the pixel. The first four elements of {v f } a,b,c are obtained by computing statistics derived from a GLCM. These statistics are ASM, homogeneity, contrast, and correlation. The GLCM is primarily calculated for grey images yet may be readily extended to individual color channels. The remaining six first-order texture features are based on measures calculated from the original pixel values mapped over a range of [0,255]. These features are Shannon entropy, mean, variance, range, skewness, and kurtosis.

The feature vector for each pixel is calculated separately for each color channel and can be combined together to form a four-dimensional array. The feature vector is generated for each pixel using a sliding window, SW, that moves throughout the image and provides the basis for the GLCM statistics and the distributions used for calculating descriptive statistics and Shannon entropy. The window started at the top lefthand corner of the image and horizontally moved in steps of one pixel until it reached the end of a row, at which point it progressed onto the leftmost point in the next row. The center is indicated as (a,b) and the size of the window (N-pixel × N-pixel) is optimized for best performance. A trial and error approach is used to determine the optimal size. The optimal size of SW is not necessarily the one which most effectively describes the textural composition of one region, but the one which provides the best differentiation between damaged and undamaged zones which have distinct textures. This optimization step may be worthwhile if large batches of images featuring similar damaged surfaces are being processed. It was experimentally found, however, that the classification accuracy of the technique was not overly sensitive to the window size. Similarly, the computational time of the technique was not significantly affected by the size of SW. An increase in the size of SW was accompanied by a marginal increase in the overall computational time. In this article, a nominal window size of 10 × 10 square pixels was used.

GLCM features.

The process in which the GLCM is created is illustrated in Figure 2.

The GLCM is a matrix of frequency values of paired combinations of pixel intensities as they appear in certain specific spatial arrangements within an image or sub-image. The GLCM for each pixel is generated through a sub-image that is a sliding window, SW, centred on the pixel. Combinations of various pixel pairs within SW were counted and the resulting total was assigned to the g ij , in the GLCM which corresponds to the spatial arrangement of the pixel pairs being summated. The spatial indices i and j of the GLCM match the grey level in the reference pixel and the destination pixel, respectively. The spatial arrangement of the reference pixel and destination pixel in relation to each other in SW are governed by two parameters: the interpixel distance, d, and the angle of offset, θ . The grey levels are defined using integer values between 1 and G. In this article, the grey levels are defined on a scale of 1-8 (G = 8) instead of a larger scale such as [0,255]. Quantizing in this manner increases computational parsimony at the expense of making the GLCM less sensitive to minute fluctuations in pixel intensity values within the sliding window. Despite this reduced sensitivity, the discrimination capabilities of the GLCM remain largely unperturbed as perceivable changes in intensity values between neighboring pixels continue to be taken into account, thus creating condensed yet still descriptive matrices. An illustrated example of the creation process for a GLCM is presented in Figure 2. In this example, the number of occurrences of pixels with a quantized grey level of 4 and 5 appearing horizontally alongside each other in the sliding window (d = 1 and θ = 0 • ) are computed. The number of occurrences of this pair is then assigned to the (4, 5) element in the GLCM corresponding to the chosen value of d and θ. It was experimentally found that paired combinations of intensity values of pixels that are spatially neighboring tend to be more relevant than combinations that involve spatially distant pixels. Accordingly, a value of 1 was chosen for d to ensure a certain level of spatial proximity. The angle along which the interpixel distance was counted, was defined as the angles of offset and the four angles for the offset that were chosen are, θ = 0

• , θ = 45 • , θ = 90 • , θ = 135 • . So, this generated a set of 4 GLCMs (d = 1; θ = 0 • , 45 • , 90 • , 135 • ) for each color channel at each pixel.
The GLCM for each pixel is populated as:

(g i j ) d,θ = N u=1 N z=1 A whereA = 1 if s uz = iand s d,θ uz = j 0 otherwise (1)
where s uz is the pixel intensity expressed in quantized grey levels for the reference pixel located at row u and column z within the sliding window; s d,θ uz , is the pixel intensity expressed in quantized grey levels for the destination pixel located at an interpixel distance d along an angle θ from the reference pixel. The GLCMs are normalized as:

p(i, j) d,θ = (g i j ) d,θ N(N -1) (2)
The following four texture features are determined from the GLCM:

Angular Second Moment (ASM) represents the uniformity of distribution of grey level in the image.

(v f =1 ) d,θ = G i=1 G j=1 p(i, j) d,θ 2 (3) v 1 ranges from 1/G 2 to 1. A value of 1 indicates a uni- form image.
Homogeneity gives a measure of the similarity of grey levels in the image.

(v f =2 ) d,θ = G i=1 G j=1 m • p(i, j) d,θ where m = |i -j| (4)
v 2 ranges from 0 to G-1. A value of 0 indicates a strong similarity of grey levels in the image.

Contrast is a measure of the local variations present in an image. If there is a high amount of variation the contrast will be high.

(v f =3 ) d,θ = G-1 m=0 m 2 ⎧ ⎨ ⎩ G i=1 G j=1 p(i, j) d,θ ⎫ ⎬ ⎭ (5) v 3 ranges from 0 to (G-1) 2 . A value of 0 indicates a uni- form image.
Correlation is a measure of the grey level lineardependencies in an image. Correlation will be high if an image contains a considerable amount of linear structure.

(v f =4 ) d,θ = G i=1 G j=1 (i j) p(i, j) d,θ -μ1 μ2 σ 1 σ 2 (6)
where μ1 , μ2 , σ 1 , and σ 2 are the means and standard deviations of the marginal probability matrices, P 1 and P 2 , obtained by summing the rows and columns of p(i, j) d,θ , respectively. v 4 ranges from -1 to 1. A value of 1 indicates a perfectly positively correlated image. An undefined value is returned in the case of a uniform image.

2.1.2 Descriptive statistics and Shannon entropy. The feature vector was further populated by considering five descriptive statistics of the pixel intensity values, along with Shannon entropy. These six features were derived for each pixel using the same sliding window approach employed to calculate the GLCM features. Unlike the GLCM approach, the intensity values used in the distribution adopted the scale [0,255] for several reasons. First, the nature of the statistics generated directly from the intensity values differed from that of the GLCM statistics as it was the magnitude of the intensity values that was considered and not their frequency of occurrence. As such, it was more important for the intensity values to contain as much information as possible which required them to be accurately and precisely defined.

Having a bigger sample space provided more sensitive information for characterizing texture. Conversely, the GLCM statistics produced more meaningful results by having similar intensity values grouped together as separately counting perceptually close values may understate their prominence in the sliding window. Secondly, the number of grey-levels employed in the GLCM generation stage directly affected the size of the GLCM, which in turn affected the computational time of the algorithm. The intermediate GLCM generation stage already accounted for a significant portion of the algorithm time so for this reason it was desirable to keep the size of the GLCM to a minimum. Employing quantized intensity values for the descriptive statistics and Shannon entropy on the other hand resulted in no benefits in terms of increased computational efficiency.

A key point to note is that the range of intensity values differed for various color channels. To ensure equality and compatibility, a standardization procedure was employed which linearly scaled the original pixel intensity value I a,b,c in each plane to a new pixel intensity, I a,b,c , such that it was in the range [0,255] as per:

I a,b,c = 255 × I a,b,c -min(I a,b,c ) max(I a,b,c ) -min(I a,b,c ) (7)
This standardization procedure was necessary as the customary range of values in some color channels would be less conducive to statistical analysis. For instance, the typical range for the a * and b * channels in the L * a * b * color space is [-128,128]. Proceeding with this range would lead to the Shannon entropy producing meaningless and undefined results for all distributions that had values in the bottom half of the range, [-128,0]. Using scaled pixel intensity values in the sliding window, denoted by suz , avoids this problem.

As with the GLCM based statistics, each of the FOS in the feature vector describes some aspect of the textural composition in a sliding window. The meaning and contribution of each statistic is discussed. Shannon entropy, v 5 , is a statistical measure of the uncertainty associated with a random variable.

(v f =5 ) = -
(v f =6 ) = 1 N 2 N u=1 N z=1 suz (9)
v 6 can range from the minimum value of suz , 0, to maximum value of suz , 255.

Variance is a measure of how far a set of numbers is spread out from the mean.

(v f =7 ) = 1 N 2 N u=1 N z=1 (s uz -v 6 ) 2 (10)
v 7 ranges from 0 to (max(s uz )-min(s uz )) 2 4

, which equates to 1.625×10 4 for the [0,255] range.

Range gives the difference between the maximum and minimum intensity values in the distribution:

(v f =8 ) = max(s uz ) -min(s uz ) ∀(u, z) (11)
Skewness is a measure of the asymmetry of the data around the sample mean.

An estimate for the skewness is

(v f =9 ) = 1 v 3 / 2 7 N u=1 N z=1 (s uz -v 6 ) 3 (12) v 9 ranges from -∞ to ∞.
Kurtosis is a measure of the peakedness of a distribution. A positive value for kurtosis indicates that the distribution has a greater peakedness than that predicted by a normal distribution, although a negative value indicates that the distribution is less peaked than predicted by a normal distribution.

An estimate for the kurtosis is given by

(v f =10 ) = 1 v 2 7 N u=1 N z=1 (s uz -v 6 ) 4 -3 (13) v 10 ranges from -2 to ∞.
As with the GLCM statistics, undefined values, or infinite values, can result for certain descriptive statistics such as skewness and kurtosis when the intensity values in the window are perfectly uniform, that is, when the standard deviation is equal to zero. The value of entropy may also be undefined in the case of pixel intensities having a value of zero in a given distribution. These undefined values are ignored by the SVM classifier. Their influence on the classification accuracy is negligible however as not only do the undefined values tend to appear infrequently, but by having a large feature vector containing a greater number of correctly defined texture measures, their effect is vastly diminished. Moreover, since the sensitivity of each texture measure varies according to the surface type and damage form, having a large feature vector is useful as it ensures that the influence of any texture measures that is ineffective at differentiating between damaged and undamaged regions is offset by other texture features that have a higher sensitivity to regions of contrasting texture.

Non-linear SVM classification

SVM are used to classify pixels as being either damaged or undamaged, based on the texture feature vector assigned to each pixel. SVM is a supervised learning classifier based on statistical learning theory. The linear SVM is used for linearly separable data using a (k-1) dimensional hyperplane in k-dimensional feature space [START_REF] Vapnik | The Nature of Statistical Learning Theory[END_REF]. This hyperplane is called a maximummargin hyperplane which ensures maximized distance from the hyper plane to the nearest data points on either side in a transformed space. For linearly non-separable data a non-linear SVM is used which relies on kernel function and maximum-margin hyperplane. The kernel function is adopted for non-linear classification instead of the dot product between the data points and the normal vector to the hyperplane as used for the linear classification. The kernel function concept is used to simplify the identification of the hyperplane by transforming the feature space into a high dimensional space [START_REF] Boser | A training algorithm for optimal margin classifiers[END_REF][START_REF] Cortes | Support-vector networks[END_REF][START_REF] Cristianini | An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods[END_REF]. The hyperplane found in the high dimensional feature space corresponds to a nonlinear decision boundary in the input space.

In SVM the classifier hyperplane is generated based on training data sets. Given a training data set of l points in the form {(x h , y h )} l h=1 , where h denotes the hth vector in the data set, x h is a k-dimensional input vector (x h ∈ R n ), and y h is an instance label vector (y h ∈ {1, -1} l ); for this study, a value of +1 indicates presence of damage and -1 indicated absence of damage. To identify the maximum-margin hyperplane in the feature space, the SVM requires the solution of the following optimization problem:

{w, e} = arg min w,e,ξ 1 2 w T w + C l h=1 ξ h ; C > 0 subject to y h (w T ϕ(x h ) + e) ≥ 1 -ξ h ; ξ h ≥ 0 (14)
The function ϕ maps the training vectors x h into a higher dimensional space. The vector w is the weight vector which is normal to the hyperplane, e is the bias, ξ is the misclassification error, and C is the cost or penalty parameter related to ξ . The solution to the problem is given by min

α 1 2 l h=1 l q=1 α h α q y h y q K(x h , x q ) - l h=1 α h (15)
With Constraints:

l h=1 α h y h = 0 0 ≤ α h ≤ C, h = 1, . . . l (16)
where K is the kernel function, α is the Lagrange multiplier, q is the index of the input point x q . The Radial Basis Function (RBF) kernel has been used here,

K(x h , x q ) = exp -γ x h -x q 2 , γ > 0 (17)
where γ is a kernel parameter. There are two preselected parameter values for the SVM, C and γ . To estimate the optimum parameter values, a novel ROC curve based optimization framework was employed. In this article, the training data set was obtained using texture features from both damaged and undamaged regions in an analyzed image.

SVM models

Two models were explored to determine the most accurate and efficient approach. The first stage was common to both models and involved training of the SVM with a training set of data. However, the dimensions of the input vectors in the training data sets were different for each model. Also, different implementation methods were carried out in the SVM classification stage. The performance and computational time of each approach was noted.

2.

3.1 CWI model. The input vector of each pixel comprises 30 elements: the 10 features stacked together for 3 color channels. A single binary output was achieved by introducing a weighting system which gave a greater prominence to texture measures relating to greater difference between damaged and undamaged regions. The damaged and undamaged zones were identified from the training data. The equation for the weight, W, is as follows:

W v f = v f, damaged -v f, undamaged v f, total (18)
where v f, damaged and v f, undamaged are the averages for the f th texture descriptor in the feature vector for the damaged and undamaged regions in the training data respectively. The average of the overall training data is v f,total . The normalized weight, ω v f , is then assigned to each texture feature, v f .

ω v f = W v f v f W v f (19)
A fundamental issue with the CWI model was that it required 30 separate applications of SVM classifier, one for each of the 10 texture features in each of the three color channels. This does not represent the most effective approach in terms of computational time. However, the weighting system was found to be quite successful in terms of classification accuracy.

4DIS model. The other model considered a 4D

input space where the feature vector and the color channels create two dimensions along with image coordinates for the remaining two dimensions. The SVM is applied once to separate the 4D input space into damaged and undamaged segments using a cubical space. It was found that this approach offered the fastest classification time with comparable classification accuracy to the CWI model.

Performance indicators

The performance of the texture analysis based detection in conjunction with each of the SVM models is evaluated by plotting performance points as a coordinate in the ROC space where the Detection Rate (DR) and the Misclassification Rate (MCR) are the vertical and horizontal coordinates respectively (Rouhan and Schoefs [START_REF] Rouhan | Probabilistic modelling of inspection results for offshore structures[END_REF][START_REF] Schoefs | Assessment of spatially dependent ROC curves for inspection of random fields of defects[END_REF]. The DR and MCR are represented as a percentage between 0% and 100%.

The DR and MCR are defined as:

DR ≈ Card(Q) n c with Q = {g ∈ ; y k = 1} (20) MC R ≈ Card(T) n with T = {g ∈ ; y k = -1} (21)
where Card(.) indicates the cardinality of a particular set, = {1, . . . , n}, n c denotes the number of corroded pixels, and T gathers situations of incorrectly detected pixels and undetected corroded pixels although Q gathers the correctly detected ones. The ROC space provides a common and convenient tool for graphically characterizing the performance of NDT techniques and its usage has been extended to image detection [START_REF] Pakrashi | ROC dependent event isolation method for image processing based assessment of corroded harbour structures[END_REF]. A box counting approach [START_REF] O'byrne | Receiver operating characteristics of a modified edge detection for corrosion classification[END_REF] was employed to calculate n c for each image in each color space. The DR and the MCR values formed the basis of selecting the performance point in the ROC space employing the α-δ method [START_REF] Baroth | Construction Reliability[END_REF][START_REF] Schoefs | The αδ method for modelling expert judgment and combination of NDT tools in RBI context: application to marine structures, structure and infrastructure engineering: maintenance, management, life-cycle design and performance (NSIE)[END_REF]. This method relies on calculating the angle, α, and the Euclidean distance, δ, between the best performance point and the considered point to give a measure of the performance of the NDT associated with the point under consideration. The best performance point is defined as an ideal NDT with 100% detection and 0% misclassification rates and represented in the ROC space with coordinates (0,100). For the current article the δ parameter alone may be used for comparison. A low value for δ is indicative of a strong performing technique.

EVALUATION OF PROPOSED SEGMENTATION TECHNIQUE

The proposed segmentation technique was applied on six images of various forms of damage on the surface of infrastructural elements. To assess the robustness of the technique, the six images were chosen to reflect a broad range of surfaces, damage forms, viewing angles, lighting conditions, and image resolutions as shown in ) so as to determine whether a particular color space offered a superior level of performance.

Results

The following subsections present the results obtained from the CWI and 4DIS models for each color space. The final subsection details the procedure for selecting the SVM parameters: the penalty parameter of the error term, C, and the kernel parameter, γ , so as to optimize the classification accuracy. 3.1.1 CWI model. The detected regions using the CWI model are shown for each color space in Figures 678.

The detection and misclassification rates are summarized in Table 1. The CWI model performed well in terms of identifying the locations of the damaged regions in the sample images. However, these regions were often poorly defined in many instances, resulting in reduced DR. An example of this is Figure 6c, where the damaged regions in the RGB space have been located, but the identified damage is not homogeneous and the outer boundaries of the damaged areas are inadequately identified. This problem is also observed in the other color spaces.

The classification accuracy was found to be dependent on color space. The δ values in Table 1 provide generally yielded quite poor results as it was found that these planes were relatively nondescript and offered little distinction between damaged and undamaged regions in terms of texture. As a consequence, the CWI model had to rely disproportionately on the texture descriptors in the L * plane to obtain a reasonable result. However, this issue was largely offset in the case of image (f) as the dominant colors in the image were varying shades of grey resulting in the L * plane containing a high proportion of the original image data.

As a result, this image was largely unaffected by the poor performances of the statistical analysis in a * and b * planes.

The explanation for the poor performance in the RGB color space may be attributed to the high correlation between its RGB components [START_REF] Cheng | Color image segmentation: advances and prospects[END_REF]. The pixel intensities from the Red, Green, and Blue color channels are all correlated as they contain the same light and contrast information as received by the scene. Hence, the image descriptions in terms of these components make discriminating damaged and undamaged regions difficult. Descriptions in terms of hue-saturation-brightness are often more distinct and therefore more relevant for detection purposes, a point reinforced by the good results attained from the six sample images in the HSV color space. 2. The 4DIS model succeeded at defining the damaged regions to a better extent than the CWI model. In the majority of the cases, there were only a few spurious regions that were misclassified as being damaged. However, there were occasional cases where comparatively large portions of undamaged regions in the images were misclassified such as in Figure 10f and Figure 11a. As with the CWI model, the performance levels varied significantly between the color spaces. The HSV color space was deemed as the best option. The slightly worse performances in the RGB and L * a * b * color spaces may be attributed to the same reasons as outlined for the CWI model.

Comparison of model performances

A graphical comparison of the models for each image and color space is provided in Figure 12, in which the performance points corresponding to each model-color space combination are plotted in the ROC space.

To ensure comparability between the CWI and 4DIS model, the same training data were used for each model and color space. Of the six images tested in each color space, the 4DIS model outperformed the CWI model in 61% of cases. However, the performance of the models varied from image to image, with some images responding better to classification via CWI although the other images attained comparatively better results with the 4DIS model (Figure 12).

Both the models performed consistently in different color spaces and the HSV color space typically provided the best results. It is evident in Figure 12, where the performance points corresponding to the HSV color space for both the CWI and 4DIS models are far closer to the best performance point as compared to the other points in the ROC space. The RGB color space used in conjunction with the 4DIS model and the L * a * b * color space used with both the CWI and 4DIS models achieved similar performance levels, reflected by the δ values for a given image in these color space-model combinations. The images in the RGB color space analyzed using the CWI model showed the poorest performance accuracy although the images in the HSV color space analyzed using 4DIS model achieved the best performance accuracy.

Computation times of models

Whilst both models produced apparently comparable classification accuracy, their respective computational times provide a conclusive source of differentiation, with the 4DIS model being the superior option. The computation times for all sample images in RGB space for both models are presented in Table 3 for illustrative purposes. The other color spaces demonstrate similar results. The different computation times for the sample images can be attributed to the image size. A significant portion of the time in the CWI model may be attributed to its weighting system, which is required to calculate the dissimilarity between texture fea- tures in the damaged and undamaged zones, along with the inefficient application of the SVM classifier which is required to be iteratively performed for 30 times.

Parameters of the SVM classifier

SVM classification requires a penalty parameter of the error term, C, and the kernel parameter, γ , which define the decision boundary. The parameters should be chosen carefully to produce an effective classifier.

An ROC based optimization framework was adopted through which the two components were independently optimized (Schoefs et al., 2012a). Whilst theoretically it would be preferable to search for the optimum (C, γ ) pairing, it was found experimentally that this exhaustive and computationally intensive approach was largely unnecessary as the C and γ values were largely independent of each other. For illustrative purposes, the performances of various C and γ values for Figure 3a, analyzed through the 4DIS model, are presented in Tables 4 and5 respectively. The corresponding ROC curves are displayed in Figures 13a andb, respectively. The δ values attained for the set of parameter values trialled indicate that the optimum values for C and γ were 0.5 and 1, respectively. It was found that combining these independently optimized parameters provided satisfactory results, with negligible differences from that of the jointly optimized (C,γ ) pair. Moreover, the classifier demonstrated a low sensitivity to deviations from the optimal pairing suggesting that a highly optimized pairing was not integral to the classifier performance. This was especially true for the penalty parameter, C, which returned similar performance levels across a range spanning multiple orders of magnitude.

CONCLUSION

This article presents a semi-automatic texture analysis based technique for the detection and classification of damaged regions on the surface of infrastructural elements. The technique involves generating a texture feature vector for each pixel in the image including information derived from GLCM based on a quantized greylevel scale along with statistical and energy information from the pixel intensity values. The pixels are consequently classified through non-linear SVM models.

The proposed technique has a number of favorable aspects:

1. Each pixel is qualified through a large feature vector containing ten texture related measures representing both grey-levels and pixel intensities in appropriate scales providing a well-rounded description of the image in terms of the textural characterizing. This aspect also increases the robustness of the technique as some measures may be good at differentiating regions in one image and may not necessarily be particularly useful in another image. This robustness is showcased by the ability of the technique to perform effectively when applied to images featuring a broad range of surfaces and damage forms, exposed to various lighting conditions, viewing angles, and resolutions. 2. The technique is more immune to variations in lighting conditions than color based techniques, where only the pixel intensity values are considered as opposed to texture based segmentation techniques in which the relationship between adjacent pixel intensity values are considered. This relationship is often maintained to a significant extent even when inherent chromatic and luminous complexities are introduced to the scene. 3. The technique requires only three parameters to be optimized: the size of the sliding window and two SVM parameters. A ROC curve-based optimization framework has been presented which shows a simple means of attaining suitable values for the SVM parameters. The size of window can be independently chosen through trial-and-error.

Two SVM classification models have been explored: a CWI model and a 4DIS model. The CWI model employed a weighting scheme based on the relative differences of textural descriptors in the damaged and undamaged training data. The SVM was applied iteratively to each texture measure in each color channel. The 4DIS model offered a more efficient approach, requiring only one application of the SVM. The model had the fastest computational time and, overall, achieved slightly better classification accuracy over the CWI model.

The proposed technique was performed in RGB, HSV, and L * a * b * color spaces. The HSV color space, in conjunction with the 4DIS model, offered a consistently high level of performance in a time efficient manner, and is thus concluded to be the best combination.

Fig. 1 .

 1 Fig. 1. Methodology Flowchart.

Fig. 2 .

 2 Fig. 2. Overview of the GLCM process.

  from -(N 2 max(s uz ) log 2 (max(s uz ))) to infinity, which for a pixel intensity range of[0,255] becomes [-2039 N 2 ,∞].Mean gives the arithmetic average of the intensity values in a window.

Fig. 3 .

 3 Fig. 3. Sample images in the RGB color space: (a) pitting corrosion on metal sheet piling, (b) corroded metal sheeting, (c) corrosion at a half joint, (d) staining through bridge deck, (e) marine growth on underwater steel surface, (f) exposed bridge deck through pavement.

Figure 3 .

 3 The sample images in the figure depict, (a) pitting corrosion on metal sheet piling in marine conditions, (b) corroded metal sheeting in coastal regions, (c) corrosion at a half joint on bridge span, (d) staining through bridge deck shown from underneath, (e) marine growth on the surface of underwater steel pile wharf, and (f) exposed concrete bridge deck through wear of pavement surfacing; all in RGB color space. The sample images are shown in HSV and L * a * b * in Figures 4 and 5, respectively. The technique was performed on the images in all three color spaces (RGB, HSV and L * a * b *

Fig. 4 .

 4 Fig. 4. Sample images in the HSV color space.

Fig. 5 .

 5 Fig. 5. Sample images in the L * a * b * color space.

  a quantitative measure of the variation in classification accuracy among color spaces. HSV color space achieved a high level of performance on a consistent basis although the RGB and L * a * b * color spaces were prone to more varied performance levels. The DR values for the sample images in L * a * b * space were generally high but were accompanied with a high MCR as well. The images in RGB color space on the other hand showed moderate DR and high MCR values. Although the HSV color space generally outperformed the RGB and L * a * b * color spaces, a notable exception to this was in the case of image (f) which produced the best result in the L * a * b * color space. The L * , or lightness, plane in L * a * b * is essentially the original image with the color data reduced to certain shades of grey only. The L * plane typically responded well to feature extraction by means of statistical analysis. Conversely, extracting textural features through statistical analysis in the a * and b * planes in L * a * b *

Fig. 6 .

 6 Fig. 6. Detected regions using the CWI model for the RGB color space.

Fig. 7 .

 7 Fig. 7. Detected regions using the CWI model for the HSV color space.

Fig. 8 .

 8 Fig. 8. Detected regions using the CWI model for the L * a * b * color space.

Fig. 9 .

 9 Fig. 9. Detected regions using the 4DIS model for the RGB color space.

  3.1.2 4DIS model. The detected regions for the 4DIS are shown for each color space in Figures 9-11. The detection and misclassification rates are summarized in Table

Fig. 10 .

 10 Fig. 10. Detected regions using the 4DIS model for the HSV color space.

Fig. 11 .

 11 Fig. 11. Detected regions using the 4DIS model for the L * a * b * color space.

Fig. 12 .

 12 Fig. 12. Performance points in the ROC space showing the performance of the classification models in each color space for images a-f.

Fig. 13 .

 13 Fig. 13. (a) ROC curve for varying values of C (γ kept constant at 1). (b) ROC curve for varying values of Gamma (C kept constant at 0.5).

Table 1

 1 Performance for the six images in each color space using the CWI model

						Color space
	RGB L Sample image HSV DR MCR δ DR MCR δ DR	MCR	δ
	(a) Pitting corrosion	83.96%	30.89%	0.35	84.30%	7.20%	0.17	85.50%	34.64%	0.38
	(b) Corroded metal	96.31%	19.69%	0.20	98.00%	20.10%	0.20	94.05%	25.99%	0.27
	(c) Half-joint damage	70.82%	10.77%	0.31	94.65%	11.98%	0.13	99.96%	23.23%	0.23
	(d) Stained deck	83.57%	36.44%	0.40	79.49%	23.50%	0.31	80.58%	26.10%	0.33
	(e) Marine growth	70.16%	40.48%	0.50	43.80%	20.93%	0.60	43.25%	20.12%	0.60
	(f) Exposed deck	66.61%	29.05%	0.44	54.38%	6.06%	0.46	73.85%		9.09%	0.28

* a * b *

Table 2

 2 Performance for the six images in each color space using the 4DIS model

						Color space
	RGB L Sample image HSV DR MCR δ DR MCR δ DR	MCR	δ
	(a) Pitting corrosion	77.78%	32.04%	0.39	88.66%	10.47%	0.15	89.92%	30.22%	0.32
	(b) Corroded metal	95.92%	24.14%	0.24	80.25%	9.95%	0.22	86.10%	14.69%	0.2
	(c) Half joint damage	94.69%	25.91%	0.26	92.45%	8.02%	0.11	83.22%	10.96%	0.2
	(d) Stained deck	71.43%	19.35%	0.35	67.19%	22.96%	0.4	53.27%	16.05%	0.49
	(e) Marine growth	64.11%	28.96%	0.46	67.89%	22.45%	0.39	43.49%	15.98%	0.59
	(f) Exposed deck	52.06%	10.11%	0.49	86.47%	23.72%	0.27	96.70%	36.21%	0.36

* a * b *

Table 3

 3 Classification times for the 4DIS and CWI models

			Time taken
			(seconds)
		Image size	4DIS	CWI
	Image	(sq pixels)	model	model
	(a) Pitting corrosion	1,056×1,408	46.5	893.6
	(b) Corroded metal	635×846	12.6	322.3
	(c) Half joint damage	436×648	6.5	130.6
	(d) Stained deck	441×427	3.0	97.6
	(e) Marine growth	1,056×1,408	51.5	875.2
	(f) Exposed bridge deck	255×391	2.5	47.0

Table 4

 4 Performance of SVM for various C values (γ kept constant at 1)

	C-value	DR	MCR	δ
	0.001	57.00%	11.88%	0.45
	0.25	67.90%	17.65%	0.37
	0.5	70.20%	20.20%	0.36
	0.75	74.90%	26.92%	0.37
	1	77.80%	31.90%	0.39
	10,000	77.61%	33.13%	0.40
		Table 5
	Performance of SVM for various γ values (C kept constant
		at 0.5)
	γ -value	DR	MCR	δ
	0.25	39.72%	6.24%	0.61
	0.5	57.24%	10.65%	0.44
	0.66	62.45%	13.72%	0.40
	0.75	64.44%	15.44%	0.39
	0.8	66.14%	16.74%	0.38
	1	70.28%	20.22%	0.36
	1.33	77.74%	31.45%	0.39
	2	86.79%	53.83%	0.55

ACKNOWLEDGMENT

The authors wish to thank the Irish Research Council for Science, Engineering and Technology (IRCSET) for providing a grant to support this research and CA-PACITES/IXEAD society for the practical and financial assistance.


Achim Ebert
email: ebert@cs.uni-kl.de

Alan Dix

Nahum Gershon
email: gershon@mitre.org

Margit Pohl
email: margit@igw.tuwien.ac.at

Human Aspects of Visualization

Introduction

This book is intended to give an overview of important issues concerning Human-Computer Interaction and information visualization. It is based on an IFIP [START_REF]International Federation for Information Processing[END_REF] workshop that took place during the Interact'09 conference in Uppsala [3]. This workshop, co-organized by IFIP WG TC13.7 on "HCI and Visualization" [START_REF]Proceedings of the 12th IFIP TC 13 International Conference INTERACT'09[END_REF] and the European VisMaster Coordinated Action [START_REF]VisMaster: Visual Analytics -Mastering The Information Age[END_REF], surveyed and expanded our understanding of the cognitive and perceptual issues of interactive visualization and visual analytics and brought together researchers interested in these issues. It outlined the research required to understand what aspects of analysis match human capabilities most closely, and how interactive visual support should be designed and adapted to make optimal use of human capabilities in terms of information perception and processing. This is a challenging agenda that has to take into account many factors including user roles, tasks, collaborations, interests and previous knowledge as well as understanding the capabilities of display devices and computation methods for dealing with often very large data sets. The workshop goals required the cooperation of scientists from different disciplines addressing aspects of visualization and analysis. This enabled a mapping between foundational theories in the different research areas to support collaboration among researchers in these fields. This included researchers and developers already working at the intersection of HCI, visualization, visual analytics, and related areas such as data mining, data management, perception and cognition.

Goals and Issues

Displays affect the visual interaction of human with computer devices more than any other hardware component. The display determines, for example, the physical size of the viewport, the range of effective field-of-view and resolution that is possible, and the types of feasible interaction modalities. All other interactions must be placed in the context of the display. Thus in recent years, two trends have been emerging.

First, visualization on large displays has become an increasingly important research topic. This is manifested in many applications in the use of wall-sized display devices. For example, large public screens enable a bigger audience to view the contents. Large displays for collaborative work provide the necessary screen real estate to allow the display of different points of view of the different users. The use of large display areas enables a more thorough immersion in virtual reality applications. And finally, large high-resolution displays enable visualization of large and complex datasets by facilitating both overview and detail views at the same time.

On the other hand, the proliferation of mobile devices requires visual applications on very small screens. With these smaller-scale displays, assumptions of the normal desktop metaphor no longer apply and mice and keyboards, designed for desktop interaction, become unsatisfactory. This calls for novel interaction devices and techniques designed for small screens of mobile devices.

Regardless of the device in use, interactive visualization methods are needed to augment cognitive resources by providing an additional, external visual resource to the human memory. They may reduce the amount of searching and make it easier to recognize patterns as well as enhance the understanding of relationships especially in large amounts of data and information. In addition, interactive visualization methods provide a medium that enables the user to have a representation of information that he or she can quickly and easily modify, restructure or consider from a different perspective. This ability to manipulate the data is of extreme importance especially for analytical reasoning and sensemaking.

Research in HCI and visualization has produced numerous forms of visualization and methods for interacting with visualizations of large data sets. However, systematic understanding of the interaction issues with visualization and the human cognitive and perceptual processes involved is perhaps less well developed. Therefore, the workshop at Interact 2009 tackled the human aspects of visualization and built an inspiring platform for presentations and intensive and controversial discussions. The articles in this book capture the exciting presentations and discussions about these issues.

The Articles in this Book

The following section is a short overview of the papers in this book. All but the paper of Bob Spence -the keynote speaker at the workshop -are listed in alphabetical order (name of the first author).

The Broker Bob Spence (Keynote)

There is a considerable amount of existing perceptual and cognitive knowledge that is highly relevant to information visualization. Nevertheless, it is difficult to directly apply it to the design of information visualization. Much of the research is very abstract in nature. Some results might be context-dependent and could not be easily generalized to information visualization applications. In his keynote lecture, Bob Spence, therefore, advocated that "Brokers" should mediate between cognitive scientists and information visualization designers. "The task of a Broker is to interpret relevant knowledge acquired by cognitive and perceptual psychologists and bring it suitably to the notice of interaction designers, thereby avoiding the need for that designer to have knowledge of cognition and perception." A suitable method to do this brokerage is to develop Design Actions. Design Actions are structured forms that provide design guidance at a useful level of detail. In a way, they translate knowledge from cognitive psychology into a "language" that can be understood by designers. Basically, they are guidelines, but they also contain information about the empirical evidence supporting them, and advantages and disadvantages of following the guideline. Bob Spence's article provides a valuable foundation for the following papers, many of which try to connect psychological research with design issues of information visualization.

Comparing Different Layouts of Tag Clouds: Findings on Visual Perception

Stephanie Deutsch, Johann Schrammel, Manfred Tscheligi Tag clouds are an example of casual information visualization as described in the article of Andreas Kerren (see below). They can support users who want to get a comprehensive overview of the annotations that are often chaotically made by communities on the web. Clustering can produce meaningful units within tag clouds, but these units have to be presented in a way that can be easily perceived and understood. The authors argue that, while tag clouds are intuitively comprehensible and attractive for users, they have certain limitations, e.g. the amount of redundant annotations or the problem that concepts are understood differently by various users. In addition, more possibilities for interaction would be advantageous. They also emphasize the importance of developing a solid foundation in perceptual psychology, especially with regard to visual perception and eye movement. Their empirical study indicates that semantic clustering (in contrast to traditional methods of tag-cloud-generation) might be useful for classifying annotations. However, they point out that there are still open questions regarding the design of such visualizations.

The Personal Equation of Complex Individual Cognition during

Visual Interface Interaction Tera Marie Green, Brian Fisher Green and Fisher discuss approaches from cognitive psychology that might guide the design of interactive visual interfaces. They point out that the interaction with visual interfaces consists of complex cognitive behavior. So far, evaluation studies have focused on low-level cognitive processes (e.g. pre-attentive processing). This turns out to be interesting but not sufficient. In this context, they discuss theories of reasoning, especially the so-called sense-making loop. They point out that human reasoning processes can vary widely and therefore it is not easy to develop comprehensive theories of human reasoning. Models like the sense-making loop may form a general framework, but cannot guide concrete design decisions.

The authors also argue that human cognition is fundamentally influenced by individual differences (e.g. how humans categorize information, how they perceive visually or which problem-solving behaviors they adopt). Based on this assumption, they developed the Human Cognition Model, an operational framework to guide the design of collaborative interactive systems.

Faceted Visual Exploration of Semantic Data

Philipp Heim, Jürgen Ziegler Heim and Ziegler address the problem of searching and exploring semantic data, in particular whether faceted search might be a possible way to support such search processes in Semantic Web data. In faceted search, the search space is divided based on independent search dimensions that serve as filters. To support users in their exploration of semantic data they developed a graphical tool representing facets and search results as nodes in a visualization that exploits the graph-based structure of linked semantic data. This graphical representation can help users to develop a coherent cognitive model of the target domain.

Heim and Ziegler argue that searching semantic data differs from using relational databases and conventional information retrieval. The tool they developed is specifically targeted at searching and exploring semantic data and especially supports exploration and hypotheses refinement processes.

Fisheye Interfaces -Research Problems and Practical

Challenges Mikkel Rønne Jakobsen, Kasper Hornbaek One of the major problems in information visualization is the representation of large amounts of data and information. Cognitive overload resulting from the display of large amounts of data has to be avoided by an appropriate and effective design. There are several possibilities to reach this goal, one of which is focus+context interfaces (e.g. fisheye interfaces). Although using fisheye interfaces is a well-known technique dating back many years, Jakobsen and Hornbaek argue in their article that many problems concerning this technique are still not solved. They point out that, for example, the notion of focus is not very well defined in the literature. Consequently, different systems are developed with a large variety of interaction strategies. This makes it difficult to give an overall assessment of such systems. In addition, fisheye interfaces are seldom integrated into practical applications where they could probably be most useful. Jakobsen and Hornbaek address some of the open questions they describe in their empirical research. Their work indicates that even for well-known techniques, many usability issues are still not solved. Apart from that, their discussion also deals with more general questions of the representation of large datasets on the screen and raises issues relevant for the information visualization community as a whole.

Visualization of Workaday Data Clarified by Means of Wine

Fingerprints Andreas Kerren

In his contribution, Kerren discusses the topic of information visualization for the masses, also called casual information visualization. Originally, information visualization was predominantly developed for expert users who needed sophisticated applications for complex problems. Nowadays, an increasing number of non-specialist users are interested in such tools, but may only occasionally need them or encounter them. Such systems have to be more intuitive and easier to learn than the ones geared for professional users. Potential visualizations for such systems are simple node-link diagrams, tag clouds or treemaps. Kerren points out that intuitive and easy to learn interfaces are not always obvious. In his research, he initially assumed that treemaps would be easier to understand for casual users than bubble diagrams; however, this assumption was found to be incorrect. It is anticipated that simple visualizations will become even more important because of the increasing utilization of social software to support, for example, the perception of relationships between members of communities in Facebook.

Staying Focused: Highlighting-on-Demand as Situational

Awareness Support for Groups in Multi-display Environments Olga Kulyk, Tijs de Kler, Wim de Leeuw, Gerrit van der Veer, Betsy van Dijk

In recent years, there has been an increase in research on cooperative problem solving using information visualization techniques. It is hoped that multi-display environments may support group decision-making capabilities, as the displays can act both as communication tools and information devices. The authors point out that group decision processes are often flawed, for example, because of the dominance of single group members or because group members ignore alternative solutions.

Cooperative information visualization tools might overcome such problems by giving a comprehensive overview of existing possibilities and enabling group members to draw attention to various solutions by highlighting them. Technologies like tabletop computers or large tablets might be especially valuable in this context if they are appropriately designed and well integrated into the decision process.

Using Gaze Data in Evaluating Interactive Visualizations

Harri Siirtola, Kari-Jouko Räihä Siirtola and Räihä discuss the difficulties in carrying out effective evaluations of interactive visualizations. Because of these difficulties, evaluations of such systems have often been missing. Thus, it is difficult to assess whether interactive visualizations are really useful. One of the reasons for this lack of evaluation studies might be that carrying out effective evaluations is both time-consuming and costly. They propose a simpler methodology for evaluation by using eye tracking.

In recent years, eye tracking has become more attractive for evaluation studies because the systems have become cheaper and easier to use. Nevertheless, the authors point out that the volume of data and the lack of methodologies to generalize these data is still a problem. This is especially difficult for animations when it is difficult to measure fixations on moving objects. In addition, there are still many open questions as far as interpretation of the data is concerned.

Giga-Scale Multi-resolution Volume Rendering on Distributed

Display Clusters Sebastian Thelen, Jörg Meyer, Achim Ebert, Hans Hagen Large high-resolution displays have proven to be beneficial in various kinds of collaborative scenarios. They allow users to physically navigate the space in front of the screen and interact with applications in a highly dynamic way. Due to their extended screen space, large high-resolution displays are often employed by user groups to explore data sets that, because of their size, could not be perceived entirely on a regular small desktop display. However, visualizing such data sets is challenging since their size can exceed the computational resources of render nodes by several orders of magnitude, thus making data exploration more complicated. Thelen et al. describe a technique to visualize and explore gigabyte-sized volumetric data sets on distributed display clusters, such as tiled monitor walls or tiled projector walls. Their method uses a wavelet-based multi-resolution approach in combination with octree-based space subdivision to significantly increase the level-of-detail that each render node in the cluster is able to display. The paper describes the underlying visualization approach and analyzes the results obtained with various data sets. The implementation was successfully tested on a tiled display comprised of 25 compute nodes driving 50 LCD panels. The system was able to produce renderings of volumetric data sets larger than the texture buffer size of a single graphics card at significantly higher levels of detail than on a single desktop display.

Teaching Visual Design as a Holistic Enterprise

Gerrit C. van der Veer, Corné Verbruggen

Van der Veer and Verbruggen argue that approaches in Visual Design education are often limited because of a focus on the design of the computer screen. Such an approach does not take into consideration that the interpretation of visual representations is often influenced by a specific context and by the experiences and attitudes of the observers. Designers of the visualization tools usually cannot anticipate users' reactions and therefore develop tools that are not adaptable to users' needs. This gap between designers' and users' mental models has often been described in Human-Computer Interaction textbooks and it is the main reason for recommending user studies. But, the argumentation of the authors goes beyond this; they point out, for example, that visualization tools do not merely produce representations on the screen but also introduce physical artifacts that are part of the display device -each device has specific characteristics like portability, opportunity for dialogue, visibility in daylight or in a building. Design education has to take all of these aspects into consideration. In this sense, it should be holistic and address more than screen design. Consequently, usability is just one aspect of the story. Other factors to be considered are, for example, that the audience is often heterogeneous and comes from different backgrounds and that one needs to take into account the rules of human perception that could be analyzed by Cognitive Psychology.

Summary

Throughout the discussions during the "Human Aspects of Visualization" workshop, several common topics among the participants emerged and these common topics are reflected in the papers.

A common theme that is probably essential for this area is the relationship between research from cognitive psychology and design of interactive visualization systems. This problem was mentioned by Bob Spence in his keynote address, but can also be found in other articles in this book. Green and Fisher, for example, describe theories of reasoning and problem-solving and point out that there is no single theory in this area that might effectively explain the design of information visualizations, and therefore, one needs to make use of a number of different theories. This makes it difficult to formulate guidelines for explaining effective designs of information visualizations. Deutsch et al. also discuss psychological theories as foundation for research in information visualization. Kulyk at al. extensively discuss research from social psychology and sociology to substantiate their results. Related issues are also elaborated in van der Veer and Verbruggen's discussion of a holistic approach in design education; they point out that many different disciplines are necessary to develop successful visual designs, not only psychology but also, for example, media theory or sociology because the functioning of visual design depends on the social context and on the characteristics of the media being used.

Another common topic was information visualization for non-experts. As information visualization systems are increasingly adopted by non-expert users, their design will have to be adapted to this population. Kerren explicitly discusses this problem and tries to give an outline of possible solutions. This is also mentioned in the paper by Deutsch et al.

Methodological issues also played an important role in the workshop discussions. One open question is whether traditional methods of cognitive psychology or HCI are appropriate for the investigation of perceptual and cognitive aspects of information visualization. Methodologies such as eye-tracking (discussed by Siirtola and Räihä as well as by Deutsch et al.) offer the potential for radically different ways of approaching evaluation. However, these are themselves areas of substantial complexity. One problem is the appropriate interpretation of the data gained from eye-tracking studies and the definition of the variables that could be measured by this technology.

Another important issue that should play an important role in the future of visualization and HCI is the design of visualization systems supporting cooperative activities (see the contribution of Kulyk et al.). Technologies like tabletop computers might be the medium of choice in group situations. The papers indicate that there are still many open research questions even when one considers topics that have been studied for a considerable time like fisheye interfaces. The book can only outline some of the most prominent problems and some potential solutions to the effective application of human-computer interaction in information visualization.

As already mentioned above, the workshop at the Interact conference was co-organized by IFIP WG TC13.7 on "HCI and Visualization" (HCIV) and the European VisMaster Coordinated Action.

HCIV is a major program in Human Computer Interaction and Visualization that started in 2006. The aim of this initiative is to establish a study and research program that will combine the knowledge of both the science and the practice in the fields of HCI and Visualization. One of the main steps in organizing this program is a workshop series with world-renowned experts in the fields of Human Computer Interaction and Visualization. Since 2009, HCIV is an official working group of IFIP Technical Committee on Human-Computer Interaction (TC.13). In addition, all HCIV workshops since 2009 have been approved as an official IFIP event (including the Interact 2009 workshop on human aspects of visualization).

VisMaster is a European Coordination Action Project focused on the research discipline of Visual Analytics. The main strategic goal of this action is the shaping of a new research community for the field of Visual Analytics. It explicitly addresses issues of human cognition, perception and decision-making. To support the human visual and reasoning abilities efficiently, tools for visual analysis tasks have to be designed appropriately. The workshop helped in shaping the Visual Analytics community and hereby provided valuable input to the VisMaster project; in particular creating a foundation for the discussion of perceptual and cognitive issues in the European Visual Analytics roadmap "Mastering the Information Age" [START_REF]Mastering The Information Age: Solving Problems with Visual Analytics[END_REF].