Anders Persson
email: anders.persson@it.uu.se

Implicit Bias in Predictive Data Profiling Within Recruitments

Keywords: Data mining, Social exclusion, Discrimination, Implicit bias, Recruitment, Big Data, People analytics, Machine-learning

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

How can a company looking for a specific person for a specific role find the most likely candidate to succeed among thousands of applicants? This is a problem facing many recruiters that are either headhunting, or have a job advertisement for popular positions, with often 500-1000 applicants applying for a single job.

To meet this challenge, recruiters within many firms and organizations use profiling to screen applicants for the most promising candidates. Typically it is used as an initial big cut, where some candidates later will be interviewed and evaluated in more detail. Today, this initial big cut of candidates is often made "automatically", by an algorithm, with the selection based on a set of values. This is a technological feat that simplifies the life of a recruiter tremendously, and really can be seen as a necessary advancement to meet the conditions of hundreds and thousands of applicants for single positions, in today's job market.

What companies' uses are more specifically is data mining: machine learning with algorithms and statistical learning. This if often referred to as using "Big Data", or "People Analytics"; i.e. using large datasets to identify parameters that represents the best candidates for various positions. The motivation for companies is often the claim to become more objective in their assessment [START_REF] Chien | Data mining to improve personnel selection and enhance human capital: A case study in high-technology industry[END_REF][START_REF] Barocas | Big Data's Disparate Impact[END_REF]. They can also be claimed to want to become more certain in their decision making for hiring. But what consequences does this reliance on technology have on the process of selecting and discarding applicants? Let us start by taking a step back; if there are no other means at the recruiters and managers disposal to make judgements, they will have to rely on their intuition. Field-expert's intuition can very well be very good and accurate for making estimations within their specific area of expertise, but they will nevertheless be estimations based on subjective value assessment, and at times faulty. There is also the problem of discrimination, even without conscious intent. Today this is controlled by equal opportunities law; it is illegal to (explicitly) discriminate based on categories such as skin color, ethnicity, gender and age (and to some extent avoid also implicit discrimination). Yet, even with such countermeasures in place, discrimination seems to remain. As Pager and Shepherd exemplifies: "today discrimination is less readily identifiable, posing problems for social scientific conceptualization and measurement" [START_REF] Pager | The Sociology of Discrimination: Racial Discrimination in Employment, Housing, Credit, and Consumer Markets[END_REF]. This is often explained as the unconscious phenomenon of implicit bias.

Implicit Bias and Predictive Machines

More generally, implicit bias is often referred to as unconscious associations. This is what could lead to so-called stereotype threats; to attribute certain abilities and predicted behavior to generalized categories of people. For example, several studies have examined how different individuals judge a written application, as a resume or CV [START_REF] Bertrand | Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination[END_REF][START_REF] Jost | The existence of implicit bias is beyond reasonable doubt: A refutation of ideological and methodological objections and executive summary of ten studies that no manager should ignore[END_REF]. The only independent variable in the studies has been the name of the applicant; either native or foreign sounding, alternatively a name associated to a minority group. In USA as well as in Sweden, an applicant with a native name is much more likely to be called to an interview, than one with a foreign name, even if their resumes were identical, and their merits therefore should have been identical.

There is a similar effect for hiring women to management positions. An illuminating example of the effect of how implicit bias could work is the following: "There is good news and bad news about actual gender-related managerial differences. The good news is that some do exist. The bad news is that they are overused as the basis for sexual stereotyping." [START_REF] Sexton | Female and Male Entrepreneurs: Psychological Characteristics and their Role in Gender[END_REF] This might be a controversial finding, that there seemed to be a difference between female and male performance related to management positions. However, the authors also note that the difference is overused in recruitment. This is very much what can be seen happening when over-categorizing. While there might be a difference in performance, the difference is neither black and white, nor clear-cut. Yet it continues to affect diversity, and remains as an obstacle for diversity and equality.

I will exemplify this further with a more concrete example (see subsection 3.5 about predictive brains), but let us first get back to the machines and algorithms that try to predict the likely performance of applicants for a job position. Barocas and Selbst [START_REF] Barocas | Big Data's Disparate Impact[END_REF] argue that an analysis and an algorithm are never better than the dataset on which they are based; if the dataset is biased, so will the result be. Data sets are, just as in empirical science, historical data, which may themselves be laden with implicit biases of a discriminatory nature. Thus, even if there is no explicit rule or class inferences made in the algorithm to look for sensitive categories, the result from the algorithm may still be (implicitly) biased.

One example of this is in plain view; our language and our use of lexical words. Within our semantic use of words, implicit association tests (IAT) show how we, for example, associate the word "dangerous" more to "black", rather than "white" (mainly in a context of US). This can feel troublesome, if you discover an implicit bias in yourself, and there are several IAT's on the web that anyone can use to investigate their own implicit biases. When it comes to machine learning and algorithms, a standard use is keyword-associations, and pure statistical learning. What Caliskan-Islam et al. [START_REF] Caliskan-Islam | Semantics derived automatically from language corpora necessarily contain human biases[END_REF] show is that when running IAT's on algorithms such as these, they show similar kind of implicit biases as humans do. It makes sense, if you accept Barocas and Selbst [START_REF] Barocas | Big Data's Disparate Impact[END_REF] message that an analysis is never better than the data set, and if there are (historical) biases embedded, as there seems to be in language, the results of this kind of machine-learning algorithms will also be biased.

But, discrimination is punishable by law, you might object, and has been for quite a while in most western countries. However, as you may already have realized from the section above, the use of algorithms is technically quite complicated, and their effects are not at all transparent. Therefore, there are currently holes in the available laws, allowing some types of biased machine-based profiling, for example of applicants and employees. I will discuss if better laws could solve this, such as those that are under way in EU (see subsection 4.2), and I will also discuss solutions involving "better" algorithms and machine learning techniques (see subsection 4.1).

Solution with human thinking. My main claim in this paper, however, is that it is necessary to look deeper on the human side of the equation. Behind all the use of algorithms and computers, there is at least at some stage, a human considering a selection of applicants, or making a decision. Another approach could be to try to stimulate better thinking, with the use of assistive technological tools, and I will make an initial exploration if lessons from methodology of science can be used. As mentioned above, when using machines and data, there is often a perception that you gain certainty and objectivity; cold hard facts, if you like, that are mined from the ground like minerals [START_REF] Portmess | Data barns, ambient intelligence and cloud computing: the tacit epistemology and linguistic representation of Big Data[END_REF]. Science, however, would hold that what you need is uncertainty and doubt, and to always be open minded to question, critique, and try to falsify what you think is true. Empirical conclusions in science are, in other words, never facts set in stone, but always changing and open to new interpretations and new explanations.

Methodology and disposition.

There is an inherent problem in this sphere of investigation: companies using Big Data and data profiling. Namely that currently there are insufficient laws to force them to disclose their processes, or algorithms. As Frank Pasquele names it, we currently very much live in "The Black Box Society" [START_REF] Pasquale | The black box society: the secret algorithms that control money and information[END_REF], where we do not know what algorithms and technologies are affecting us. In lack of empirical data concerning the effects on society, what are left are either journalistic endeavors trying to peek into those black boxes, or theoretical works, looking at the mechanism of the algorithms that we at least to some extent know to be at work in the background. The latter is my approach in this paper, and in section 2 I start by looking into the mechanisms within machine learning and data profiling. This is continued with an analysis of potential problem, in section 3, and a continued discussion follows in section 4, in which I also propose different solutions and remedies to tackle the problems at hand. The methodological limitation of limited empirical data in this sphere is addressed more in the ending sections.

Organizations using Big Data

The term Big Data is fairly new, but the phenomenon behind it is not necessarily as new. Broadly speaking, it concerns the acquisition of useable, meaningful data, out of larger data set, with the emphasis of the scale of the data set, or multiple data sets. This can also be called data mining, and they remain closely related [START_REF] Portmess | Data barns, ambient intelligence and cloud computing: the tacit epistemology and linguistic representation of Big Data[END_REF]. Another term that is ubiquitous within the domain is machine learning, which focuses on using algorithms that "learn" to identify patterns for classification, from training data. Data Profiling is a more specific term within data management, and is defined as: "a specific kind of data analysis used to discover and characterize important features of the data sets. Profiling provides a picture of the data structure, content, rules and relationships, by applying statistical methodologies to return a set of standard characteristics about data." [START_REF] Sebastian-Coleman | Measuring data quality for ongoing improvement: a data quality assessment framework[END_REF].

In other words, what you get and can use from data mining and profiling is much like data from empirical, statistical research. What the different fields also have in common is that they mainly look for patterns in the data/training set; in much the same manner as a speculative scientist could do to generate hypotheses from empirical data.

The usage of big data and data mining is widespread. According to a survey in North America in 2014, 73% of companies stated they had invested, or planned to, invest in Big Data analytic tools [START_REF]Gartner Survey Reveals That 73 Percent of Organizations Have Invested or Plan to Invest in Big Data in the Next Two Years[END_REF]. Larger employers today are, almost, forced into using some kind of Internet-based management systems for handling job applications, either themselves or through an external agency. In such systems, they typically use Applicant Tracking Systems (ATS), to electronically manage information [START_REF] Rosenblat | others: Networked Employment Discrimination[END_REF]. From this, it is a small to also rank applicants, and typically present only the select few applicants that are deemed to be the best fit, for recruiters and managers. The rest are, in other words, disregarded and never considered by anyone but the machine and the ranking system. Below I give a glimpse into the kind of parameters that can be used in ranking systems like these, with an example of Xerox looking for customer service personnel.

Example of Xerox using Data Profiling

Xerox is a global business services company with over 130,000 employees within various technological fields, as well as organizational development. An important element in their business model is customer service support. To manage their humanresources within this field they have implemented big data analyses. One explicit performance value that they look for in a candidate is longevity within the company, as in how long a person stays within the employer [START_REF] Morse | Big data can take the guesswork out of the hiring process[END_REF]. The motivation is often that longevity of workers builds organization-specific knowledge over time, and it is important that this stays within the company. As an analytic tool they have been using Evolv, currently part of a larger platform called Cornerstone OnDemand [START_REF] Evolv | [END_REF]. Xerox state that they put applicants through a series of tests to predict how they would perform within customer service [START_REF] Morse | Big data can take the guesswork out of the hiring process[END_REF]. This is later followed up in regards to performance measures, such as longevity. Evolv claim to have "500 million points of employment data on over 3 million employees" (see Christl and Spiekerman [START_REF] Christl | Networks of Control: A Report on Corporate Surveillance, Digital Tracking, Big Data & Privacy[END_REF] for a more detailed overview of software used in hiring today). Part of the tests are surveys regarding attitudes towards work, such as how long they are willing to travel/commute to work, and how much overtime per week they would be willing to work. Additional information is also gathered from CV's.

Four measures were correlated with longevity: (1) Willingness to work 1-3 hours overtime per week, 15 times more likely, (2) Applicants with bachelor's degrees stay 5% longer, and those with technical diplomas stay 26% longer, than those with high school diplomas, (3) Those that have had a customer service job where they have had to use empathy, rather than just taking orders, and, finally, (4) living closer to job, or having reliable transportation [START_REF] Morse | Big data can take the guesswork out of the hiring process[END_REF].

It is also claimed that their ideal customer worker is someone that (1) scores well at typing tests, to be better at taking in background documentation on clients, (2) is creative, and (3) uses social media (but not too much). It is unclear how this data is correlated to performance [START_REF] Morse | Big data can take the guesswork out of the hiring process[END_REF].

The result here is a profile, with a set of variables and parameters (a pattern), which human-resource management use as a predictor of performance and how well an applicant fits for the position considered. In other words, someone that lives further away from the working place, only has a high school diploma, and does not use social media very much or at all; ends up with a low predictor and is likely to be cut at an early stage in the hiring process. Some also propose using data mining profiles together with expert systems, to help make the selection of applicant [START_REF] Mehrabad | The development of an expert system for effective selection and appointment of the jobs applicants in human resource management[END_REF]. This could be done with more or less automation, either by giving suggestions to a recruiter (a suggestion the recruiter is probably likely to follow), or by the system itself making initial selections. As mentioned above with ATS's, this process of profiling was previously more manual, and is today highly automated.

No example has been found of recruiters using content data extracted from social media to include in a job profile predictor. However, a study in Sweden showed that at least half of the interviewed recruiters in big and middle-sized organizations did scan applicant social media profiles themselves, at some point before hiring [START_REF] Backman | Will your Facebook profile get you hired? Employers use of information seeking online during the recruitment process[END_REF]. In the survey they stated they looked for risk behaviors, where the applicant shared the organization's values, and things that might damage a company's reputation. It would not be a difficult step to include data from publicly, obtainable, social media profiles of applicants, and make a keywords-association analysis to (try to) capture potential risk behavior.

This gets us into an even more intricate sphere of privacy and the use of personal data, and it will expand the ways in which a person can be unfairly disregarded and discriminated. However, it may be enough to look at the data that an applicant has to include in a resume, as I will argue for in this paper.

On the face of it, when it comes to unfairness, social exclusion and discrimination, it is important to look for diversity, and if you like, uniqueness. This term is also used in data profiling, which I will consider next.

Uniqueness in Data Profiling

An important concept with data profiling is uniqueness. Technically understood, one value for a parameter, or a set of values for a set of parameters, has maximum uniqueness when there are no other sets of values like it. Non-uniqueness is achieved when two or more sets of values are all the same, which is the strongest kind of pattern and correlation. Translated to a dataset of applicants and their level of academic degree, it would mean that maximum uniqueness is achieved when every candidate has a different level. On the other hand, if, for example, a test of programming skills is included in the dataset, maximum uniqueness would be when every individual has a different test score. This might become a bit more complicated when you have numbers with decimals, and in general higher uniqueness when there are a greater number of possible values. The principle, however, remains the same: that minimal uniqueness will be acquired when (groups of) individuals are (nearly) identical.

Taking the same dataset and looking at a relation between variables of programming skills, as a performance measure, and academic degree, minimal uniqueness is achieved if all individuals with high programming skills also are at a certain level of degree. This would make these easy to identify; just by looking at their academic degree it would be possible to tell something about their programming skills. On the other hand, if programming skills and academic degree are not well correlated but varies greatly, with good and bad programmer within all degrees, then uniqueness is high.

What does this mean for a recruiter who is looking at applicants and wants to find the best fit, given the motivation that you want to be as certain as possible? On one hand, you would like to be able to make the clearest distinction between, for example, a high and low performer. That you would get by having low uniqueness: if all low performers have a high school diploma, and all high performers have a bachelor's degree, the inference is easy to make, solely by looking at academic achievement. On the other hand, having high uniqueness will give you a much more narrow set of candidates that may be the best fit, and be able to disregard a larger portion of applicants. The latter likely depends on having a profile with a specialized pattern of skills and parameters.

Analysis of Potential Problems

Using profiles to cut a large portion of applicants might be seen as an efficient way of siphoning out the candidates less likely to perform well, and consequently a mere "fair and square" competition between applicants. Overall, employers and companies should be able to perform better, with more people in the right positions. Companies like Xerox also report results going upwards after using data profiling for hiring [START_REF] Morse | Big data can take the guesswork out of the hiring process[END_REF]. I will start by analyzing some potential problems in the use of data profiling as described above. Initially, there might be a problem of effectiveness, and long-term performance, and I will use the conceptualization of local optimums. Secondly, and thirdly, I will get to the problems that are of more concern in this paper; regarding discrimination and unfairness. Further on in this section I will look at some initial technological remedies to the problems, with re-profiling, which seem to deepen the problems at hand.

Local Optimums and Stagnation

An ongoing research question within product development concerns innovation and success in different economic contexts. To shed light on this matter, some have applied an evolutionary perspective, and something called a fitness landscape [START_REF] Kwasnicki | Market, innovation, competition: An evolutionary model of industrial dynamics[END_REF]. This is related to the theory of evolution, in the sense that you make an analogy to which "ideas" will survive and progress, and will have the best fitness for a specific environment. Within this concept you can also make a difference between local and global maximums, and this related to what machine learning call optimums as I will henceforth use. To explain it further; an organism, with certain attributes at its disposal (a certain genotype), adapting to a certain environment, is theorized to sooner or later end up on a local peak (performance) optimization, for that specific environment and context. Related to humans and organizations, this could mean that given a certain type of workforce you will end up at a certain level of performance. Within your own workforce you should with a large enough dataset also be able to observe different peaks. This could probably be done in larger organizations with (big) data analyses, or large collections of data points as in the program Evolv and Cornerstone (see subsection 2.1). You would then typically want to identify where the peak performances are, and spread the type of abilities that those people have to other positions in the same occupation. To put it differently; you would like to have the profile of peak performers in all relevant positions, given the current conditions.

However, when changing to a different environment, the same set of optimized abilities may not be as effective or competitive, compared to other sets of abilities. The same thing happens in the natural environment with highly specialized organism. If you change their environment, they might not be able to adapt, and dies out. The data that is used to make job profiles, like all data in empirical research; it is historical data. They do not really say anything about the future, and they only say what worked well in that historical context/environment. This could be problematic, given that the global market, influenced by people and societies all around the globe, can involve quickly changing conditions for success. This is a problem for any use of profiling, of course, not just data mined. I will discuss a potential solution with re-profiling below (section 3.4), and get back to this problem in section 4.

Problem with Discrimination

The most ethically relevant problem is probably that of discrimination in the selection process. As mentioned in the introduction, discrimination is controlled by legislation in most countries, and is usually defined as: "the treatment of a person or particular group of people differently, in a way that is worse than the way people are usually treated" [START_REF]discrimination Meaning in the Cambridge English Dictionary[END_REF]. Expressed in a different way, it is about judging people based on things like social group categories, rather than individual merit. Most commonly this is an issue about features such as: (1) gender, (2) ethnical background, and (3) sexual orientation.

It is important to note that discrimination can be direct and based on discriminatory categories, such as those mentioned. It can also be indirect; a selection based on non-discriminatory categories, which, however, are strongly correlated to the discriminatory ones. There are, for example, reports of companies scanning employee's health records, to predict those likely to get on sick-leave, or those that might be pregnant [START_REF] Wilkinson | Companies secretly tracking employees' health and lives in "big data[END_REF]. Why this is possible, and not an offence, currently, is highlighted in discussing solution with law and regulation (see subsection 4.2).

Unfairness and Loss of Opportunities

A more general problem and much related to discrimination, is simply unfairness in the selection process, and a loss of opportunity for those who deviate from (outliers) the resulting profile. That is, this is the individuals that differ from the norm of the (predicted) most likely high performers, but who in reality still is just as likely to perform at the same level. This is likely to occur, given the premise that measures for the applicants have statistical normal distributions. To use the example of Xerox (see subsection 2.1), it is unlikely that all applicants with high school diplomas would perform worse than all the applicants with a bachelor's degree. This is to some extent related to uniqueness in data profiling. Given that if the more informative measures related to performance will have lower uniqueness, they will also be less forgiving towards outliers. Those outliers could be seen as losing an opportunity, and this loss could be an ethical problem of social exclusion.

It is important to note, though, that we would probably not perceive it as unfair if other candidates in fact had stronger merits, but it would be perceived as unfair if you are excluded based on something not directly associated as a merit for the position. Xerox was, for example, looking for people with mid-range social media use, for a position in customer service. Using social media is not, at least, directly related to the task that they are expected to do, but apparently it might be indirectly related to a performance measure. It could possibly exclude applicants solely based on not using social media, or not using it enough. Given that people are adapting to the demands of the job markets they want to get into, it might mean that people will have to conform their private life, as well.

Reprofiling for Quality of Data

One approach to remedy some of the problems just mentioned might be to simply seek better quality of the data. It could be claimed that the problems of local optimums could be avoided if the profiles were continuously updated to reflect what kind of workers are needed at a specific moment in time. Even more so, if the quality of the (meaningful) data extracted from the big data set is good enough, and detailed enough, it should be possible to pick out those individual outliers who nevertheless are performing at a high level. This does not seem to be an unreasonable claim, but it might be problematic in practice. As I will show further down, the process of applying a profile might reduce all the outliers. I will also relate to this process to intuition and implicit bias in the next subsection.

Data quality is first and foremost an issue when it comes to, so to speak, living data sets that change over time (such as what correlates with performance within a certain organization on renewed global markets). To remedy this, most data sets can benefit from what is called reprofiling; a new analysis of the dataset to see if statistical patterns and relations have changed [START_REF] Sebastian-Coleman | Measuring data quality for ongoing improvement: a data quality assessment framework[END_REF].

However, a risk, as far as I can analyze it, is that rather than including outliers it might exclude them further. Let us say that we mine out a profile P1, with features such as those that Xerox are looking for in customer service personal. You then choose people based on profile P1 to be added to the workforce for the specific occupation, and then re-profile on this new population with the new selection of individuals added, yielding a profile P2.

If nothing significant has changed in regards to what measures are correlated to high performance between P1 and P2, the added employees will have reinforced the pattern, which statistically will become "stronger", as in, having less deviations from means, and lower p-values. This will most likely mean that P2 would be even less forgiving to outliers, unless your data analysis is detailed enough. Big organizations like Xerox with abundant resources at their hand might be able to make the profiles detailed enough, but would probably be difficult for others.

Revisiting Local Optimums.

So, what does this then mean for the problem of ending up on low local peaks of performance in changing market-environment? If profiles P1, or P2, have had any impact on the workforce, you would after some time have few outliers to rely on. With fewer outliers it will be more difficult to find the pattern for peaks of performance in a new environment. The workers who would be better suited for this new environment might simply not be left in the company.

On the face of it, then, reprofiling might not be enough to solve the problem of local optimums. I would like to note three premises for this effect to be relevant: [START_REF] Chien | Data mining to improve personnel selection and enhance human capital: A case study in high-technology industry[END_REF] profiling and reprofiling is made on a closed group, as in not including a larger part of a specific market environment, (2) that using profile P1 actually makes the pattern P2 have lower p-values, and stronger correlations with performance measures, and (3) that the analysis is not focusing on specific, high performing outliers, but rather use the profile for generalized big cuts of applicants, like Xerox seems to use it (see subsection 2.1). Otherwise the theorized effect would not be there.

Human Intuition and our Predictive Mind

As already has been touched upon in the introduction, the problem with profiling and reprofiling might not be too different from what we initially wanted to get away from; the human, intuitive way of reasoning and categorizing. The data could be biased, implicitly towards certain categorizations. What I want to propose here is that profiling and trying to predict future high performance might be even more directly related to human cognition. An explanatory theory within neuroscience that has gained momentum lately propose that the brain essentially should be seen as a hypothesis-testing mechanism, that performs a kind of statistical analysis within the neurons [START_REF] Hohwy | The Predictive Mind[END_REF]. From this perspective, the main goal is for the brain to be able to predict what will happen, before it happens. Within the connections between the neurons there is both a forward progressive movement; receiving input from nerve-endings, for example our eyes, to be processed in succession in the neural cortex. But there is also a backward progression, where signals are constantly being sent backwards, from highercognitive areas of the brain, towards lower. This has previously been explained as a kind of feedback-loop, but lately the backward progression has been given more emphasis and use to explained human cognition the other way around.

Rather than starting blank and use input to create a model of the surroundings, the human brain starts with a working model of the world that is projected backwards within our connective network; and ultimately onto the world around us. The signal input functions as an error-correction function to adjust our working model of the world, where it is needed in order to explain what is happening in a specific moment. Accordingly, no corrections of our model will be made, as long the model can explain what is going on around us.

Studies also show that the kinds of categorizations we do to learn about the world are very much like statistical analyses. We more or less make a mirrored casual structure of the world in our brain, based on what we have experienced [START_REF] Hohwy | The Predictive Mind[END_REF].

All this can be related to something called confirmation bias, which means that humans have a tendency to direct attention, unconsciously, towards aspects which confirm our prior beliefs. In relation to the theory of our predictive mind, this would mean that in a backward loop we project a working model of the surroundings, and the attention is drawn towards that which confirms this model to be true. In a sense then, our prior beliefs and categorizations made from experience, determines what kind of information we will look for when facing a new situation; that which confirm what we already know, or believe that we know. Only when there are too much obvious errors, we start to look for a different explanation, and correct the working model. It is then also important to note that often there is insufficient feedback to correct our model when it comes to valuation of people, or groups of people, that we encounter. In the absence of sufficient errors, the working model will be reinforced; as functional to predict the outcome.

In some sense this can be related to profiling and reprofiling. After making a categorization, like for a profile, it will direct attention only to the feature you know they are related to. Even when attempting to reassess that knowledge, as in if you reprofile a population that has been modified by the previous profile, it is likely that you will end up looking at the same features. These features have then also become statistically stronger, as was hypothesized in the previous subsection and that will further reinforce your prior belief. A risk with human intuition is that prior beliefs dominates, controls your attention, and determines your future beliefs; the same risk can be found with reprofiling in data mining.

Revisiting implicit bias. What seems to happen with the described function of our predictive mind is essentially what is happening with implicit biases; namely an attempt for the brain to predict future outcomes based on previous experience and working models. As already mentioned, recruiters seem to try to predict performance of applicant, solely by looking at their names (see subsection 1.1, [START_REF] Bertrand | Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination[END_REF]). Implicitly, we seem to have working models with rough estimations, based on categorizations of people.

The delicate problem of this can perhaps be emphasized more clearly with an example of gender difference in math-skill. SAT-results in USA from 2013 show that male students performed better; an average of 531 (SD = 121) for male students, versus 499 (SD = 114) for female students (p<.001) [START_REF] Cummins | Why the Gender Difference on SAT Math Doesn't Matter[END_REF]. However, given the standard deviations, and normal distribution around the mean, 38% of female participants outscored 50% of the male participants (this is illustrated in Fig. 1). [START_REF] Cummins | Why the Gender Difference on SAT Math Doesn't Matter[END_REF]. Even if there is a significant difference in average score between genders (p<.001), 38% female student will still outperform the average male participant, statistically. This is represented by the black part of the graph.

Fig 1. Division of SAT math-test scores of 2013 in US

Faced with one male and one female in front of us, and the task of selecting one we deem most likely be best in math; then the difference in score is not very informative. There is almost 40% chances to be wrong if choosing the one associated to the group with the highest mean. That is not very good odds, considering that 50% is pure chance. Despite this, most people will feel confidence in their intuition in cases like this, and gender based stereotyping and over-categorization will therefore remain. If this is anything like implicit biases it will also be difficult to get direct feedback on this implicit bias to adjust our belief, to at least not be as certain in our categorization.

This is ultimately a limitation of statistical significance measures, that even if you have as strong of a significance as you basically can get (p<.001), you end up with a "meaningless" knowledge. This has, of course, also been recognized by others , and a way to counter that limitation is to look for statistical power, or importance; more specifically to include a measure of effect size [START_REF] Kelley | On effect size[END_REF][START_REF] Sedlmeier | Do studies of statistical power have an effect on the power of studies?[END_REF][START_REF] Smaldino | The natural selection of bad science[END_REF].

Analyzing effect size, the gender difference shown in the SAT math-test can be concluded to represent only about 3% of the difference attributed to gender, and the other 97% attributed to other factors (Choen's d = .37, effect size r-squared = .03) [START_REF] Cummins | Why the Gender Difference on SAT Math Doesn't Matter[END_REF]. This would be another way, a statistical way, to say something about how informative a significant difference is, as in the situation of choosing between two applicants of different gender based on average SAT-math score.

Revisiting the customer service profile. Making an analogy between the example of SAT-math score above, and the profile measures from Xerox, it may be significantly more likely that applicants that live closer to work will on average perform better, and stay longer at the company. However, some applicants living further away might be just as likely to perform well at the job. Nonetheless, if all other parameters are the same, the one who live further away will likely not be presented to the managers that hire the new customer service personnel.

Possible Solutions

What I have argued for so far are potential risks for using data profiling when hiring. I have also tried to show that these are not necessarily any new risks, but can be seen in any use of profiles. It even seems to relate to the way our human intuition categorizes from experience; we try to predict future outcomes just like algorithms.

There may be different ways to try to attack the problems highlighted in the previous section; i.e. local optimums and stagnation, discrimination and unfairness. I will recapitulate some previously discussed approaches, mainly; (1) create better and more accurate algorithms and technology, and (2) introduce more extensive laws and regulations. I will add to the discussion a third approach: (3) promote better thinking for the individuals who, at some stage or another, still will be involved in the decision and selection of applicants. They are not intended to be mutually exclusive measures; on the contrary, all could contribute and be equally as important for a sustainable solution.

Solving it with technology and better algorithms

An initial concept to consider is that of overfitting in machine learning. It is a term related to "fitting a model" to a training set of data, as in trying to explain the dataset, its relations, and its values. In overfitting, the statistical model has included irrelevant features, random error, or just noise, rather than only valid relationships and correlations. Consider the SAT scores discussed above; adding an analysis of effect size reduced the importance of the significant result of gender difference. You could do the same analysis of statistical power, such as looking at effect size, on a model in machine learning. What you can do after that is something called "pruning", which means, analogically, to cut off the branches of the decision tree that lack predictive power (like gender difference in the above case), which will improve the overall predictive power of the model (or pattern).

There is reason to be a bit skeptical whether this is a solution that will be implemented in practice. In psychology research, a standing advice for 25 years has been to include analyses such as effect size. Sedlemeir and Gigerenzer [START_REF] Sedlmeier | Do studies of statistical power have an effect on the power of studies?[END_REF] could conclude that only 2 out of 64 experiments even mentioned power and effect size, and instead solely relied on statistical significance. Another meta-analysis concludes that statistical power has not improved significantly in the 60 years since the concept was introduced [START_REF] Smaldino | The natural selection of bad science[END_REF]. Thus, it is likely even more difficult to get employers and companies in the job market to use measures of statistical power instead of relying on (simple) significantly correlated relationships.

Including Diversity. Another way of solving the problem of discrimination could perhaps be to directly include diversity. Potentially this could also solve problems of stagnating at local maximums. If conditions in the environment change, a diversity of skills in the workforce would make a company more likely to be able to adapt. But is diversity enough to remain innovative and adaptive? Some research seems to suggest just that, where one study finds a correlation between occupational diversity and likelihood of innovative research [START_REF] Söllner | Human capital diversity and product innovation: a micro-level analysis[END_REF], and another study finds that openness to cultural diversity was correlated with the likelihood for a company's continued economic performance, which was interpreted as being able to renew itself and to innovate [START_REF] Østergaard | Does a different view create something new? The effect of employee diversity on innovation[END_REF].

Whether it would be a solution against discrimination might depend on in what categories, and how, you create diversity. However, you could at least try to account for actual differences, rather than overgeneralize. For example, let's say we have a specific task to find someone solely based on math skill. Related to the example of gender differences in SAT-math score in subsection 3.4, implementing diversity could be to, at least, have result distributions similar to the test scores. In other words, you would want to see about 38% female and 62% male candidates, instead of over-generalizing to only look for male applicants. Just any man will not be the most likely best mathematician.

The same thing, then, could be applied to the relation between academic degree and customer service workers within Xerox (see subsection 2.1). Without having specific numbers or distributions available, but based on the assumption that these will follow normal distributions, you would want to, at least, see a similar spread and diversity in the applicants that you accept. Those with bachelor's degree will be more in numbers, but not exclusively. This could also be a possible way to open up for equal opportunities for people that are just as likely to perform well within a certain occupation, that otherwise would not have gotten the chance. In a very direct way the purpose can be said to be to not limit uniqueness too much in your data as well as with individuals, when including diversity.

If this is the case, that diversity is wanted, you could ask yourself why profiling at all. Why data profiling at all, if it is good to have a diverse workforce instead of one with certain, highlighted, high performing features? To some extent it might be a question of balancing between long-term and short-term goals. On one hand, the data probably doesn't lie, and if a company adopts the profile they get from current data, the company would probably perform well in the near future. On the other hand, it might become difficult to adapt to changing conditions, and will not perform as well long-term.

Including anonymity. Another technical solution to unfairness in selection processes could perhaps be to make applicants anonymous. This is an often proposed remedy for discrimination; to anonymize resumes [START_REF] Hutchison | Women in Philosophy: What Needs to Change?[END_REF]. Technically, it could for example be to use k-anonymity [START_REF] Lefevre | Incognito: Efficient full-domain kanonymity[END_REF]; that a person is anonymized with k-other people. It could mean that applicants are anonymized with all the other (k-amount) applicants for the position; as in, removing all personally identifiable data, like names and address. It remains a problem that smart algorithms can correlate impersonal data to other databases, and still figure out who is who, but this remains a separate problem.

Anonymizing could then be a solution to unfairness, to not include categorizes of people that are laden with pre-conceptions in the evaluation and prediction of future performance. However, it seems less of a solution for some of the problems discussed so far in this paper. Like the use of academic degree, and other data that inevitably must the included in a resume and CV.

This can be discussed further; what should be considered as merits for a position? Basic qualifications should be met, of course, but should it always be in terms of an academic degree for example? Working with telephone marketing, or customer support, may have a performance measure related to academic degree, but is it really necessary? This is a bigger discussion that does not fit within the limited space for this paper. More importantly, discussed below is the use of laws that effectively could function as anonymization if the use of personal data is prohibited.

Solving it by Law

The skepticism mentioned above, regarding wether companies would really implement certain solutions, could perhaps simply be regulated by law. Currently, laws are not proficient to hinder (potentially) harmful profiling. Barocas and Selbst [START_REF] Barocas | Big Data's Disparate Impact[END_REF], for example, examines anti-discrimination laws in the US, and conclude there is limited support if there is no (conscious) intent involved. The "Equal Employment Opportunity Commission Uniform Guidelines", as well, does not seem to be able to hinder such analyses. Rather they seem to explicitly allow the necessity for predicting future outcomes of recruitment and employment; very much what profiling on the surface is doing [START_REF] Barocas | Big Data's Disparate Impact[END_REF].

Crawford [START_REF] Crawford | Big data and due process: Toward a framework to redress predictive privacy harms[END_REF] also notes that Big Data systems seem to have similar problems as many governmental administration systems have; like the lack of notification for affected individuals. The proposed solution is to, at least, notify an individual potentially affected to "predictive privacy harms" [START_REF] Crawford | Big data and due process: Toward a framework to redress predictive privacy harms[END_REF]. And if they do not agree with the use of their personal data, individuals should be able to retract it; also from the predictive algorithms that does not affect them personally.

In just a couple of years, 2018, EU will incorporate a new set of rules; the "General Data Protection Regulation" (GDPR). In it, there is a specific section (Article 22) for decision-making of individuals, including profiling. Data processing is defined as profiling when "it involves (a) automated processing of personal data; and (b) using that personal data to evaluate certain personal aspects relating to a natural person" [START_REF] Heimas | Top 10 operational impacts of the GDPR: Part 5 -Profiling[END_REF]. In other words, it does not have to be yourself that the usage of your personal data affects. This would often be the case in the examples used in this paper, with personal data from previous applicant and employees that is applied on new applicants. Personal data is defined as any data that relates to a person's private, professional or public life; like a name, photo, posts on social networks, or your computer IP address.

This seems to be quite extensive, and my analysis will not go into detail of how effective they will be put into practice. I will merely note that there is an exception clause if you give consent to its use. You are still able to retract it afterwards, but on the face of it, there seems to be an opt-out implementation; as in, only those that actively opt to restrict their use of personal data, like in a consent form, will hinder the use. It is not likely a majority of people will opt-out of this kind of use, and there for there will probably still be much data to be data mined. The transparency of when you have been affected by an algorithm-based decision, is not straight forward; unless, perhaps, if you do not also adopt the notification principles as Crawford [START_REF] Crawford | Big data and due process: Toward a framework to redress predictive privacy harms[END_REF] proposed above.

Promoting better thinking

The last approach would be to more explicitly look at the human side of the decision process. Will regulations and technical opportunities be enough to guide human behavior, towards fair and non-discriminatory behavior? Laws and technology can create restrictions and constraints for what kind of behavior is likely to be performed, but if profiling in an unrestricted fashion remains profitable, it is likely many will find a way through the blockades, so to speak. It would demand further analysis if it would be enough to change recruiter's and manager's behavior, and at this point I will merely remain skeptical. This skepticism can neatly bring us to the main argument in this section. What recruiters and managers often seek when hiring, is support in their decision; they want to be more certain of their choice [START_REF] Chien | Data mining to improve personnel selection and enhance human capital: A case study in high-technology industry[END_REF][START_REF] Barocas | Big Data's Disparate Impact[END_REF][START_REF] Mehrabad | The development of an expert system for effective selection and appointment of the jobs applicants in human resource management[END_REF]. Similarly the brain wants to be more certain of the future, predicting what comes next. As was touched upon in the previous section, the way our brain functions in this regard may also lead us a bit astray at times, leading to implicit biases and negatively value laden pre-conceptions. What might be needed, then, is a bit more uncertainty, and a motivation for this can be found in the methodology of how good science is performed.

A key conception in modern empirical science is to not try to confirm what you believe to be true, but to falsify it. This relates very much to our confirmation biases and our tendency to look for that which confirms our prior beliefs and current working model in our brain (see subsection 3.5 for details). What the methodology of science does is, instead, to critique and to doubt, and try to disprove what we believe to be true. As Kothari puts it: "All progress is born of inquiry. Doubt is often better than overconfidence, for it leads to inquiry, and inquiry leads to invention" [START_REF] Kothari | Research methodology: methods & techniques[END_REF]. A similar message is delivered by Nobel Prize awarded physicist Richard Feynman:

"We have found it of paramount importance that in order to progress we must recognize our ignorance and leave room for doubt. Scientific knowledge is a body of statements of varying degrees of certaintysome most unsure, some nearly sure, but none absolutely certain." [START_REF] Feynman | The pleasure of finding things out: the best short works of Richard P. Feynman[END_REF].

So, rather than certainty, what you would like for scientifically more sound knowledge, is a promotion of uncertainty; or varied degrees of certainty and uncertainty, based on empirical data and arguments. What a scientific approach to data mining for applicant prediction could mean is that you as a recruiter should actively try to disprove your own hypothesis regarding who you believe is the best candidate for the position.

Conclusions

The main purpose of this paper is to present an initial, explorative investigation of ethical aspects within recruitment and hiring using Big Data and data profiling. It is found that the use of data mining in companies is often motivated as an attempt to increase objectivity in regards to performance predictions. As far as has been analyzed here, there seems to be some potential problems that companies risk running into. The first is to end up on low peaks of local maximums and therefore miss out on achieving long-term performance gains and resilience against changing conditions. The second is a concern for discrimination when using data mining algorithm. The third and final, and perhaps a sometimes forgotten focus, is a concern for loss of equal opportunities and unfairness for applicants in a selection process.

The problems discussed here are not new, in two ways. First of all, the problem of biases in data mining and data sets is well known, and even more specifically, implicit bias has been shown to exist in the use of data mining [START_REF] Barocas | Big Data's Disparate Impact[END_REF][START_REF] Caliskan-Islam | Semantics derived automatically from language corpora necessarily contain human biases[END_REF]. Secondly, problems with discrimination and (human) biases are well known and studied, and as I propose in this paper, these phenomena seem to be related in more intricate ways than perhaps often considered. Once again relating to the prediction-error theory of our mind (see subsection 3.5), data mining and profiling is an attempt to predict the future, in a similar way as the brain is trying to predict the immediate surrounding and what will come next. An inherent limitation in data mining and the ability to predict the future is that it is based on historical data. Similarly, the brain's predictions are based on historical experiences, and the brain can also be understood to do a kind of statistical analysis, leading to estimations and generalizations. This makes the brain prone to faulty predictions and discriminatory categorizations of people, and similar effects can be seen in statistical methods of data mining.

To continue on that analogy, it could be seen as if (big) data profiling, purely by its statistical method, can have implicit biases. Alternatively, it can reinforce the idea that humans through the perception of the world are (imperfect) statistical machines.

In this paper I have briefly touched upon some previously discussed ways to remedy these limitations in data mining and machine learning; through technological means (see subsection 4.1), or by laws and regulations (see subsection 4.2). A third option proposed here is to look at the quality of the human thinker and decision maker, and more precisely what could be the remedy in the scenarios of discussed with recruiters and managers. If it is better knowledge in our predictions we want, the proposition is to look at scientific methodology (see subsection 4.3). In that case, rather than looking for objectivity and certainty in decisions and selections of applicants, we should look for uncertainty, and doubt, and try to falsify our initial judgements of applicants ability to perform well.

How this openness to uncertainty, and a varied degree of certainty, is not either so clear. These are abilities that can relate to critical thinking skills, and how this is taught and improved is still an open problem without a clear answer. Even if we are able to define such a skill-set, and how to promote them using the predictive datamining tools recruiters can use, it is an inherent difficulty why this would ever be implemented by companies doing the hiring. In the environment of big companies using Big Data, a common understanding often heard is that "all you need is correlations", with no need to go beyond that understanding (Roger Clarke in lecture at IFIP 2016, Aug 21st). As has been shown in this paper, using only significance and p-values as statistical mean, you might end up with limited knowledge, and at times knowledge that breeds implicit biases (see subsection 3.5).

To conclude; what might be needed to promote a more conscious use of predictive tools, then, is a mix of technology and laws that is grounded in an understanding of how human decision-making and prediction-making is made.

Further Investigations

As was mentioned in the methodological subsection (1.2), there are currently some obstacles in doing research in the field of recruitment using data mining algorithms. Presently, companies and agencies do not have to disclose what algorithms they use, and how it affects populations [START_REF] Pasquale | The black box society: the secret algorithms that control money and information[END_REF]. This paper remains mostly theoretical. To get more insight into the effects on individuals and society, researchers would need access to more information about what kind of algorithms are in use, and also data from populations (applicants) it is applied on. Some of the things argued for in this paper could be simulated and tested. For example, the effect argued for with reprofiling; that statistical patterns would become more significant and with lower p-value, and consequently result in less deviation from means for individual features. Another effect to expect if lots of data points are added is that p-value will be low.

Finally, the discussion on what constitutes discrimination and unfairness would serve to be expanded. What counts as merits? When is it ok to use predictive measurements? Should they never be allowed? If they are statistical estimations and generalizations, they would inevitably lead to excluding some individuals. Should profiling be allowed if the statistical measure is good enough, and what constitutes good enough? These are some questions that do not seem to have clear and immediate answers.

Acknowledgements. I would like thank my supervisors Mikael Laaksoharju and Iordanis Kavathatzopoulos of Uppsala University, for extensive feedback and conceptual contributions.


Uchechi Nwadike

Thomas Groß

Kovila P L Coopamootoo

Evaluating Users' Affect States: Towards a Study on Privacy Concerns

Keywords: privacy concerns, affect states, PANAS-X, Facereader, Emotion Recognition, psycho-physiological

établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Users' concern over the safety of their personal details has been a long-standing issue in privacy research. To evaluate users' privacy concerns, the methodologies offered have been based on self-report and give a subjective account of users' privacy concerns. These were mostly questionnaires ranging from Westin's Privacy Segmentation Index [START_REF] Kumaraguru | Privacy indexes: A survey of westin's studies[END_REF] that group individuals in broad categorizations of privacy fundamentalists, pragmatists or unconcerned to those more focused on online privacy such as the Internet Users' Information Privacy Concerns [START_REF] Malhotra | Internet users' information privacy concerns (iuipc): The construct, the scale, and a causal model[END_REF].

First, Preibusch [START_REF] Preibusch | Guide to measuring privacy concern: Review of survey and observational instruments[END_REF] observed that evaluation of these measurement instruments and the methodology at large has been "fragmented and ad-hoc." We take this as a call to action to invest in the validation of tools for privacy research, especially those suitable to support evidence-based contributions. Second, users' privacy concern, intention and subsequent behavior, at the time of evaluation, is under the influence of their current internal states. We believe that eliciting affect states would provide an important dimension that impact privacy concerns. We therefore set out to investigate the influence of users' affect states on privacy concerns.

We report on a pretest which evaluates users' affect states when exposed to standardized video stimuli for happiness and sadness. We investigate two facegeometry-based affect analysis tools (Facereader and Emotion Recognition) and evaluate their properties systematically as components for future experiments. We validate these instruments against the Positive and Negative Affect Schedule (PANAS-X) [START_REF] Watson | The panas-x: Manual for the positive and negative affect schedule -expanded form[END_REF], a well-vetted self-report questionnaire.

Contribution. Our pre-test findings indicated that the two psycho-physiological tools accurately measured the users' affect states. Our findings not only provide a valuable systematic comparison of the measurement tools, but also techniques for inducing and measuring affect states, beneficial for other researchers. We also provide re-usable building blocks that can be plugged into further research. In addition, to the best to our knowledge, this is the first study employing affect inducing and psycho-physiological tools in usable privacy research.

Outline. The paper is organized as follows: first, we provide background information on privacy and emotion; then present our research model and hypotheses. Next we report on the pretest experiment conducted and the results obtained. Subsequently we present the structured abstract for the main experiment. We conclude the paper by discussing the implications and limitations of our work.

Background

In this section, we begin with the issues associated with privacy definition, its multidimensional characteristics, then review existing literature on privacy concerns, and affect states with their measurement methods. Subsequently we describe the use of stimuli from affective psychology to induce emotions, and use of affect measurement as manipulation checks. In conclusion, we report on existing measurement instruments for privacy concerns.

Information Privacy

Nissenbaum [START_REF] Nissenbaum | Privacy in Context: Technology, Policy, and the Integrity of Social Life[END_REF] proposed that privacy is a contextual concept that occurs in different spheres of life: legal, medical, information technology to mention a few. This has led scholars to propose different definitions: starting from "the right of an individual to be left alone" [START_REF] Warren | The right to privacy[END_REF] to "the claim of individuals, groups, or institutions to determine for themselves when, how, and to what extent information about them is communicated to others" [START_REF] Westin | Privacy and freedom[END_REF] amongst other privacy definitions.

Privacy Definition. Burgoon et al., Clarke,and DeCew [4,[START_REF] Clarke | What's privacy[END_REF][START_REF] Decew | In pursuit of privacy: Law, ethics, and the rise of technology[END_REF] are known for their multidimensional definitions of privacy. For the purpose of this paper we adopt the definition by Smith et al. [START_REF] Smith | Information privacy: measuring individuals' concerns about organizational practices[END_REF] as stated in Li [START_REF] Li | Empirical studies on online information privacy concerns: literature review and an integrative framework[END_REF], information privacy refers "to the ability of individuals to personally control information about themselves." Information privacy enables individuals or groups or organizations to protect themselves against actual or perceived intrusions on the information about them [START_REF] Westin | Privacy and freedom[END_REF][START_REF] Decew | In pursuit of privacy: Law, ethics, and the rise of technology[END_REF]. The possible occurrence of privacy intrusions can trigger a sense of panic or anxiety in users. This causes them to express their concerns over maintaining adequate access protection to their personal details.

Privacy Concerns. Privacy concerns can be described as "concerns about possible loss of privacy as a result of information disclosure to an online business" [START_REF] Xu | Examining the formation of individual's privacy concerns: Toward an integrative view[END_REF]. Scholars rely on the users' expression of their privacy concerns to measure the privacy levels and classify users" [START_REF] Westin | Privacy and freedom[END_REF]. Given the multidimensional properties of privacy, it is not a surprise that different survey tools have been developed for measuring privacy concerns. Some of the survey tools which adopted the use of a multidimensional approach in measuring privacy have been considered as validated and reliable. These include Concerns for Information Privacy (CFIP), Internet Users Information Privacy Concerns (IUIPC) [START_REF] Malhotra | Internet users' information privacy concerns (iuipc): The construct, the scale, and a causal model[END_REF][START_REF] Smith | Information privacy: measuring individuals' concerns about organizational practices[END_REF]. These are widely used as the standard surveys for privacy concerns.

Measuring Privacy Concerns. The development and use of differing scales have not been without issues. In his overview of the existing survey instruments used in measuring privacy concerns, Preibusch pointed out that "approaches to measure privacy concerns are fragmented and often ad-hoc, at the detriment of reliable results" [START_REF] Preibusch | Guide to measuring privacy concern: Review of survey and observational instruments[END_REF]. The survey results derived from these tools rely on users' feedback, memories, and rated perception of subjective factors considered to affect privacy concerns, [START_REF] Hall | Psychophysiological assessment tools for evaluation of learning technologies[END_REF].

Emotion and Affect.

In this section, first we present the definition of emotion,affect, and highlight their differences. We highlight the differing views on the relationship between emotion and behavior, followed by a brief overview on the effect of emotion on behaviors, concerns, decision making.

We adopt Baumeister's description of emotion as "a conscious feeling state". It is stimulated either by actual events that happen to the individual ("actions") or anticipated events that are yet to occur ("outcomes").

Emotion has been classified based on the duration of the feeling state [START_REF] Slovic | Affect, risk, and decision making[END_REF]. Affect has been described as the "faint whisper of emotion" [START_REF] Slovic | Affect, risk, and decision making[END_REF]. Affect is said to have more impact on behavior than emotion [START_REF] Baumeister | How emotion shapes behavior: Feedback, anticipation, and reflection, rather than direct causation[END_REF][START_REF] Nyshadham | Affect and online privacy concerns[END_REF]. Hence in this paper, we use the term affect state to describe the state of feelings experienced by the participants. This is because the stimuli films can trigger a type of quick reaction within the individual.

The sole use of surveys as the main measurement tool of a multidimensional concept like privacy is inadequate. This is in line with the findings by Paine et al., which point out that "the concept of privacy is highly complex, therefore it is unlikely that surveys can accurately reflect respondents' true concerns" [START_REF] Paine | Internet users' perceptions of 'privacy concerns' and 'privacy actions[END_REF]. We suggest the use of a complimentary set of survey and psycho-physiological tools such as facial and emotion recognition devices. We believe users' privacy concerns, are associated with non verbal expressions, which are unconscious, facially expressed and fleeting in nature [START_REF] Ekman | An argument for basic emotions[END_REF]. They cannot be captured by survey tools, hence the need for psycho-physiological tools. Hall et al. [START_REF] Hall | Psychophysiological assessment tools for evaluation of learning technologies[END_REF] noted "psychophysiological measures are particularly sensitive to the fleeting and non-conscious nature of emotional experience." In this report we discuss the use of Facereader and Microsoft Emotion Recognition in the studies presented in this paper.

The literature review we conducted, revealed the contrasting views on the causative relationship between emotion and behavior. Loewenstein et al. [START_REF] Loewenstein | Risk as feelings[END_REF] suggests that "the idea that emotions exert a direct and powerful influence on behavior receives ample support in the psychological literature on emotions." In a similar vein, Frijda [START_REF] Frijda | The emoticons[END_REF] had suggested that "emotion arouses behavior and drives it forth." On the other hand, Baumeister et al. [START_REF] Baumeister | How emotion shapes behavior: Feedback, anticipation, and reflection, rather than direct causation[END_REF] suggest opposing views. In their review of the direct causation theory, they argue that "if a given emotion does not consistently cause same specific behavior, then again the influence of emotion on behavior can hardly be considered as direct." Rather they suggested that behavior is indirectly influenced by anticipated emotional outcomes.

Theory and Research model

The Theory of Planned Behavior (TPB) states that attitude, subjective norms and preconceived behavioral control have a direct influence on behavioral intention which in turn influences actual behavior. However scholars have argued that subjective norms are "inadequate and rarely predicts behavioral intentions" as stated in Armitage et al. [START_REF] Armitage | Efficacy of the theory of planned behaviour: A metaanalytic review[END_REF]. Researchers have also highlighted the inefficacy of the TPB to influence or predict behaviors especially in the health field, this can be extended to privacy research based on observed privacy paradox [START_REF] Sniehotta | Time to retire the theory of planned behaviour[END_REF]. Hence in our research model above, we investigate the influence of stimuli films, S, on users' affect states, and consequently investigate the influence of affect states on their privacy concerns. We recognize that confounding factors, F 1...n such as user's consumption of alcohol and recreational drugs could have an influence on the affect state. To test our research model, we first explore the influence of stimuli films on users affect states by carrying out a pilot study or pretest as it is referred to in this paper. We build on the outcomes of the pretest and present the design of an upcoming main experiment in Section 4.

Pretest

Affective psychology predicts that stimuli from films impact human affective states [START_REF] Schaefer | Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers[END_REF]. We designed a pretest study to assess and validate the manipulation from such stimuli and their measurement.

RQ1: Manipulation Method. How do standardized stimuli films (for happiness and sadness) influence the user's affect state?

H 1,0 : There is no change in users' happy and sad affect states under induced happy and sad stimuli films.

H 1,1 : Users' happy and sad affect states are impacted by induced standardized happy and sad stimuli films.

RQ2: Measurement Tools. We make a systematic comparison between the manipulation test in the validated PANAS-X questionnaire and the psychophysiological measurement tools. What are the tools' sensitivity, confidence intervals, their strengths and weaknesses? For the operationalization of the hypotheses, we define sensitivity as the effect size (in difference between means) between measuring the affect state of a participant exposed to a happy stimulus versus the affect state of the same participant exposed to a sad stimulus. We refer to the 95% confidence interval on the effect size.

H 2,0 : There is no difference in the sensitivity and confidence intervals on happiness and sadness measurements of the tools PANAS-X, Facereader and Emotion Recognition.

H 2,1 : The tools PANAS-X, Facereader and Emotion Recognition differ in either sensitivity or confidence interval when measuring happiness or sadness affect states.

Method

Participants were exposed to video stimuli to induce diametrical emotions in a within-subject design. They received a happiness as well as a sadness stimulus in random order. The observed affect was measured with PANAS-X [START_REF] Watson | The panas-x: Manual for the positive and negative affect schedule -expanded form[END_REF], Noldus FaceReader and Microsoft Emotion Recognition and compared across conditions.

Participants. N = 9 students from Computing Science Department of Newcastle University, of whom six males and three females, participated in the study. The participants' age range was from 23 to 30 years, (M = 26.43,SD = 2.23).

Operationalization. We induce the independent variable (IV) affect with three levels: (a) neutral baseline, (b) happy, and (c) sad.

We check this manipulation with a self-report questionnaire, a 60-item PANAS-X [START_REF] Watson | The panas-x: Manual for the positive and negative affect schedule -expanded form[END_REF] (joviality and sadness) with a designated time horizon "at this moment."

We measure the Dependent Variable(DV) affect (happiness and sadness) on a scale of [0, . . . , 1] with the psycho-physiological measurement tools (a) Facereader (FR) [START_REF] Den Uyl | The facereader: Online facial expression recognition[END_REF][START_REF] Brodny | Comparison of selected off-the-shelf solutions for emotion recognition based on facial expressions[END_REF], (b) Emotion Recognition (ER). During the stimulus exposure, a video of the participant's face is recorded. The video is inputed into FR; a still image of the face is taken at the end of the corresponding stimulus and inputed into ER.

Procedure. The pretest proceeded in the following steps, where Fig. 2 illustrates the main experiment design:

(a) a demographics questionnaire, (b) Neutral state.

-Induction of a neutral baseline affect state, -Measurement of manipulation check (PANAS-X), ER and FR. (c) Affect State 1: Either happy or sad, determined by random assignment.

-Show video stimulus to induce affect.

-Measurement of manipulation check (PANAS-X), ER and FR. (d) Affect State 2: Complement of Affect State 1.

-Show video stimulus to induce affect.

-Measurement of manipulation check (PANAS-X), ER and FR. (e) a debriefing check, which checks for the participants feedback regarding the affect state experienced.

Measurement (DV)

Manipulation (IV)

Elicited Emotion

Watching happy and sad stimulus films.

Manipulation Check

60-Item PANAS-X FR Scores ER Scores

Self-report on positive and negative affects.

Psycho-phsyiological measure of affect (face geometry).

Fig. 2: Experiment design template for the pretest.

Inducing & Measuring Affect State. We adopted the induction of happy and sad from standardized stimuli defined in Gross et al. [START_REF] Gross | Emotion elicitation using films[END_REF] For the induction of happiness, and sadness affect states, we used the restaurant scene from the movie When Harry meets Sally and the dying scene from the movie The Champ as stimuli films. Participants were exposed to both stimuli films clips in a withinsubject experiment. Whether they received the happy or the sad film first was determined by random block assignment. After the neutral state and after each film participants filled a full 60-item PANAS-X questionnaire, with a designated time horizon of "at this moment."

During the neutral state and during watching each film, the faces of the participants were filmed with a high-resolution video camera. The video feeds constituted the inputs for the Facereader, which computed affect scores based on changes in face geometry. At the end of the stimulus exposure, a still image is taken from the video feed, which serves as input to Emotion Recognition. Both Facereader and Emotion Recognition compute scores on the scale [0, . . . , 1] for the variables anger, contempt, disgust, fear, happiness, neutral state, sadness and surprise. Only happiness and sadness were considered for further analysis.

Results

Fig. 3 contains an overview of the results, in which we have normalized PANAS-X to the interval [0, . . . , 1] to put all tools on the same scale. All inferential statistics are computed with two-tailed tests and at an alpha level of .05. We report asymptotic significance values.

Assumptions. We tested the the normality of the measurements from PANAS-X, ER and FR towards the eligibility of parametric statistics. The Shapiro-Wilk test was statistically significant for PANAS-X Sadness, all Emotion Recognition and Facereader measurements (all p < .001). The PANAS-X Joviality results were borderline, W = 0.92, p = .087. Consequently we are not entitled to use parametric tests and opt for a two-tailed Wilcoxon signed-rank test.

Manipulation Check: PANAS-X. A self report-based manipulation check was carried out. We used the 60-item full PANAS-X questionnaire [START_REF] Watson | The panas-x: Manual for the positive and negative affect schedule -expanded form[END_REF] as manipulation check on the induced affect state, following the methodology endorsed by Rottenberg et al. [START_REF] Ray | Emotion elicitation using films[END_REF]. Of the different variables PANAS-X provides, we focus on sadness and joviality as equivalent of happiness.

There is a statistically significant difference between both videos stimuli for both measurements on joviality and sadness measurements. We offer a comparison of PANAS-X results for both stimuli in Table 1a. Consequently, we reject the null hypothesis H 1,0 and accept that the video stimuli have a measurable impact.

Emotion Recognition. We observe with the measurements of the Emotion Recognition tool that there are statistically significant differences between the stimulus conditions, for happiness measurements as well as sadness measurements. Table 1b contains an overview of the ER results. This informs RQ2 that ER is a suitable measurement tool for affect comparisons with small samples.

Facereader. The Facereader measurements across video stimuli were neither statistically significant for the happiness nor for the sadness measurements. Table 1c contains an overview of the FR results. This informs RQ2 in that Facereader-based measurements do not have sufficient power to differentiate between these emotions at the small sample size of the pretest.

Comparison of Measurement Tools.

One of the key outcomes of the pretest is a systematic comparison of the measurement tools (PANAS-X, FR and ER) while ascertaining the overall effectiveness of the induction of emotions with standardized video stimuli.

Qualitative. We first make qualitative observations based on the boxplot comparison in Fig. 3. We are aware that we had one participant who entered the experiment in a morose state, which shows as an out-lier throughout the measurements. We observe that PANAS-X provides a clear distinction between happiness and sadness stimuli in both measurements. As one can expect from a standardized and validated measurement instrument for affect, PANAS-X can be considered a sound benchmark. Emotion Recognition (ER) offers a precise recognition of happiness. While it was able to distinguish the stimuli on the sadness measurement, as well, this difference was less pronounced.

Facereader (FR) recognized happiness in face of a participant exposed to a happy stimulus, however, FR does not use the full scale, reporting a Mdn ∼ .3. The result of the FR sadness measurement is striking in that it only uses < .025 of the scale [0, . . . , 1]. At the same time, the interquartile range is closely bracketed.

Meta Analysis. We compare the standardized mean differences for measuring either happiness or sadness across happy or sad stimuli. Fig. 4 summarizes the outcome of this comparison in a meta-analysis forest plot. The meta-analysis was computed with the R package metafor [START_REF] Viechtbauer | Conducting meta-analyses in r with the metafor package[END_REF].

Let us consider the left-hand-side Fig. 4a, which contains measurements of happiness with the three tools in question. For each measurement tool, we computed the mean difference between the happy video stimulus and the sad video stimulus, standardized over the joint standard deviation of the respective tool's measurements.

For happiness scores, we see that all tools measure positive difference (i.e., a higher mean happiness in the case of the happy video vs. the case of the sad video). We observe that FR has the smallest mean difference, which can be interpreted as being least sensitive to measuring happiness differences. We observe further that ER shows the greatest mean difference, however is also impacted by the greatest confidence interval. The line "FE Model" below the three measurement tools offers a combined fixed-effect model of all three measurements, which informs us how our strongly happiness induced by the given videos registers in our measurement apparatus. This model is weighted by the standard deviations of the respective tools. Finally, we expect to measure happiness with a standardized mean difference of about 1 SD, which is a large effect.

The right-hand-side Fig. 4b compares the results for sadness measurements. All tools measure a negative difference (i.e., a lower mean sadness in case of the happy video vs. the case of the sad video). We notice that PANAS-X even though observing the greatest mean difference also comes with the greatest confidence interval. Again, FR reports a lower mean difference than ER. Overall, the combined fixed-effect model shows a standardized mean difference of -0.72, also a large effect.

In conclusion, we observe that all three measurement tools have picked up happiness and sadness as expected from the video stimuli. Consequently, we know that the video stimuli work for inducing the emotions happiness and sadness, resulting in a medium to large effect size depending on the measurement device. This answers RQ1 and gives evidence to reject the null hypothesis H 1,0 . FR as well as ER worked as psycho-physiological measurement tools picking up the participant's emotional state without the interference of a self-report question- The position of the square dot determines the effect size, the diameter of the dot shows the weight, the whiskers the 95% confidence interval on it.

naire. ER obtained the largest effect sizes for the case of measuring happiness as well as sadness. FR obtained the lowest effect size of the field, especially in the case of measuring sadness. From these observations, we can answer RQ2 in terms of qualitative differences sensitivity and confidence intervals. However, these differences are not statistically significant, by which we will retain the null hypothesis H 2,0 .

Discussion

We answer the research questions as follows: For RQ1, we observe that the standardized video stimuli [START_REF] Ray | Emotion elicitation using films[END_REF] can indeed be employed to induce affects. Our manipulation check with PANAS-X shows large effect sizes in the differences between video stimuli conditions. Consequently, we can use video stimuli to establish experiment conditions for true experiments in privacy and identity management, such as the main experiment we design in Section 4. We thereby recommend to replicate existing manipulation apparatuses from affective psychology.

For RQ2, we observe that the different measurement tools at our disposal differ in sensitivity and confidence intervals even if the evaluation did not turn out to yield statistical significance. While the psycho-physiological measurements (ER and FR) both worked by and large, we observed weaknesses of FR in the measurement of sadness. In addition, the meta and power analyses will need to inform future experiment designs. In particular, FR had the least power to distinguish between happiness and sadness conditions, which directly translates to a higher required sample size.

The three measurement devices exhibit strengths and weaknesses which need to be taken into account in experiment design. PANAS-X has been validated and used frequently in psychology research. However, it is a self-report questionnaire, which takes about 10 minutes to fill in for the full 60-item version.

Consequently, we need to expect that emotional stimuli are wearing off over the time the questionnaire is answered. Even if the time horizon is set to "at this moment," the outcomes will not be as immediate as with psycho-physiological measurements. ER works on still images and can thereby be used to measure momentary affect of the user. However, then the decision which time instant to use for the measurement becomes crucial. FR operates on video streams and comes with the capability to track affects over time. This, however, comes at a cost of less sensitivity to distinguish between conditions.

Main Experiment

We took on board a comment from the IFIP workshop, which highlighted the necessity to assess user's privacy behavioral intentions whilst measuring privacy concerns. The reason given was privacy concerns questionnaires seem to be based on subjective norms, which are long term and not easily influenced. This was confirmed by a pretest we conducted on privacy concerns surveys and has led to the inclusion of a survey on behavioral intentions. The selected questionnaires are same as those used by Yang and Wang [START_REF] Yang | The influence of information sensitivity compensation on privacy concern and behavioral intention[END_REF]. A structured abstract of the upcoming experiment is presented in the next section.

RQ3 Impact of Affect on Privacy Concern. The upcoming experiment will investigate to what extent affect state causes differences in privacy concern. The research hypotheses being tested are:

H 3,0 : There is no difference in privacy concern scores between cases with induced happiness and induced fear.

H 3,1 : Privacy concern scores differ between cases of induced happiness and induced fear.

In particular, we hypothesize as a refinement of H 3,1 that users exhibit higher scores on privacy concerns when they feel fear than when they feel happiness. However, with H 3,1 we retain the capacity to evaluate two-tailed tests.

Method

A sample of N = 60 participants will be exposed to standardized video stimuli [START_REF] Gross | Emotion elicitation using films[END_REF][START_REF] Schaefer | Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers[END_REF] to induce emotions (happiness and fear ) in a within-subjects design.

The participants receive the video stimuli in random order. Privacy concern and behavioral intention scores will be measured and compared across video conditions.

Operationalization. We induce the independent variable (IV) affect with three levels: (a) neutral baseline, (b) happy, and (c) fearful.

We check this manipulation with self-report and psycho-physiological measurement tools: (a) 60-item PANAS-X [START_REF] Watson | The panas-x: Manual for the positive and negative affect schedule -expanded form[END_REF] (joviality and fear) with a designated time horizon "at this moment." (b) FR (happiness and fear), (c) ER (happiness and fear). For the manipulation check, a video of the participant's face is recorded during the stimulus exposure. The video stream serves as input for FR, a still image of said face-recording is taken at the end of the corresponding stimulus and used as input for ER. There is a time of 10 minutes allocated to fill in the PANAS-X after the stimulus exposure.

We measure the DV, the user's behavioral intent on privacy concerns, using the same self-report questionnaires used by Yang and Wang [START_REF] Malhotra | Internet users' information privacy concerns (iuipc): The construct, the scale, and a causal model[END_REF][START_REF] Smith | Information privacy: measuring individuals' concerns about organizational practices[END_REF], because they have been rigorously tested and found reliable [START_REF] Stewart | An empirical examination of the concern for information privacy instrument[END_REF].

Participants. The sample size of N = 60 is chosen following an a priori power analysis, informed by the pretest in Section 3. As one constraint, we have seen a minimum sample size of N = 39 for a within-subject experiment using the Wilcoxon signed-rank test to reach 95% power across the board. We choose a greater sample size, because we prepare for the use of a two-tailed test and expect a smaller effect size in the impact of affect on privacy concerns. With N = 60 we can expect a sensitivity of .49, a medium effect size.

Procedure. The main experiment is designed to enable a comparison of the influence of affect states on privacy concerns and privacy behavioral intentions. The study will be spread over two days; the first day will entail the participants carrying out the first three steps, i.e. (a)-(c). On the second day, the participants will first be induced to a neutral state and then complete steps (d) and (e). The reason for this is to minimize the carryover effects of the video stimuli and effect of "questionnaire fatigue."

The procedure consists of the following steps, where Fig. 5 illustrates the key elements of the experiment design: -Show video stimulus to induce affect.

-Manipulation check with PANAS-X, ER and FR, -DV questionnaire on privacy behavioral intentions. (e) a debriefing questionnaire, used to check for missed or misreported information, subjective thoughts during study session.

The analysis compares the DV privacy concern measurements across the main levels of the IV (happy and fearful), using a two-tailed matched-pairs Wilcoxon signed rank test.

Depending on the properties of the sample (e.g., normality, homogeneity of variances) further analysis of the impact of the IV on privacy concern as target variable is possible with Univariate Analysis of Variances (ANOVA/GLM) or Linear Regression.

Measurement (DV)

Manipulation (IV)

Elicited Emotion

Watching happy and fearful stimulus films. Randomly assigned per IV-level in within-subject design.

Fig. 5: Design for the main experiment.

Conclusion

While Wakefield [START_REF] Wakefield | The influence of user affect in online information disclosure[END_REF], and Nyshadham and Castano [START_REF] Nyshadham | Affect and online privacy concerns[END_REF] have explored the relationship between affect, information disclosure and online privacy concerns, we employ induced emotions and psycho-physiological tools in our empirical study of users' affect states. To the best of our knowledge, there is currently, no such endeavor in usable privacy research.

Our pretest results provide empirical evidence that the specific stimuli films used had significant influence on users' happiness and sadness. The pretest showed a successful manipulation of users' affect states. The pretest results also indicate that ER, FR, and PANAS-X can measure users' happiness and sadness, where ER is more sensitive in particular small sample sizes due to its large effect size.

Our pretest has therefore systematically evaluated and validated the tools for the upcoming main experiment. It further yields a detailed analysis of effect sizes and power of different psycho-physiological measurement tools that are of independent interest for usable privacy research. Other researchers can glean insights from the pretest results, use the tools employed her as validated components to induce or measure affects. Furthermore, with the design for the main experiment, we offer a template for true experiments that induce affect, control the manipulation tightly and then measure the impact on privacy concerns. The reported effect sizes and power calculations can form the basis for rigorous design for future experiments.

Fig. 1 :

 1 Fig. 1: Research model for the experiment.

Fig. 3 :

 3 Fig.3: Comparative boxplots for happiness and sadness measurements, with stimuli "happy" and "sad" on the x-axes. The y-axes are normalized to [0, . . . , 1].

Fig. 4 :

 4 Fig.4: Meta-Analysis forest plot of measurement tools across induced emotions. The position of the square dot determines the effect size, the diameter of the dot shows the weight, the whiskers the 95% confidence interval on it.

( a )

 a Completion of pre-task questionnaire on demographics, alcohol/recreational drug use, IUIPC and CFIP surveys. (b) Neutral state. -Induction of a neutral baseline affect state, -Manipulation check with PANAS-X, ER and FR, -DV questionnaires on privacy behavioral intentions. (c) Affect State 1: Either happy or fearful, determined by random assignment. -Show video stimulus to induce affect. -Manipulation check with PANAS-X, ER and FR, -DV questionnaire on privacy behavioral intentions. (d) Affect State 2: Complement of Affect State 1.

Table 1 :

 1 Overview of results for measurement devices PANAS-X, FR, and ER.

			(a) PANAS-X Results.
	Measure Stimulus Mdn	Z	p r
	Joviality Happy 21 -2.207 .027 .59 Sad 14
	Sadness Happy 5 Sad 10		-2.023 .043 .54
	(b) ER Results.					(c) FR Results.
	Measure Stimulus Mdn	Z	p r	Measure Stimulus Mdn	Z	p r
	Happiness Happy 0.95 -2.366 .018 .63 Sad 0.011	Happiness Happy 0.29 Sad 0.0008	-1.483 .14 .37
	Sadness Happy 2.37 -2.366 .018 .63 Sad 0.038	Sadness Happy < 0.00001 -1.483 .14 .37 Sad 0.0002