Vinicius Ferreira
email: vinicius.ferreira@dc.ufscar.br

Junia Anacleto

Andre Bueno
email: andre.obueno@dc.ufscar.br

Sharing Wishes on Public Displays: Using Technology to Create Social Places

Keywords: Self-expression, socialization, ICT interactive installation, art installation, public display in social spaces, WishBoard, thirdplaceness

We present a public-display-and-mobile-based digital art installation named WishBoard that translates the essence of a chalk-and-wall based art installation called 'Before I Die' into a technological communal expression. Our studies show that public sharing of personal wishes leverage the connection inside a community, creating a feeling of neutral ground for gathering, giving a sense of third place independent of physical constrains, named thirdplaceness. The installation allows people to freely express their wishes, with certain personalization, and provides a public visualization of all shared wishes promoting face-to-face conversations. We conducted two deployments using our installation in a 'socially abandoned' space. Analyzing the shared wishes content and people's behavior when observing the wishes popping up on the public displays, we were able to notice the essential role that public displays combined to mobile devices interaction can play in creating a community meeting place when providing a way for self-expression.

Introduction

Expression of thoughts, preferences, and feelings is a common and habitual practice among human beings. This expression reveals people's internal attributes, such as preferences, beliefs, and values. In Western culture, self-expression is valued as a powerful sign of individual freedom in which involves projecting one's own thoughts and ideas into the world [START_REF] Kim | Express yourself": culture and the effect of self-expression on choice[END_REF]. With the rise of social media, new possibilities of expression and communication are emerging. People can keep in touch with friends and family on-the-go through their mobile devices creating new social practices. However, supporting traditional social interaction (face-to-face) is still an important issue in order to support local communities and to avoid its fragmentation [START_REF] Brenny | Social Connectedness and Inclusion by Digital Augmentation in Public Spaces[END_REF]. Community fragmentation is a problem that has been increasing due to urbanization and decline of places outside work or family-based communication contexts. Such places, described as third places, provide the feeling of inclusiveness and belonging to a community. Coffee shops, pubs, and squares are typical examples of third places. These places have an important role in promoting quality of life for individuals reinforcing in their regulars the sense of community [START_REF] Oldenburg | The great good place: Cafes, coffee shops, bookstores, bars, hair salons, and other hangouts at the heart of a community[END_REF]. According to Sarason [START_REF] Sarason | The emergence of a conceptual center[END_REF], sense of community is "the perception of similarity to others, an acknowledged interdependence with others, a willingness to maintain this interdependence by giving to or doing for others what one expects from them, and the feeling that one is part of a larger dependable and stable structure". This sense has two dimensions: territorial and relation [START_REF] Mcmillan | Sense of community: A definition and theory[END_REF]. The territorial dimension has relation with the territory, shared space, and proximity. However, not all cases constitute a community, the relational dimension is also essential. The relational dimension includes factors of nature and quality of relationships.

In order to support such sense of community, we translated the essence of a chalkand-wall based art installation called Before I Die [START_REF] Chang | Before I Die[END_REF] that promotes self-expression through public sharing into a mobile-device-and-public-display based art installation named WishBoard. This public sharing can leverage the feeling of a neutral ground for gathering and, consequently, a sense of third place independent of place constrains, defined as thirdplaceness and discussed in section 3.

Several studies have used public displays and/or interactive art installations to design a space for self-expression and socialization. In these studies, they found that public displays and interactive art installations can foster civic engagement, place attachment and community awareness [START_REF] Chang | Before I Die[END_REF] [START_REF] Farnham | Measuring the impact of third place attachment on the adoption of a place-based community technology[END_REF][18] [START_REF] Martin | Engaging with a Situated Display Via Picture Messaging[END_REF][29] [START_REF] Salvador | Less cyber, more cafe: Enhancing existing small businesses across the digital divide with ICTs[END_REF]. Moreover, such spaces give people a place and a chance to expose their inner thoughts and feelings in order to express their individuality [START_REF] Brenny | Social Connectedness and Inclusion by Digital Augmentation in Public Spaces[END_REF].

In this context, our goal was to understand if a digital art installation, called Wish-Board, which allows self-expression, combining interaction between mobile devices and public displays, can support community awareness and thirdplaceness experience. This paper focus on the impact the system had on the sense of community by promoting thirdplaceness in a place that is not a third place, which it differs from others [START_REF] Farnham | Measuring the impact of third place attachment on the adoption of a place-based community technology[END_REF] [START_REF] Hosio | Social Surroundings: Bridging the Virtual and Physical Divide[END_REF] [START_REF] Mccarthy | Supporting community in third places with situated social software[END_REF]. To address this, we conducted an empirical study at a workplace to investigate the experience in-the-wild and observe cultural trails in anonymous posts shared. As a result, we believe that public sharing represents a promising model for mobile social collaboration in promoting and reinforcing the sense of community.

Self-expression supporting the sense of community

According to Kiesler & Sakumura [START_REF] Kiesler | A test of a model of commitment[END_REF] when people express their thoughts, using words, they may feel more committed to their thoughts and bound by them. They explain that expressing involves aspects of selfhood. Furthermore, the self-perception theory [START_REF] Bem | Self-perception theory[END_REF] suggests that observing their actions and behaviors is a key way for people to be aware of their own internal states. Designing tools for self-expression in public spaces has an important role for people in a community, remembering and celebrating their own culture [START_REF] Brenny | Social Connectedness and Inclusion by Digital Augmentation in Public Spaces[END_REF]. This public authoring enriches the space by sharing local information, knowledge and experiences [START_REF] Lane | Urban Tapestries: Public Authoring, Place and Mobility[END_REF]. In order to improve people's engagement with their community, interactive public displays and public art installations are mechanisms widely used. Several studies show that public displays can promote place attachment, community awareness, colocated interactions and technology-supported relationships. In addition, public dis-plays can foster face-to-face human interaction and encourage collaboration among community members [START_REF] Agren | Virtual community life: A disappearance to third places for social capital[END_REF][10] [13][18][28] [START_REF] Mccarthy | Supporting community in third places with situated social software[END_REF][32] [START_REF] Salvador | Less cyber, more cafe: Enhancing existing small businesses across the digital divide with ICTs[END_REF]. Furthermore, public art installations can give people the opportunity to express their individuality and aspirations with their community, improving or creating relationships between them [START_REF] Brenny | Social Connectedness and Inclusion by Digital Augmentation in Public Spaces[END_REF]. Regarding to design tools for self-expression, several studies have focused on elements for self-representation in online environments [START_REF] Brignull | Enticing People to Interact with Large Public Displays in Public Spaces[END_REF] [START_REF] Martin | Engaging with a Situated Display Via Picture Messaging[END_REF]. Although, more means to express this dynamic and innate nature of self-expressions in public spaces are essential.

In this context, we have used an interactive art installation that promotes selfexpression in a community exploring the interaction between mobile devices and public displays. In addition, this installation creates a space for people to freely socialize supporting the sense of community belonging and thirdplaceness experience, beyond architectural and temporal restrictions.

3

Beyond third places

Third places are places where people gather and enjoy each other's company. Oldenburg [START_REF] Oldenburg | The great good place: Cafes, coffee shops, bookstores, bars, hair salons, and other hangouts at the heart of a community[END_REF] describes third places as a means of 'keeping in touch with reality' promoting intimate personal ties outside the home (first place) and workplace (second place).

In order to be a third place, the place has to be a neutral ground where everyone is free to come and go, and also, welcome without caring about the individual's status in the society. There is lively conversation inside and it is easy to access. Moreover, there are regulars who shape and bring the place to life, attracting newcomers. The place has no extravagances and the mood inside is playful. Summarizing, it is a home away from home for their regulars [START_REF] Oldenburg | The great good place: Cafes, coffee shops, bookstores, bars, hair salons, and other hangouts at the heart of a community[END_REF]. Within these descriptions, ethnographic studies of virtual environments such as chat rooms, multi-user environments and bulletin board systems suggest those settings can often function as a third place [19][43]. However, these virtual environments differ from third places in respect of the 'realness' of the interaction or dependence upon simulation [START_REF] Turkle | Virtuality and its Discontents Searching for Community in Cyberspace[END_REF]. According to Doheny-Farina [START_REF] Doheny-Farina | The wired neighborhood[END_REF], a third place cannot exist separate from a locality. For him, this occurs for the reason that a 'third place exists only in comparison to its neighborhoods, to local work, play and family life, to the institutions and formal rituals that encompass daily life'. Perhaps as a reaction to the disappearance of third places in the physical world, people found in virtual environments their third place [START_REF] Agren | Virtual community life: A disappearance to third places for social capital[END_REF]. These virtual environments have provided opportunities for people with similar interest to meet both in cyberspace and in face-to-face fostering elements of third places [START_REF] Pasick | More than love to be found on networking sites[END_REF]. Nevertheless, both public places (such as third places) and more private or exclusive places have importance in the personal and collective value of social capital [START_REF] Putnam | Bowling Alone: The Collapse and Revival of American Community[END_REF]. Taking the absence of such informal public spaces, companies spread the idea that their stores are third places, e.g., Starbucks and Applebee's. Despite that, these companies fail to meet many aspects of third places and rarely do patrons engage others in lively, extended and informal interaction [START_REF] Khermouch | Third places[END_REF]. For Oldenburg [START_REF] Oldenburg | The great good place: Cafes, coffee shops, bookstores, bars, hair salons, and other hangouts at the heart of a community[END_REF], third places are about civic responsibility and localized community maintenance and revitalization. On the other hand, Starbucks prefers an interpretation of 'third-place-ness' restricted to customer satis-faction, not civics [START_REF] Yan | Learning the behavior of users in a public space through video tracking[END_REF]. This interpretation of 'third-place-ness' involves a set of characteristics of a third place that are often at odds with the social needs of a third place. Considering the need for promoting the sense of being in a third place not limited to architectural constrains of a third place, we prefer the definition of thirdplaceness as the 'event' of achieving the third place's characteristics in a certain place and time.

Thirdplaceness is constructed and sustained through experiences and interactions with and in the place. The thirdplaceness experience can transform a place in a third place-like permanently or just for that period. For example, in a 'happy hour' can occur thirdplaceness giving people the temporary sense of being in a third place. In third places, thirdplaceness occurs very often maintaining and reinforcing in the community this sense of third place. Such places are democratic and accessible, allowing people to discuss politics, the last football game, or the next music festivals, strengthening the notion of community in those people. However, a room full of individuals talking is not a third place [START_REF] Oldenburg | The great good place: Cafes, coffee shops, bookstores, bars, hair salons, and other hangouts at the heart of a community[END_REF]. Thirdplaceness has been observed in ethnographic studies, describing that third places can have an emergent nature happening in spontaneous, sporadic and nomadic way, and even in unexpected places, e.g., in a sidewalk [START_REF] Calderon | Understanding NUI-supported nomadic social places in a Brazilian health care facility[END_REF] [START_REF] Memarovic | Rethinking Third Places: Contemporary Design With Technology[END_REF].

Aiming at observing thirdplaceness supported by technology, we designed a technological art installation, called WishBoard [START_REF] Ferreira | Translating Art Installation into ICT: Lessons Learned from an Experience at Workspace[END_REF], in a 'socially abandoned' space. WishBoard creates a favorable space for occurrence of the sense of being in a third place, independent of where and when, a thirdplaceness.

ICT for self-expression and contemplation -WishBoard

WishBoard [START_REF] Garcia | On the quest of discovering cultural trails in social media[END_REF] is an interactive art project that invites people to share their individual aspirations with the community. The installation gives people a reason and an opportunity to express openly their thoughts and feelings in a public and common space. WishBoard offers a space for self-reflection, which differs it from similar works, for example, The Facebook Wall [START_REF] Bahr | Why Internet companies are linked to some of the happiest employees and customers[END_REF] as seen in figure 1a, and others, such as, CitySpeak [START_REF] Lévesque | p2P: Cityspeak's reconfiguration of public media space[END_REF], TexTales [START_REF] Agren | Virtual community life: A disappearance to third places for social capital[END_REF], and Discussions In Space (DIS) [START_REF] Schroeter | Engaging new digital locals with interactive urban screens to collaboratively improve the city[END_REF], that encourage civic engagement through the reflection about the community problems. WishBoard also differs from other installations, such as, "A Wall of Wishes" [44] and Wishing Wall [START_REF] Guljajeva | [END_REF] (see figure 1(b) and 1(c)) in the way of engaging people by asking them to complete a fixed prefix about their expectations for the future and presenting the wishes. WishBoard translates the essence of Chang's installation [START_REF] Chang | Before I Die[END_REF], consisting of five screens presenting the sent wishes dynamically and the inspiration considered imaginary and dreams, as present in figure 2, leveraging discussions and socialization. Every time a wish is sent, WishBoard creates a typographic art animation displaying the wish on the screens. After that, the sentence 'flies through the clouds' and joins the other sent wishes giving a certain sense of community belonging. Users send an anonymous wish accessing the WishBoard website through their mobile devices. Once this wish is sent, the WishBoard system presents it with an animation on the screen using a WebSocket server to synchronize all screens and exploring the potential of HTML5 and CSS3.

Chang [START_REF] Chang | Before I Die[END_REF] built a public art installation in which people could reflect their aspirations for a life that has an end. The installation consists of a wall painted in black stenciled with a grid of the sentence "Before I die I want to...". Chang observed that the space, once abandoned, became a meeting point in the neighborhood, as shown in figure 3. Moreover, Before I Die affords people to socialize, collaborate, appropriate the space and use, share democratically the space, and express their feelings considering the finitude of life [START_REF] Chang | Before I Die[END_REF]. Translating the essence of an art installation needs to consider cultural issues and adaptations for the new context [START_REF] Ferreira | Translating Art Installation into ICT: Lessons Learned from an Experience at Workspace[END_REF]. WishBoard has some adaptations maintaining the proposal of sharing in a public space, promoting individual reflection, supporting self-expression (without identification), and allowing variations (remixes) adapting to other contexts, as presented in figure 4. Before I Die [START_REF] Chang | Before I Die[END_REF] aims at contemplating death to make lives meaningful. On the other hand, WishBoard celebrates life reflecting about expectations for the future. WishBoard also has a keyword-based filter to unsuitable sentences due to the experience reported by Chang [START_REF] Chang | Before I Die[END_REF] about people writing bad things. Such corpus demands future analysis.

Study design

Aiming at investigating the audience behavior, we carried out an in-the-wild study using the WishBoard installation in order to analyze collected messages comparing with the Before I Die installation. According to Rogers [START_REF] Rogers | Interaction design gone wild: striving for wild theory[END_REF], this kind of study involves deploying new technologies in real-world conditions to observe the real-use of these technologies by people in their real context of use. We conducted this investigation dividing into two deployments using the WishBoard allowing us collect data from messages, spontaneous comments and video recordings filming both the users and the installation. Besides that, one observer was present, in a strategic place, taking field notes without disturbing the normal characteristics of the space. The advantage of using observations, taking notes and recording video, is that users behave most natural making the findings ecologically highly valid. On the other hand, a concern about the privacy of users is required.

In both deployments, we used the lobby of the Department of Computing at [university's name], attracting passersby to interact with the installation. In addition, the installation was set following the model of central and peripheral attention [START_REF] Müller | Requirements and design space for interactive public displays[END_REF] and observations done in our previous work. Furthermore, that space is a transitional and socially non-used space, as well as, the space in the first Before I Die installation. Taking the advantage of the festive spirit around the New Year, in the first deployment, we used the prefix 'For next year I want …'. And, in the second deployment, conducted in April, we adapted the prefix to 'For this year I want …'. In both deployments, the installation was available for five days.

In order to create an inviting place and to allow people to interact with the Wish-Board, we provided a beanbag (giving a space to sit, talk, and send wishes) and information about how to interact with the installation, as presented in figure 5. Wireless signal was available for people to submit wishes accessing a website with a QR code or a tiny URL. We have used the department' mailing list for informing people of the installation, collecting of data, presence of cameras, and the privacy policies. In addition, before posting a phrase, the system presented to user the ethical agreement on keeping the anonymity of senders and use the content for research only. Moreover, we did not request register information from the participants, e.g., name and email.

Behavior Discussion & Wishes Analysis & Cultural Findings

Aiming at observing the impact of the WishBoard deployments in supporting community awareness and thirdplaceness experience, we gather all collected data and we started an investigation of the audience behavior and also the content shared on the displays discussing our findings.

Understanding the audience behavior

In public space, the efforts in evaluation have been on identifying humans and clustering behaviors aiming at finding unusual behaviors that may help to model human interactions. Automated methods from computer vision as systems to track people sometimes can help in identifying people and audience behavior. However, uncontrolled environment and complicated scenes can decrease the accuracy of these systems [START_REF] Yan | Learning the behavior of users in a public space through video tracking[END_REF]. In this work, we manually analyzed fifty hours of recording from both deployments, captured by two cameras, making annotations, coding, and then summarizing the results. Inter-rater reliability was substantial (Cohen's Kappa = 0.75) [START_REF] Landis | The measurement of observer agreement for categorical data[END_REF]. We observed the aspects described by Alt et al. [START_REF] Alt | How to evaluate public displays[END_REF] basing on a literature survey and their expertise on evaluating public displays installations. The aspects were audience behavior around the installation, effectiveness of the installation, social interaction among users, users' acceptance by engaging and inviting others, and people's concerns about their privacy. For each aspect, we observed some variables comparing the results with the reported in the Before I Die [START_REF] Chang | Before I Die[END_REF] discussing the findings.

Audience behavior

In order to understand how people behave around the installation, we examined our annotations and then we reviewed the recording for a better comprehension. In Wish-Board, we observed that users felt engaged in posting a message when they were in a group, about 68% of the messages. Groups formed around the screens started a competition for the most creative message. We noticed many situations in which the audience around the installation progressively increased forming a sociable 'buzz' in the area, creating the Honey-pot effect [START_REF] Brignull | Enticing People to Interact with Large Public Displays in Public Spaces[END_REF]. Sometimes people stopped late and had to walk back trying to explore/understand more. This effect, known as Landing effect [START_REF] Müller | Looking glass: a field study on noticing interactivity of a shop window[END_REF], occurred more often when there was no one in the installation or people were in a hurry noticing something different on site.

During the experiment and the analysis, we found in few moments people appropriating the installation. According to Salovaara et al. [START_REF] Salovaara | Appropriation and creative use: linking user studies and design[END_REF] appropriation refers to the creative ways in which people adapt and repurpose technologies to serve their own goals. In WishBoard installation, people subverted, in a few situations, the installation proposal by sending messages for the immediate present instead of future. For example, 'Para o próximo ano eu quero um café' (In English: 'For next year I want a coffee'). People began to exchange messages to each other and to joke with each other, e.g. 'Para este ano eu quero que o Lucas realmente trabalhe' (In English: 'For this year I want that Lucas works really hard'). People sent emoticons and internet slang expressions, e.g., 'hauhauhau' to express laughs, commonly used by Brazilians.

In the Before I Die, it was common people see drawings on the wall, such as, hearts, stars and rainbows. In addition, people wrote wishes for the present, e.g., in Berlin, a man wrote on the wall that he wanted to eat a strawberry cake and then immediately he entered in the nearest café [START_REF] Chang | Before I Die[END_REF].

These findings suggest that, in public spaces, installations must be prepared for the Honey-pot effect and the Landing effect. In addition, tools for self-expression need to allow appropriation of use, which may be a sign of new uses for the installation, reflecting the necessities of some users. Furthermore, allowing appropriation of use plays an essential role in designing tools for self-expression.

Display effectiveness

Attracting attention with interactive installations is not easy. In public spaces, this is more challenging due many other objects strive for the user's attention [START_REF] Müller | Looking glass: a field study on noticing interactivity of a shop window[END_REF]. Analyzing the footage, we observed if people looked at the installation, how many stopped and how long they stayed on site, and how many started interacting with the installation sending a message in front of the screens.

In the WishBoard installation, it was evident the passersby turning their head to the installation area attracted by the screens. Some people kept on walking looking at the displays and some people mentioned curiosity in testing the system. In the first deployment, about thirty percent of the passersby stayed in the installation and more than half of these people remained there for more than one minute [START_REF] Ferreira | Translating Art Installation into ICT: Lessons Learned from an Experience at Workspace[END_REF]. In the second one, about one quarter of the passersby stayed in the installation and about sixty percent of these people remained there for more than one minute. We counted approximately 325 users sending messages in front of the screens.

According to Chang [START_REF] Chang | Before I Die[END_REF], with the first Before I Die wall, people paid closer attention to the abandoned house. People started to know more about the place, once forgotten, asking others what happened there and where the house's owners were living.

For achieving effectiveness, attractiveness plays an important role drawing audience attention and engaging them to use the installation, having correlation with to the chosen location for the deployment and the ease of access to the place. However, display effectiveness raises the question of whether the system is sustainable for gaining feedback from the public beyond the novelty factor of the technology.

Social impact

Aiming at observing how an interactive installation could foster social interaction, we analyzed the video recordings and the annotations aiming at finding social experiences as interactions promoted by the installation.

In both deployments, people used the WishBoard space, never used before except to cross the hall, to gather, to introduce themselves, and to talk about their common interests. People democratically shared devices, the pouf and even the floor. The wishes leveraged discussions and sometimes laughs. For example, a group of professors began discussing the wishes shown on the displays and guessing who sent those wishes showing interest in knowing more about the community. We observed people experiencing unexpected encounters with their acquaintances as shown in figure 6. The place became a space for students, professors, employees and visitors to share playful conversations. For example, a group of students was asking the results of the exams for a professor and this professor answered sending a message for them saying to stop annoying him with the following sentence 'Para o próximo ano eu quero que os alunos me deixem em paz!!!'(In English: 'For next year I want that the students leave me alone!!!'). Regarding to public displays, they can give people a chance to socialize taking the advantage of the Honey-pot effect [START_REF] Brignull | Enticing People to Interact with Large Public Displays in Public Spaces[END_REF]. Our findings support the notion of community, constructed and sustained through experiences and interactions with and in the place. Besides that, self-expression and information provided by these public displays leveraged discussions.

User acceptance

Designing engaging systems, which make the users feel invited, interested and encouraged to interact, is very challenging. Engaging people to use continually the installation is still more difficult to achieve. These regulars are important shaping the "tone" of the place [START_REF] Oldenburg | The great good place: Cafes, coffee shops, bookstores, bars, hair salons, and other hangouts at the heart of a community[END_REF]. Reviewing the videos, we observed these regulars considering, in this work, as people who used the installation more than twice and also invited others to use it.

In the first deployment, we already had identified about seven regulars [START_REF] Ferreira | Translating Art Installation into ICT: Lessons Learned from an Experience at Workspace[END_REF]. In the second deployment, we found four regulars. Figure 7 shows some of these regulars.

In several Before I Die installations, it was evident the presence of regulars [START_REF] Chang | Before I Die[END_REF]. Regulars bring the place to social life maintaining the playful mood [START_REF] Oldenburg | The great good place: Cafes, coffee shops, bookstores, bars, hair salons, and other hangouts at the heart of a community[END_REF].

Privacy concerns

Regarding to privacy concerns, the wishers were anonymous, and we observed how people perceive privacy issues reviewing videos, annotations, audience comments.

In WishBoard, we noticed many situations in that people wanted to show their message to others, pointing to their message or even calling someone to see their message. For example, a user called a girl friend who was passing by and sent the following phrase for her: 'For the next year I want <girl's name> finishing her PhD!!!'. She smiled and raised her hands celebrating, as shown in figure 8a. Some people took photo of the message, as present in figure 8b. In the comments from the audience, people said that they perceived that it was not possible to identify their anonymous wishes, making the use of the system comfortable for them. Nobody expressed any privacy concerns. Similarly, in the Before I Die some people took photos in front of the installation and showed their message to others. Furthermore, some people reported that the anonymous nature of the contribution could allow shy people to express themselves [START_REF] Chang | Before I Die[END_REF]. This anonymous nature of wishes creates a neutral and leveler ground for all users without caring about the individual's status in community [START_REF] Oldenburg | The great good place: Cafes, coffee shops, bookstores, bars, hair salons, and other hangouts at the heart of a community[END_REF].

Observing thirdplaceness supported by ICT

Several studies have been focusing on observing how ICT can support communities or socialization in third places [START_REF] Farnham | Measuring the impact of third place attachment on the adoption of a place-based community technology[END_REF][18] [START_REF] Mccarthy | Supporting community in third places with situated social software[END_REF]. According to Oldenburg [START_REF] Oldenburg | The great good place: Cafes, coffee shops, bookstores, bars, hair salons, and other hangouts at the heart of a community[END_REF], a real third place is a community-meeting place where the owner plays a key part in creating social connections by welcoming strangers and introducing them to the regulars. Such places are conversational zones where people can talk freely, openly, and entertain without care about their social status. In our study, we aimed at observe the occurrence of thirdplaceness in a place augmented by ICT that is not a third place. For this, we reviewed the videos, the annotations, and the comments listing our findings related to thirdplaceness for each characteristic of third places. In order to achieve thirdplaceness these characteristics are essentials:

 Neutral ground: People need to feel free to express their thoughts and opinions.

Promoting tools for self-expression can support to achieve this feeling. Moreover, we observed in comments that the anonymity can comfort audience giving them a chance to express themselves.  Leveler: People might have the same privileges and opportunities to express themselves or to participate of the activities in such place. Similarly as for neutral ground, anonymity can give the leveler awareness. This leveler awareness was present on the messages sent. Students, professors, employees and visitors used the installation sharing democratically their thoughts.  Conversation: The experiences and interactions with and in the place can empower relationships and promote place attachment. We observed that ICTs can promote activities among its users such as competitions, and provide information that can leverage discussions and conversations. Besides that, people had a chance to socialize taking the advantage of the Honey-pot effect [START_REF] Brignull | Enticing People to Interact with Large Public Displays in Public Spaces[END_REF].  Accessibility and accommodation: The place needs to be easy to access and comfortable. In order to promote the feeling of fulfilled needs in the occupants, allowing appropriation of use plays an essential role. In the comments, people said the WishBoard was ease to use and accessible providing information and Wi-Fi signal allowing users to interact with it. Regarding to the few non-users of smartphones, we observed people sharing their devices and a tablet was available in the second deployment. After the installation, the department perceived the lack of accommodation and installed chairs around the department providing more comfort for all.  Having regulars: Regulars bring the place to social life making newcomers welcomed and shaping the "tone" of a place. We noticed that themes or activities can also support to shape the "tone" [START_REF] Memarovic | Rethinking Third Places: Contemporary Design With Technology[END_REF], but people will appropriate the space according to their needs or possibilities.  Low profile: The place needs to provide some comfort for their occupants, but this cannot be the main reference of the place. People are the main reference of a place that promotes thirdplaceness. We observed people sharing the space, the pouf and even the floor enjoying each other's company and having playful conversations.  The mood is playful: The mood is not hostile and people always try to find ways to maintain their spirit high, even when few people want to bring this feeling down.

In the WishBoard, people maintained a lively mood avoiding sharing unsuitable things. We also observed people very excited in seeing their messages on the screens, sharing smiles and laughs.  A home away from home: People often have the relative feelings of warmth, possession, and belonging of being in their own homes. People can feel a piece of themselves is rooted in the space through their collaboration sharing a message shown on the screens.

Thirdplaceness is the 'event' of achieving these third place's characteristics in a certain place and time. Furthermore, this emergent nature transforms the concept of third place, making it independent of architectural constrains. This thirdplaceness experience can transform a place in a third place-like permanently or just for that period. However, in a third place will always occur thirdplaceness.

Finding cultural trails in anonymous posts

Hofstede et al. [START_REF] Hofstede | Cultures and Organizations: Software of the Mind: intercultural cooperation and its importance for survival[END_REF] defines culture as "the collective programming of the mind that distinguishes the member of one group or category of people from others". Several studies have been analyzed culture from data on the Web [START_REF] Garcia | On the quest of discovering cultural trails in social media[END_REF]. According to Garcia [START_REF] Garcia | On the quest of discovering cultural trails in social media[END_REF], making analysis of culture can benefit cultural social studies and the understanding on how culture affects user behavior giving insights to the designers. In addition, this analysis can model differences and similarities between cultures leading to serendipity by the discovery of unexpected data [START_REF] Garcia | On the quest of discovering cultural trails in social media[END_REF]. However, our work differs from those cultural researches due to restricted data. We plan to analyze a set of phrases without related personal information (e.g., gender and age) or systematized correlations (e.g., replies). For that reason, we propose to conduct a language dependent analysis using: the classification used in the Before I Die, the Affective Norms for English Words, the Hofstede's cultural dimensions, and morphologic analysis. Our goal is to find cultural trails considering the 391 messages collected by WishBoard.

Classifying the messages

Aiming at understanding the topics shared in the WishBoard, we performed a classification. In our previous work [START_REF] Ferreira | Translating Art Installation into ICT: Lessons Learned from an Experience at Workspace[END_REF], we classified the 244 messages collected in the first deployment into the themes presented in 'Before I Die, the book' [START_REF] Chang | Before I Die[END_REF]: helping others, love, well-being, celebrities, fame, family, religion, wealth, work, travel, outer space and sex. Moreover, we introduced three new themes: study, humor and politics. For each sentence, we classified in at least one of these themes. For sentences classified by the coders in different categories, we made decisions about the category chosen. In this work, we did the same for the second deployment comparing with Before I Die. In this analysis, we had a good inter-coder reliability (Cohen's Kappa = 0.61).

Our findings show that the both deployments had a similar pattern, suggesting their preferred topics. People discussed about ongoing Brazilian soccer championship, upcoming episodes of their favorite TV series and movies, their wishes for love, peace and success. Comparing the results of Before I Die with the WishBoard results there is also a similar pattern, as presented in figure 9. Fig. 9. Comparing the concentration of messages in WishBoard and Before I Die. Observing the graph, there is a higher expression about study, work and wealth in the WishBoard, what not happen with Before I Die. We believe that is because aspects related to the place context may lead people to think and talk more about related themes, in this case a workplace for some and study place for others. On the other side, there is a bigger concentration of phrases about love, travel, helping others, family and religion in the Before I Die, once it promotes a reflection about the end of life leading people to express religious values and feelings for family.

Regarding to the sentences, there was a match between WishBoard and Before I Die. For example, people want to live intensely, lose weight, find a love, be rich and visit other countries. The mains topics discussed for each theme in the WishBoard were:

(1) Love: find a love and declarations of love; (2) Well-being: things that make them happy (e.g., parties); (3) Celebrities: soccer, sayings, music lyrics, movies, and games; (4) Fame: be heard and have their wishes coming true; (5) Family: Friendship; (6) Wealth: be rich, win the lottery, and bought things; [START_REF] Brenny | Social Connectedness and Inclusion by Digital Augmentation in Public Spaces[END_REF] Work: fewer meetings, bureaucracy, and scholarship; [START_REF] Brignull | Enticing People to Interact with Large Public Displays in Public Spaces[END_REF] Study: disciplines course, improving their grades, and getting a scholarship to study abroad; [START_REF] Calderon | Understanding NUI-supported nomadic social places in a Brazilian health care facility[END_REF] Travel: travel more, visit other countries; [START_REF] Chang | Before I Die[END_REF] Sex: metaphors to reference women private parts; [START_REF] Corcoran | Professional socialization and contemporary career attitudes of three faculty generations[END_REF] Politics: be president and belong to some political party; [START_REF] Doheny-Farina | The wired neighborhood[END_REF] Humor: nonsense things and jokes. Analyzing the WishBoard messages inside the themes, the posts were casual, interesting, funny, poetic and sometimes heartbreaking. For example, in the well-being theme, people want more smiles, to live in a world without wars, to be a Jedi, to achieve nirvana and to fix everything wrong in his/her life.

Affective analysis

In order to analyze the aspects of affectiveness, we adopt the correspondent Affective Norms for English Words (ANEW) for our language -Brazilian Portuguese. The Affective Norms for English Words (ANEW) consists of a set of 1034 words, including verbs, nouns, and adjectives, based on a tri-dimensional perspective of emotions (valence, arousal, and dominance) [START_REF] Kristensen | Normas brasileiras para o Affective Norms for English Words[END_REF]. The Brazilian version of these norms, called ANEW-Br, provides affective norms only for valence (which ranges from pleasant to unpleasant) and arousal (which ranges from calm to excited) scales to judge the emotionality [START_REF] Kristensen | Normas brasileiras para o Affective Norms for English Words[END_REF]. Aiming at analyzing the audience mood through their sent wishes, we used ANEW-BR to calculate the valence and the alert. For each variable, we have used the average value. For the valence, this analysis consists in for each phrase we sum up all values of valence of words dividing it by the total number of occurrence of words. For example, in the phrase: 'Para este ano eu quero um bom emprego' (in English: 'For this year I want a good job'). Excluding the prefix 'For this year I want' we found two valence instances: good (bom) and job (emprego). Their valence values are respectively 8.19 and 7.31. Summing the values of the valences of the words and dividing by the amount of occurrence of words in the text, which in this example was 2: (8.19 + 7.31) ÷ 2 = 7.75 that means use of positive words. In order to calculate the alert, we proceed in the same way, but using the values for alert. Furthermore, we normalized all sent messages correcting typos manually. This process of normalizing phrases, transforming text into a single canonical form, can be (semi-) automatic using a parser.

Considering the collected phrases, we found a higher occurrence of valence between 7.20 and 8.75, and a concentration of alert between 3.36 and 5.45, suggesting that people used positive words, as shown in figure 10. According to Kristensen et al. [START_REF] Kristensen | Normas brasileiras para o Affective Norms for English Words[END_REF] positive words have the valence values higher than six and the alert values lower than five (V ≥ 6; A ≤ 5); negative words have the valence values lower than four and the alert values higher than five (4 > V < 6); neutral words have valence value between four and six (V ≤ 4; A > 5). Regarding to unsuitable sentences, people took care of the WishBoard installation maintaining its high spirits. In addition, we had only five blocked messages by being potentially offensive. In Before I Die installations, people also tried to maintain the mood of the installation by erasing bad things and filling with good things. These findings suggest that perhaps we need to explore ways to allow and to engage the community in managing their own messages in future work.

Individualism cultural dimension

In order to observe the Hofstede's cultural dimensions in the collected messages. We chose to measure the dimension of individualism of our audience due the restriction of our data. However, we still need to understand how to measure others dimensions with this set of data. For Hofstede et al. [START_REF] Hofstede | Cultures and Organizations: Software of the Mind: intercultural cooperation and its importance for survival[END_REF], individualism (IDV: high vs. low) can measure the level of integration into a group. Individualist societies (high IDV) represent loose ties between individuals basing on meritocracy and competition with other individuals. Collective (low IDV) societies represent individuals strongly integrated, extending sometimes the notion of family. However, the individualism dimension is not a dichotomous variable. This dimension ranges between 0 (lowest) and 100 (highest). For this work, we divided the dimension of individualism into four categories: collective, for a group of people, for other person, and individual. These categories ranges from low individualism to high individualism. For example, we classified 'For the next year I want more compassion among people' as collective and 'For the next year I want to be rich' as individual. For each sentence, we classified in at least one of those categories.

Analyzing the individualism in the collected phrases, we had an acceptable intercoder reliability (Cohen's Kappa = 0.67). We found in the average that 60% of the messages were for individual interests, followed by 12% for other person, 18% for a group of people, and 9% for collective. These results indicate a prevalence of individualism in the messages. We believe this occurred due the proposal of the installations, which it was a self-reflection about their future filling the prefix 'For next year, I want …' or 'For this year, I want …'.

Morphologic analysis

In this section, we discuss a form of morphologic analysis that can be performed (semi-) automatically using a Part Of Speech Tagger (POS-Tagger). This analysis has an objective of creating clusters for further interpretation. Aiming at analyzing tendencies in the messages, we used the PALAVRAS parser [START_REF] Bick | PALAVRAS: A Constraint Grammar-Based Parsing System for Portuguese[END_REF] for Portuguese on the collected messages with the typos already corrected. After that, we created clusters with the adjectives found in the results. Observing the clusters formed, we found that, in both deployments, about 20% of the messages had superlative and comparative adjectives, predominating 'more' and 'better' words. Interestingly, in a manual analysis, we found that 24% of the messages had metaphors. Optimistic words such as more and better can be related with the proximity and the start of a new year which it makes people thinking positively and optimistically. In addition, Brazilians are known by their playful nature and easygoing personality, what can justify the large number of statements related to metaphors, jokes, music and poetry.

Conclusion

In this paper, we demonstrate the relevance of supporting self-expression in public spaces through an interactive art installation, entitled WishBoard, at a workplace. This installation creates an environment to people express their thoughts, feelings, and aspirations. During the deployments, people shared the environment, once socially forgotten, using to discuss various topics and enjoying each other's company. The installation gave people a chance to socialize taking the advantage of the Honey-pot effect and provided information that leveraged conversations reinforcing their notion of community. Our study found that, by allowing this expression, offers the neutral ground to people express freely their individuality. In addition, we showed the importance of allowing appropriation of use in self-expression. In order to verify cultural trails in anonymous messages, we described a model to categorize, to analyze emotion related information, to verify the level of individualism in Hofstede's cultural dimensions, and to realize a morphologic analysis. Overall, it validates our previous work [START_REF] Ferreira | Translating Art Installation into ICT: Lessons Learned from an Experience at Workspace[END_REF] reinforcing the essential role that public displays and mobile devices can play in providing an accessible mean for people express their individuality.

As part of our future work, we hope to extend our evaluation methods exploring the potential of quantitative methods developed from the field of psychology, such as the "Sense of Community Index" [START_REF] Long | Confirmatory Factor Analysis of the Sense of Community Index and Development of a Brief SCI[END_REF]. Furthermore, we also hope to explore this technology intervention to support a sense of community in further different settings.

Fig. 1 .

 1 Fig. 1. Installations that promote self-expression: (a) The Facebook Wall [4], (b) "A Wall of Wishes" [44], and (c) Wishing Wall [16].

Fig. 2 .

 2 Fig. 2. (a) WishBoard installation, (b) mobile user interface, and (c) the system architecture.Users send an anonymous wish accessing the WishBoard website through their mobile devices. Once this wish is sent, the WishBoard system presents it with an animation on the screen using a WebSocket server to synchronize all screens and exploring the potential of HTML5 and CSS3.

Fig. 3 .

 3 Fig. 3. A Before I Die installation and people writing on the wall their wishes related to what they want to do before they die [10].

Fig. 4 .

 4 Fig. 4. Comparing Before I Die and WishBoard in order to: a) write a message (chalk vs. phone); b) personalize the message (color chalk vs. color font); c) read messages (blackboard vs. public screen).

Fig. 5 .

 5 Fig. 5. People hanging out observing displayed wishes (a, b, c) and (d) using cellphone to interact with the WishBoard public display.

Fig. 6 .

 6 Fig.6. User greeting an acquaintance while he was using the installation.

Fig. 7 .

 7 Fig. 7. Regulars using the WishBoard in different moments engaging other users in using it and starting a conversation about something shared on the screens in a process of socialization.

Fig. 8 .

 8 Fig. 8. (a) User celebrating a message sent for her and (b) users taking pictures of the messages.

Fig. 10 .

 10 Fig. 10. Concentration of phrases in the WishBoard regarding to their valence and alert. The higher concentration of phrases on the top-left area indicates bigger number of positive phrases.

Acknowledgments

We thank all the participants and our sponsors Boeing, FAPESP and CAPES.


Matthias Heintz

Lai-Chong Law
email: elaw@mcs.le.ac.uk

Samaneh Soleimani

Paper or Pixel? Comparing Paper-and Tool-Based Participatory Design Approaches

Keywords: Participatory design, Paper-based, Tool-based, Coding scheme

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Participatory design (PD) is a broad research area regarding the inclusion of prospective users in the design and development process for various physical as well as digital systems, products, and services ( [START_REF] Greenbaum | Design at work: Cooperative design of computer systems[END_REF][START_REF] Muller | Participatory design: The third space in HCI (revised)[END_REF][START_REF] Schuler | Participatory design: principles and practices[END_REF]). The goal is to gather their insights and input, especially design suggestions. There exists a proliferation of PD approaches, methods, techniques, and tools, for example: an expert designer co-creates with a single user to create a 3D mock-up from scratch using clays; a researcher elicits feedback on a simple 2D paper mock-up from a group of users using coloured pens [START_REF] Sanders | A framework for organizing the tools and techniques of participatory design[END_REF]. In this paper, we present our research study on PD for "webapps", a collective term we use to refer to a variety of web-based applications, websites, and online portals. With prevalent PD techniques (see [START_REF] Sanders | A framework for organizing the tools and techniques of participatory design[END_REF] for an overview) feedback is typically elicited from participants in the form of verbal comments and sketches with the help of different materials and props such as storyboards, post-it notes, paper mock-ups, and acetates (see [START_REF] Walsh | FACIT PD: a framework for analysis and creation of intergenerational techniques for participatory design[END_REF] for details). Using a software tool for PD instead, enables the user to interact with a prototype directly, creating a more realistic and engaging experience (e.g. [START_REF] Lin | DENIM: finding a tighter fit between tools and practice for Web site design[END_REF][START_REF] Rogers | Interaction design: beyond human-computer interaction[END_REF][START_REF] Sundar | Interactivity as self-expression: a field experiment with customization and blogging[END_REF][START_REF] Teo | An empirical study of the effects of interactivity on web user attitude[END_REF][START_REF] Zhao | Enhancing perceived interactivity through network externalities: An empirical study on micro-blogging service satisfaction and continuance intention[END_REF]). This could result in more feedback of possibly better quality, as compared to the paper-based approach. In addition, applying a software tool for data gathering can be advantageous for the data analysis, i.e. digitalisation of PD data at the time of capturing can enhance the effectiveness and efficiency of data processing (e.g. mitigating data loss; enabling software-supported data analysis).

However, there are also some constraints for using a software tool as compared to using pen-and-paper, e.g. computer access is needed; drawing on paper is more natural than on a screen [START_REF] Weibel | Pa-perSketch: a paper-digital collaborative remote sketching tool[END_REF].

When comparing the two approaches for PD, amount and quality of feedback is the most important aspect. Accordingly, we formulate two research questions (RQ):

RQ1: To what extent is the number of comments captured by tool-based PD activities on a specific webapp different from that by their paper-based counterparts?

RQ2: How are the comments captured by tool-based PD activities on a specific webapp qualitatively different from those by their paper-based counterparts?

The specific webapp for which our PD activities have been performed is a webbased portal under development in the European Go-Lab project. The goal of this project is to develop a portal [START_REF] Govaerts | Towards an Online Lab Portal for Inquiry-based STEM Learning at School[END_REF] and online tools to facilitate the integration of online labs in science lessons. PD activities on the associated mock-ups have been performed with the target groups of the project, namely teachers and students from primary schools up to universities. According to the IBF Participatory Continuum Model [START_REF] Read | An Investigation of Participatory Design with Children -Informant, Balanced and Facilitated Design[END_REF] our PD activities lean towards the informant design [START_REF] Walsh | FACIT PD: a framework for analysis and creation of intergenerational techniques for participatory design[END_REF]. When participants are used as informants for design decisions it is a normal PD practice to develop initial mock-ups and utilise them to gather user feedback and ideas. Specifically, myBalsamiq, the web-version of the prototyping software Balsamiq, which can be used to create interactive mock-ups from predefined or custom shapes (see section 3.1 for details), has been selected as the tool for our PD activities, because the mock-ups had been created with this tool. Therefore the results from the PD sessions would directly be visible and integrated in the design environment. Designers could not only instantly see it but also use it as a base or an input for the next re-design iteration. To further support the decision to use myBalsamiq for the PD activities and not only rely on the designers' choice of software tool, as it might be very good for design work but inappropriate for PD activities, we additionally performed a tool evaluation. From the results the use of myBalsamiq can also be inferred (see Related Work section for details). As it is not clear yet if the tool usage leads to results comparable to using the conventional way of gathering the feedback on paper, we aimed to answer the two research questions stated above to justify applying the tool over using paper in order to benefit from the described advantages. But it also goes the other way around: We are aware of some of myBalsamiq's shortcomings (e.g. the requirement for an Internet connection) and the question then becomes if paper is an appropriate way to be used as a back-up, e.g. in case of very limited Internet access.

To answer the two RQs, it is crucial to have a coding scheme for analysing and comparing PD user comments, thereby allowing systematic comparisons of the crossmedia results (paper vs. digital). As such a scheme is lacking in existing literature, we have been motivated to develop one, which we named CAt+, to fill the gap. It comprises one Categorisation and three Attributes (Impact, Specificity, Uniqueness). By rating comments based on those attributes, it allows quantifying and assessing qualita-tive characteristics of PD comments and thus enables a more thorough comparison (see section 5 for details). The coding scheme has been developed with generalizability in mind to enable other PD professionals to apply it for coding their participants' feedback. Although this generalizability of CAt+ is yet to be established, it can be considered as advancing a critical step for this specific area of PD data analysis. In this study we focus on the use of the coding scheme to compare data collected using the two different methods. In our future work, we will validate the impact of the scheme on enabling designers and developers to make sense of user feedback and implement changes.

Related Work

Some studies (e.g., [START_REF] Hundhausen | The design and experimental evaluation of a tool to support the construction and wizard-of-oz testing of low fidelity prototypes IEEE Symposium on Visual Languages and Human-Centric Computing[END_REF][START_REF] Segura | A Sketch-based Prototyping Tool for Defining and Evaluating User Interface Behavior[END_REF]) compare paper-and tool-based approaches to collect user input empirically in contexts other than but somehow related to PD, including software inspection [START_REF] Macdonald | A Comparison of Tool-Based and Paper-Based Software Inspection[END_REF] and multimedia design [START_REF] Bailey | Are Informal Tools Better?: Comparing DEMAIS, Pencil and Paper, and Authorware for Early Multimedia Design[END_REF]. These comparison studies focused mostly on quantitative results (e.g., the number of defects found during inspection; task completion time) and on subjective opinions of the participants about the use of a tool versus the use of paper for performing specific tasks. However, they hardly compared the quality of the results (except [START_REF] Bailey | Are Informal Tools Better?: Comparing DEMAIS, Pencil and Paper, and Authorware for Early Multimedia Design[END_REF] where the richness of the user-generated screens was evaluated to some extent).

The drawbacks of exclusively quantitative (or qualitative) approaches are increasingly recognized in the field of HCI (e.g., [START_REF] Law | Attitudes towards user experience (UX) measurement[END_REF]). For instance, the total number of user comments cannot tell whether the comments address the content or the user interface design and interaction concept of the system evaluated. One of the challenges of qualitative analysis is the identification or development of a viable coding scheme.

Various coding schemes have been developed for a variety of topics, ranging from user comments on machine learning [START_REF] Stumpf | Toward harnessing user feedback for machine learning[END_REF], over student comments on the teaching performance of professors [START_REF] Kindred | He Will Crush You Like an Academic Ninja!": Exploring Teacher Ratings on Ratemyprofessors.com[END_REF] to YouTube comments [START_REF] Madden | A classification scheme for content analyses of YouTube video comments[END_REF]. [START_REF] Könings | An approach to participatory instructional design in secondary education: an exploratory study[END_REF] applied PD in a school setting to plan and improve lessons together with students and developed a coding scheme to code the spoken comments made during discussions. As those schemes are tailored to specific topics and use cases, they are not general enough to be applied directly to coding comments from different domains, e.g. PD user comments on webapp design.

While paper-based approaches remain commonly used for PD of webapps, some research efforts have been undertaken to mimic properties of paper in software tools suitable for PD by supporting graphical as well as textual feedback. One of them is GABBEH [START_REF] Naghsh | A tool to support collaboration in electronic paper prototyping[END_REF], an electronic paper prototyping tool which enables users to comment on the current design. Another one is DisCo [START_REF] Walsh | DisCo: a co-design online tool for asynchronous distributed child and adult design partners[END_REF] an online tool supporting PD workshops where adults and children collaborate. However, GABBEH only works with the DENIM tool [START_REF] Newman | An informal web site design tool inspired by observations of practice[END_REF] and DisCo is not yet publicly available for use.

To the best of our knowledge, no study has been conducted to compare systematically a paper-based and tool-based approach to determine to what extent software tools can be used to support or even replace paper-based PD activities. This gap has motivated us to conduct such a study.

The first and foremost step of the planned study was to identify an appropriate paper-based approach and software tool to use. As our target group involves children (students) as well as adults (teachers), we decided to choose a paper-based approach appropriate for children, and the Layered Elaboration approach [START_REF] Walsh | Layered elaboration: a new technique for co-design with children[END_REF] was proved to serve this purpose (see section 3.1 for details). It was shown that this approach could somehow be evolved into an online tool (DisCo, see above). However, the results gathered with DisCo were not yet compared to those gathered with the paper-based Layered Elaboration approach.

Given the constraints of DisCo and GABBEH, we had to find an appropriate tool to compare with the Layered Elaboration approach. The use of myBalsamiq by the mock-up designers was a strong argument to also use it for PD. But for a proper comparison of paper-and tool-based method, a sound tool selection is necessary. To guide this, ten requirements (six user-based and four developer-based) were gathered from three main sources: a literature review (especially [START_REF] Muller | Participatory design: The third space in HCI (revised)[END_REF][START_REF] Sanders | A framework for organizing the tools and techniques of participatory design[END_REF][START_REF] Walsh | FACIT PD: a framework for analysis and creation of intergenerational techniques for participatory design[END_REF]), end-user survey results, and unstructured interviews with main developers of the portal. The requirements scoped our search that resulted in 21 tools of which only two can fulfil most of the ten requirements: Appotate and MyBalsamiq, with the latter having the higher maturity and usability (details are reported in [START_REF] Heintz | Participatory Design Online Tool[END_REF]). Therefore myBalsamiq was chosen for comparing the effectiveness of the paper-based and tool-based PD approach in terms of the quality of feedback elicited.

Design of Empirical Study

PD Study with Interactive Mock-ups

With the Balsamiq software, three mock-ups were created that shared the same basic structure but differed in the complexity of the learning content to address students of different academic levels (Fig. 1). The PD work of the Go-Lab project has been implemented with two approaches: (1) tool-based with interactive mock-ups and (2) paper-based with printouts of those mock-ups. Details are described in the following.

Paper-based:

Layered Elaboration is a more recent paper-based prototyping technique [START_REF] Walsh | Layered elaboration: a new technique for co-design with children[END_REF], which is simple to apply, and has the feature of keeping the initial prototype and comments from an iterative process intact. By overlaying different acetate sheets for the same mock-up printout, researchers can identify, for instance, which features have most frequently been commented on. The process starts by providing each individual or a small group of participants with a usage scenario, a set of ordered numbered printouts of the mock-up, a clipboard, and acetate sheets. Participants are asked to read through the scenario, put one acetate sheet on a printout, one after the other following the given order of printouts, and then provide feedback by annotating the acetate sheets with text and sketches, while working through the scenario on their own pace within a 45-or 60-minutes timeslot.

Tool-based: Participants are provided with computer access and work individually or in a small team of two or three people. They are introduced to the mock-up and then shown how to modify the mock-ups using different options provided in the myBalsamiq editor (Fig. 2). As with the paper-based approach participants are given a usage scenario specific to the mock-up to follow and are asked to give feedback while following the scenario. As the tool adds the user feedback elements in a transparent layer on top of the prototype, leaving the original prototype intact, it is comparable to the layered elaboration approach. Among the feedback options there are yellow "virtual sticky notes", which are added to the prototype by dragging them from the menu, to give textual feedback. Such notes are also commonly used in paper-based PD methods [START_REF] Druin | Cooperative inquiry: Developing new technologies for children with children[END_REF]. As with the paper-based method the participants progress through the scenario on their own pace within a 45-or 60-minute timeslot.

Fig. 2. The main feedback functions of myBalsamiq (screenshot of myBalsamiq taken and included with the permission of Balsamiq Studios, LLC)

Participants and Procedure

The mock-ups were used in PD workshops in two schools in a European country (School 1 and School 2) and in an international teachers programme in Europe (Teachers Programme) to gather ideas for the improvement of the prototypes and data for the comparison of the paper-based and tool-based approach. The workshops were started with two presentations (i.e. one briefly on the project and one on the PD process), followed by about a one-hour session of hands-on activities with the mock-ups.

School 1:

The first PD workshop took place in a high school. Thirteen science students (mean age: 17 years old) were randomly divided into two groups with six using the Layered Elaboration approach (paper-based) and seven using myBalsamiq (toolbased). Two researchers were present to provide support, and each observed one of the two student groups.

The Layered Elaboration technique is typically applied on a group basis and with young children, albeit being applicable to other age groups as well. Because of the low number of participants, they did not work in groups at this event but gave feedback individually, to maximize the number of datasets, with no intention to use it as an intervention variable to compare the results from the two settings. Although the Layered Elaboration approach was altered slightly in this case, this did not affect the comparability of the data collected with the two PD approaches, as the participants using the tool to give feedback also did not work in groups at this event.

School 2: The second PD workshop took place in an elementary (or primary) school. The procedure was the same as in School 1, except having the students working in groups instead of individually, thus following the Layered Elaboration approach as initially described. 28 students (mean age: 10 years old) were randomly assigned to one of the approaches; 13 (in three groups of three and two groups of two) used the paper-based approach and 15 (in five groups of three) used the tool-based approach to give feedback. Because of the larger number of participants four researchers (two per group) were present this time, observing and providing support.

Teachers Programme: The third PD workshop was organized as one of the activities in a programme for international high school teachers. It took place in a research institute in Europe and involved 51 science teachers from 29 countries worldwide. The participants were split in groups of three. Feedback data from 8 groups working with the paper-based approach and 6 groups using the myBalsamiq tool were collected. For further data analysis, all comments, including textual and graphical, were recorded in Microsoft Excel sheets with two columns. The first one labelled "source" contains "participant and screen ID" allowing the retrieval of the original feedback from the digital files. The second column records the textual user comment enhanced with researcher-generated details to make it easier to understand (e.g. description of the position or target) or a description of the drawing. Comments covering several ideas were split into individual rows during this step to prepare for further analysis.

Origin and Description of Coding Scheme CAt+

Each user comment was coded by two HCI researchers (i.e. fully crossed design [START_REF] Hallgren | Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial[END_REF]) with about two and six years of experience in usability research. Content analysis was applied to generate categories while coding [START_REF] Krippendorff | Content analysis: An introduction to its methodology[END_REF]. Both researchers coded the comments in chunks of about twenty, introducing new category identifiers and definitions where necessary. To make sure that all comments were coded appropriately, earlier comments were revisited whenever a new category was introduced. The results were then compared and in case of discrepancy the researchers discussed till a consensus was reached.

Classifying comments based on their content can help to get an idea about the information contained, but not necessarily enables the comparison of the two approaches used to create the comments. For instance, comments on design are not necessarily "better" than comments on functionality; on this basis no conclusion can be drawn which approach is "better". Hence, a broader set of codes with meaningful ratings on the measurable quality of feedback such as specificity (the more specific, the better) was needed. Based on the literature on user defect classification systems (e.g. [START_REF] Vilbergsdottir | Assessing the reliability, validity and acceptance of a classification scheme of usability problems (CUP)[END_REF]) and downstream utility (supporting developers in addressing user feedback, e.g. [START_REF] Hornbaek | The Interplay Between Usability Evaluation and User Interaction Design[END_REF]), we identified the following three major attributes: Impactthe extent to which the mock-up will be changed by addressing the idea expressed in the comment; Specificity (regarding target, reasoning, and solution)the detailedness and thoroughness of the comment in terms of explicitly stating the target, reasoning, and possible solution to make an improvement; Uniquenessthe distinctiveness of the idea expressed in the comment. The initial coding scheme containing categorisation was thus completed by including those Attributes with values and definitions.

Accordingly, we name the coding scheme CAt+: Categories plus Attributes, with the plus of Attributes compared to other coding schemes that only apply Category (or content) based coding.

Categories

Table 1 shows the categories of CAt+. Each rating is composed of a sub-category together with the main category, e.g. "Content-Add" (i.e. no feedback is just rated as "Content").

Attribute: Impact

The impact rating specifies how much of the user interface would change if this comment is addressed. Its possible values from 0 to 4 are defined as follows:

 No changes suggested (code: '0'). There is nothing which could have an impact on the mock-up if implemented (e.g. "good idea").  Change affecting one element (code: '1'). As implementing the suggested changes would only influence a small part or a single element, the impact of this comment on the whole prototype is small (e.g. "the next [button] needs to be in a different colour to make it clearer.").  Change affecting several elements (code: '2'). As multiple parts of a mock-up page would change if this feedback is addressed, the impact of this comment is medium (e.g. "Do titles for pictures and stuff").  Change affecting the page on a level larger than element (code: '3'), e.g. by adding/removing an element to/from the page, which would cause a change of the layout of the other elements as well. As significant parts or even the whole mock-up page would change if this suggestion is implemented, the comment is rated as having a high impact (e.g. "Add some thing (sic!) here [white space on the right]."). If the comment does not specify an element, it is assumed that the whole webpage is the target and thus affected (e.g. "More colour").

Visual

Unknown

Comments of which the coders could not make sense or it was not clear how to address this comment (e.g. could be redesign or adding content or functionality).

Not understandable

Comments of which the coder could not make sense "Isn't the video" Unreadable Comments which or important parts of which could not be deciphered.

Irrelevant

Comments not related to the mock-up itself "less scribleing (sic!) [feedback to scratched out feedback from another participant]"

 Change affecting several pages (code: '4'). As implementing the changes suggested in this comment would change various parts of the whole prototype, its impact is rated as very high (e.g. "log out option").

Attribute: Specificity

The specificity of a comment indicates how detailed the feedback is. This influences how easily and fast the designer or developer can assess and address the feedback. If the target (e.g. an interface object) is specified, the developer is able to identify which part of the mock-up should be changed. If the reasoning for a comment is given, the developer may find a solution, even if none has been specified by the participant. If a solution is specified by the participant the developer can decide to implement it or take it as further guidance in finding a feasible solution. If such information is not specified, it might still be possible and reasonable for the designers or developers to make an educated guess. But if the informativeness of a comment is too low, even guessing might not be possible. Accordingly, a Specificity rating consists of six sub-ratings based on the three aspects of the comment discussed: Target, Reasoning, and Solution, and if they are stated or guessable. The aspects "Target/Reasoning/Solution stated" can have the value 0, 0.5 or 1. If the respective information is given explicitly in the feedback, the rating is 1. If it is somewhat clear what the participant means, the rating is 0.5. If no information is given, the rating is 0. The aspects "Guessability of Target/Reasoning/Solution" have been introduced to rate if this information can be guessed [START_REF] Greenbaum | Design at work: Cooperative design of computer systems[END_REF] or not (0). If it is not necessary to guess this information (most of the time because it has been clearly stated), this aspect is rated as 1 (i.e. "it is guessable").

Attribute: Uniqueness

Each idea or issue is either coded as 1 if it has not been mentioned before or as 0 if it is a duplicate. By adding up the coding over all comments, the number of distinct comments can be known.

Results and Discussion

RQ1: the Number of Comments

Through the PD activities, 701 valid comments (720 in total of which 19 were not related to the mock-up, e.g., a participant wrote a message to the researchers on the acetate "Sorry about my messy handwriting!") were given by participants using the paper-based approach. 190 valid comments (191 in total of which one was unrelated to the mock-up) were gathered using the tool-based approach.

When comparing the number of valid comments per individual or per group, the paper-based approach resulted in more than twice (School 2: 15.2 compared to 7.0) or even three times (School 1: 51.0 compared to 15.1; Teachers Programme: 27.0 com-pared to 8.3) as many comments per individual/group as did the tool-based approach. Due to the limited space, Table 2 only shows the results grouped by the main categories. Table 3 shows the percentage (to account for and offset the vast difference in number of results gathered with paper and tool) distribution of comments to categories for the three PD events, for the categories with a difference in the percentage values larger than 5% between the paper-based and tool-based approach in at least one event. Summing up, the empirical data of all the three PD workshops with students and teachers indicate that the paper-based approach was much more effective in terms of eliciting comments.

When looking at the percentage results per category presented in Table 3 most of the differences between paper and tool in one event are contradicted by the results of another event. Thus no advantage for paper or tool can be identified for most of the categories. Two exceptions are "Functionality -Missing description" and Unknown, where on a percentage basis (far) more comments have been created when the paperbased method was used to gather the feedback as when the tool-based method was used. For the missing description of functionality this might be explained by the paper being non-interactive and presenting the mock-up out of context (e.g. not on a computer screen; not in a browser). Thus users might have a harder time to identify the functionality of screen elements and therefore give the feedback that a description would be needed. The differences in "Unknown" can be partly explained by unreadable comments but more often by comments on the paper, where the target was unclear and therefore the problem could not be understood. We assume this happened less with the tool, because to give textual feedback there, the participants had to put a yellow sticky note, which they mostly put onto the screen element causing the issue, thus at least giving a hint regarding the target.

RQ2: the Quality of Comments

All comments categorized either as 'irrelevant' or 'unknown' (either not understandable or unreadable) were removed from further analysis, as they do not contain useful information for the designers and developers. Therefore 809 comments (632 paperand 177 tool-based) were further analysed with regard to the three Attributes of CAt+.

Impact. As presented in Table 4, most of the changes proposed by the participants have an Impact of either 1 or 3affecting a single element or the whole page. This might imply that the participants tended to perceive the mock-up from a holistic perspective, although they zoomed in to explore specific elements (e.g. the next button) in detail. Regarding the comparison of paper and tool it can be seen that paper elicited (slightly) more feedback coded as 1 as compared to the tool results with this impact coding. Participants being more willing to give feedback on small details with paper, might imply that giving feedback with the tool needs slightly more effort, which was more likely spent on ideas with bigger impact. Specificity. Table 5 presents the percentage of results were the corresponding subrating was clearly stated. When looking at the Target it can be seen that this is more specific in paper-based comments than tool-based. This may be explained by the fact that paper allows for a variety of ways to highlight a target (e.g. by drawing a circle around or an arrow pointing towards something, or by underlining text), where the tool used in the evaluation was restricted to a predefined set of feedback elements. Users mainly attached virtual sticky notes, which might explain the lower precision and could be approached by enhancing the functionality of the tool.

On the other hand the use of sticky notes might explain the higher specificity of the Reasoning found for the tool-based comments, as the text field on the note might have invited the participant to further elaborate. Uniqueness. To determine how many duplicated ideas have been generated with the paper-and tool-based approach, the comments have been rated based on their uniqueness. The results are shown in Table 6.

When comparing the percentage of duplicates in the paper-and tool-based results, it can be noticed, that it is mostly higher for paper. If one assumes that there is a limited pool of possible ideas the participants can come up with, the higher total number of comments for paper also explains the higher percentage of duplicates as it becomes less likely to come up with a unique idea with an increasing number of comments. The exception of this for School 1 might partly be explained by having the highest number of comments for the tool-based method throughout the three events, but is still unexpected. Inter-rater reliability. As two researchers were involved in coding the participants' comments, weighted Cohen's kappa [START_REF] Fleiss | The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability[END_REF] was calculated to determine the inter-rater reliability for the different coding criteria. For the Categorization the weight was determined based on the agreement about the main-and sub-category. If only the subcategory differed, a weight of 1 was used (as there was at least agreement about the main category of the comment), if the main category differed, a weight of 2 was applied. For Impact and Specificity, the weight has been determined by the difference between higher and lower value. For Uniqueness, the standard weight was used. For all ratings the value of Weighted Cohen's kappa was above 0.7, ranging from 0.72 (for Uniqueness) to 0.88 (for Specificity -Solution). Although the kappa rating magnitude guidelines in the literature are inconsistent (e.g. [START_REF] Altman | Practical statistics for medical research[END_REF][START_REF] Fleiss | Statistical methods for rates and proportions[END_REF]), with all values being above 0.7 we are still confident that our results are reasonable or even good.

Pearson's χ2 analysis of category rating. Some inferential statistics on the results were performed to check whether the observed differences are significant. As we have categorical data, we used Chi square tests [START_REF] Maltby | Early success in statistics[END_REF] to verify the null hypothesis (H 0 ): H 0 : the number of comments in each of the coding dimensions (categories, impact, specificity, uniqueness) is independent of the method used to elicit and capture them (paper or tool).

Table 7 shows the results. To get expected values larger than 5 (as required by the Chi square test, e.g. [START_REF] Maltby | Early success in statistics[END_REF]) it was necessary to combine some of the results. For categories Picture, Unknown, and Irrelevant were combined to "Other". For Impact 0, 1, and 2 were combined to "less than page level" and 3 and 4 to "page level and above". For Specificity the results were combined into three groups, very specific (two or more sub-ratings that are very specific), specific (one sub-rating that is very specific), and unspecific (no sub-rating that is very specific). For Uniqueness no combination was possible, therefore the result for School 2 is included in Table 7 although the requirement for Chi square was not met, as the expected value for "not unique" in "tool" was less than 5 (~4.85). A Fisher's Exact test has therefore been performed for the latter, confirming the rejection of H 0 in this case.

As can be seen in Table 7 the results are only affirmative for Categories and Specificity ratings. For Categories, H 0 has to be rejected, meaning that the different meth-ods influence the number of comments in different categories. For Specificity, H 0 is not rejected, meaning that the specificity of a comment is independent from the method used for feedback gathering. For Impact and Uniqueness the results of the Chi square tests are ambivalent.

Conclusion and Future Work

When comparing the number of comments given, one can easily recognize that the paper-based approach created much more results than did the tool-based approach, even after normalizing it based on the number of participants or groups using the respective approach. The answer to RQ1 is then clear. One reason for this difference lies in the tool chosen for the evaluation. During the PD sessions with the students and teachers, it showed that some usability problems of myBalsamiq had undermined the efficiency of giving comments. The results of the descriptive statistics suggest that no consistent trends or meaningful patterns in terms of the qualitative differences of the comments (i.e., Categorization and three Attributes) can be observed. The answer to RQ2 remains ambivalent.

Nonetheless, the number of students and teachers involved in the current study was relatively small. We are aware of this limitation. More studies, involving more participants would be needed to further substantiate the results presented in this paper.

Given the vast difference in number of comments gathered and the dependence of feedback gathered on the method used (at least for categories), we have to conclude that myBalsamiq cannot be used to replace the paper-based method. This interesting finding and the advantages we see in using a tool, have motivated us to develop a more usable PD online tool that can enhance the value of Participatory Design in general and be a valid substitute for the currently conventional and common paperbased method, to be confirmed by further evaluations. Furthermore, our coding scheme CAt+ for PD data analysis can be useful for other researchers in this field and future comparison studies. Furthermore, we will focus not only on using the coded results to compare paper-and tool-based performance but also demonstrating how the coding can be applied by developers and designers to make better sense of user feedback and address it more effectively. For instance, sorting the list of comments in descending order of Impact or Specificity rating could support them in dealing with the most important feedback first. This approach can also be automated as a kind of tool-supported content analysis in the future.

In summary, with the groundwork built as reported in this paper, our future work will comprise three main strands: (i) To further validate and substantiate our coding scheme CAt+ in different settings (i.e., co-located, distributed, individual, groupbased) involving different stakeholders (teachers, students, researchers, designers, developers); (ii) To evaluate the downstream utility of coded PD results by examining whether and how such results support a development team in their redesign work; (iii) To develop a usable PD online tool enabling participants to give comments with ease and even fun.

Fig. 1 .

 1 Fig. 1. Mock-ups of the portal: Homepage (left) and a webpage with learning content and tools provided by the teacher for students (right).

Fig. 3 .

 3 Fig. 3. Superimposed feedback gathered using the paper-(left) and tool-based (right) approach.

Table 1 .

 1 Name, description, and example for each category of the rating scheme.

	Category	Description	Example
	Content	Comments on the learning material.
	Add	Request for more	"Put some text on this page."
	Amount of text	Comment on the number of words used "Shorten the text."
	Change	Request for alteration (i.e. what is writ-ten), including typos	"However the questions don't seem to link in with the overall subject of the page."
	Language	Comment on the wording (i.e. how it is written)	"Use more child friendly lan-guage."
	Missing de-scription	Request for explanation	"Unclear what to do with these questions."
	Positive statement	Supportive comment	"I like the idea of having a video"; "Helpful instructions"
	Remove	Request for deletion	"You don't need it to say in three minutes"
	Terminology	Comment on only single words and their definition	"What does Buoyancy mean?"
	Design	Comments on the visual appeal of the mock-up.
	Add	Request for new graphical elements or sound	"add crashing sounds"
	Colourful	Comment on the colour or aesthetics	"The tabs could be more colour-ful."
	Friendly lay-out	Comment on the suitability for children "Kid friendly layout"
	Negative statement	Criticism	"I do not like how the lines meet, it looks messy."
	Not specific	General comment without detailed information	"Nothing to grab my attention."
	Positive statement	Supportive comment	"Good use of colour to engage the students."
	Screen layout	Comment on the positioning, order and size of elements on the page	"Make this bigger to fill the page?"
	Terminology	Comment on only single words	e.g. button labels: "Complicated word ["Conceptualization" tab]"
	Text layout	Comment on the format (size, style, colour, etc.) of the writing	"Better font, bigger font."
	Remove	Request for deletion	"Don't need this [Page 2/2]."

Table 2 .

 2 Number of comments per participant (School 1) or group of participants (School 2, Teachers Programme) for each of the main categories and in total.

	Main Category		Comments per participant/group
		School 1	School 2	Teachers Programme
		Paper	Tool	Paper	Tool	Paper	Tool
	Content	08.5	03.0	03.7	03.6	13.1	05.8
	Design	14.5	08.0	06.3	00.8	04.4	01.0
	Functionality	21.0	02.7	02.3	01.4	05.9	01.2
	Picture	00.0	00.3	00.5	00.4	01.9	00.0
	Unknown	06.7	01.1	01.8	00.8	00.8	00.2
	Irrelevant	00.3	00.0	00.6	00.0	01.1	00.2
	Total	51.0	15.1	15.2	07.0	27.0	08.3

Table 3 .

 3 Distribution of comments to categories (with a difference of more than 5% between paper and tool for at least one of the three result sets) in both Schools and the Teachers Programme (in %).

	Category			Percentage of comments
		School 1	School 2	Teachers
						Programme
		Paper	Tool	Paper	Tool	Paper	Tool
	Content	16.8	19.8	25.3	51.4	50.7	71.4
	Add	05.3	11.3	06.8	02.9	27.5	26.5
	Language	01.3	00.0	05.8	14.3	01.9	04.1
	Remove	00.3	00.0	01.6	00.0	01.0	06.1
	Terminology	03.0	00.0	04.2	22.9	02.4	08.2
	Design	28.6	52.8	43.2	11.4	16.4	12.2
	Colourful	05.9	09.4	08.4	02.9	01.4	00.0
	Text layout	01.3	11.3	12.1	00.0	04.3	02.0
	Functionality	41.4	17.9	15.8	20.0	22.7	14.3
	Add	23.7	09.4	03.2	17.1	07.7	14.3
	Missing description	06.9	03.8	11.1	02.9	07.7	00.0
	Picture	00.0	01.9	03.7	05.7	07.2	00.0
	Missing description	00.0	00.9	03.7	05.7	06.3	00.0
	Unknown	13.2	07.5	12.1	11.4	02.9	02.0

Table 4 .

 4 Distribution of Impact rating (in %).

	Impact		Percentage of further analysed comments
		School 1		School 2		Teachers Programme
		Paper	Tool	Paper	Tool	Paper	Tool
	0	15.9	10.2	09.6	00.0	07.5	06.3
	1	28.4	22.5	40.7	32.3	32.3	27.1
	2	01.5	01.0	07.2	25.8	09.5	14.6
	3	35.2	58.2	31.1	32.3	32.8	33.3
	4	18.9	08.2	11.4	09.7	17.9	18.8

Table 5 .

 5 Comments coded as very specific for each sub-rating (in %).

			Percentage of further analysed comments
		School 1		School 2		Teachers Programme
		Paper	Tool	Paper	Tool	Paper	Tool
	Target	86.1	65.3	78.0	61.3	83.6	68.8
	Reasoning	23.7	41.8	26.8	32.3	32.3	45.8
	Solution	26.8	20.4	06.6	22.6	40.8	41.7

Table 6 .

 6 Results of the Uniqueness rating.

		School 1	School 2	Teachers
						Programme
		Paper	Tool	Paper	Tool	Paper	Tool
	No. of comments	264	98	167	31	201	48
	No. of unique comments	225	80	137	30	178	47
	% of duplicates	14.8	18.4	17.9	3.3	11.4	2.1

Table 7 .

 7 χ2 values for independence of number of comments per rating dimension of CAt+ on method (for all three events). Shading of cells indicates, where H 0 was rejected. *Contains one expected value less than 5!

		School 1	School 2	Teachers Programme
	Categories	χ2 (3, n=412)=27.19,	χ2 (3, n=233)=15.51,	χ2 (3, n=266)=8.41,
		p<.001	p<.05	p<.05
	Impact	χ2 (1, n=362)=4.32,	χ2 (1, n=198)=0.004,	χ2 (1, n=249)=0.03,
		p<.05	p>.05	p>.05
	Specificity	χ2 (2, n=362)=3.52,	χ2 (2, n=198)=1.96,	χ2 (2, n=249)=4.44,
		p>.05 (ns)	p>.05 (ns)	p>.05 (ns)
	Uniqueness	χ2 (1, n=362)=220.24,	χ2 (1, n=198)=4.30,	χ2 (1, n=248)=20.57,
		p>.05 (ns)	p<.05 *	p<.001

Data AnalysisAll data were digitalised for further analysis. For the paper-based data, a set of the mock-up printouts and all the annotated acetates were scanned. With the use of the Gimp software the acetate part was removed from the scanned images, making the area where there was no drawing transparent again. Then all feedback from a single session was digitally layered onto the scanned mock-ups (Fig.3).

Acknowledgements. This work was partially funded by the European Union in the context of the Go-Lab project (Grant Agreement no. 317601) under the Information and Communication Technologies (ICT) theme of the 7th Framework Programme for R&D (FP7). This document does not represent the opinion of the European Union, and the European Union is not responsible for any use that might be made of its content.