Designing a creative

Keywords: confidence, trust, youth, creativity, soft skills 1

storytelling workshop to build self-confidence and trust among adolescents

Introduction

Contrasting with previous decades where success was tied to knowledge retrieval and competition, the challenges of the 21 st century require creative and collaborative solutions (National Education Association, 2012). There is a need to empower people to take charge of their journey through life and to take an active part in shaping society. Empowerment is a process of developing a person's or a group's ability to make purposive choices and to transform them into actions and results [START_REF] Alsop | Empowerment in Practice. From analysis to implementation[END_REF]. In doing so, people will need to navigate ambiguity, given that most environments are characterized by fast and multi-faceted disruption, and long standing plans are not adapted to the present context any more [START_REF] Gibbs Howard | Theories of adolescence[END_REF]. Hence, the possibility to move along a planned path is substituted by the need to explore and build new paths in the absence of certainty of outcome. People need to come together and practice curiosity rather than apprehension, to make and test rather than to speculate and control. In a fast paced, unpredictable world, the empowerment process therefore builds on the development of self-confidence and trust among individuals (Gibbs Howard, 2018).

Self-confidence can be described as a global self-regard determined by the perception of having the capabilities to handle a variety of situations effectively [START_REF] Shrauger | Self-confidence in College Students: Conceptualization, Measurement, and Behavioral Implications[END_REF]. It is based on an internal sense of overall positive self-worth and self-efficacy [START_REF] Bowers | The Five Cs Model of Positive Youth Development: A Longitudinal Analysis of Confirmatory Factor Structure and Measurement Invariance[END_REF]. The concept of self-efficacy refers to people's beliefs in their capabilities to produce given attainments linked to distinct realms of functioning [START_REF] Bandura | Self-efficacy: The exercise of control[END_REF]. It seems to be essential to consider the larger scope of self-confidence when targeting the achievement of self-fulfilment and wellbeing of the individual in a changing society. Beside the confidence to perform competently in general, as suggested by the definition of [START_REF] Shrauger | Self-confidence in College Students: Conceptualization, Measurement, and Behavioral Implications[END_REF], it seems relevant to take into account the components of confidence that play a decisive role in an individual's capacity to thrive in today's society. Among the subscales included in Shrauger and Schohn's Personality Evaluation Inventory (PEI) in accordance with the determinants of self-confidence most frequently reported by students, confidence in speaking could be considered as a contemporary skill that deserves particular attention in youth development. In order to thrive, an individual furthermore needs to be able to respond with flexibility to changing opportunities and challenges [START_REF] Bowers | The Five Cs Model of Positive Youth Development: A Longitudinal Analysis of Confirmatory Factor Structure and Measurement Invariance[END_REF]. This means that the capacity to overcome adversity is one of the individual thriving indicators [START_REF] Theokas | Conceptualizing and modeling individual and ecological asset components of thriving in early adolescence[END_REF]. Similarly, Leffert (1997) has identified commitment to learning or achievement motivation as a developmental asset. As reported by [START_REF] Park | The development of grit and growth mindset during adolescence[END_REF], effort and success in dealing with difficult situations have proven to be related to grit and growth mindset, the belief that intellectual ability can evolve (Dweck, 2008). Developing grit and growth mindset is a vector of resilience and thriving during adolescence [START_REF] Park | The development of grit and growth mindset during adolescence[END_REF]. Both persistence in the face of difficulties and the belief on one's capacity to learn how to deal with challenging situations therefore seem to be relevant components of self-confidence to be considered in the scope of positive youth development.

When people collectively operate to make progress in a context of high ambiguity, trust is key.

Although definitions of trust vary throughout the literature, they show one common element: trust is characterized by the willingness to depend on others with a feeling of relative security, in a situation that implies a certain risk, i.e. where negative consequences are possible [START_REF] Mcknight | Trust and distrust definitions : One bite at a time[END_REF]. While trusting intentions refer to specific other people, disposition to trust indicates a consistent tendency to be willing to depend on others, without being person-specific nor situation-specific [START_REF] Rotter | Generalized Expectancies for Interpersonal Trust[END_REF]. Similarly, [START_REF] Freitag | Spheres of trust: An empirical analysis of the foundations of particularised and generalised trust[END_REF] distinguish intimate trust in people close to the truster from abstract trust in people in general. Generalized trust seems to be essential when collaborating creatively with diverse groups of people, without building longterm trusting relationships. Collaboration has been consistently identified as a 21st century skill throughout different frameworks [START_REF] Voogt | 21st Century Skills[END_REF]. Given that creative tasks imply a certain exposure to risk, mutual relationships of trust have been stated as a precondition of creative collaboration [START_REF] Storey | The emotional dance of creative collaboration[END_REF][START_REF] Searle | Creativity and innovation in teams[END_REF]. It seems thus relevant to consider the manifestations of trust in the context of cooperation and especially in situation where a challenge is faced with collective effort. In this context, it has been suggested that safe environments which minimize vulnerability are required [START_REF] Searle | Creativity and innovation in teams[END_REF]. In situations where people need to collaborate with strangers, it could be argued that the ability to make oneself vulnerable to others (as a component of trust) is needed as well. Apart from the emotional safety in collaborative situations, trust is also tied to receiving help from others in a group [START_REF] Eteläpelto | The resources and obstacles of creative collaboration in a long-term learning community[END_REF]. Being able to accept help from others in situations that are too challenging for the individual seems to be a crucial component of trust, closely related to trusting the capacity of the group to deal with that situation. This assumption is in line with the conceptualisation of trust as a dispositional willingness to rely on others [START_REF] Colquitt | Trust, trustworthiness, and trust propensity: A metaanalytic test of their unique relationships with risk taking and job performance[END_REF], which implies to be comfortable with feelings of interdependence. Finally, just as the confidence in one's ability to learn how to deal better with uncomfortable situations, trust in others could manifest as the willingness to learn from others in a cooperative context. This assumption reflects the view of a deeply social view of learning and the associated issue of trusting to learn [START_REF] Landrum | Learning to trust and trusting to learn: a theoretical framework[END_REF].

During the life span, adolescence is a difficult phase of adjustments that is especially sensitive to drops in self-confidence [START_REF] Alsaker | Self-concept, self-esteem and Identity[END_REF]. Youth are facing a variety of unknown situations as the scope of life enlarges when leaving childhood behind: They transition to new learning environments, they experience new social situations, and autonomy becomes a relevant development goal for them [START_REF] Gibbs Howard | Theories of adolescence[END_REF]. The challenges of growth are complex and adolescents have to deal with feelings of uncertainty. Spending less time in the close family circle, they have to evolve in more unknown environments [START_REF] Mcelhaney | Attachment and autonomy during adolescence[END_REF].

The need to adapt to new social interaction modalities while forging an adult identity represents a further challenge of this development stage [START_REF] Blakemore | The social brain in adolescence[END_REF][START_REF] Steinberg | A social neuroscience perspective on adolescent risk-taking[END_REF].

Adolescence is a critical age to develop self-esteem, motivation, goal setting and responsible action. At the same time, it is a period of fragility, characterized by a rapid cognitive, social and emotional development (World Health Organisation, WHO, 2017). Tensions can occur due to a widening gap between biological maturity and social transition to adulthood, and the struggle for an emerging need of autonomy and a strong attachment to the family [START_REF] Mcelhaney | Attachment and autonomy during adolescence[END_REF]. Vulnerability can also increase due to some aspects of globalization, such as gaming or online bullying (WHO, 2017). In this context, doubts tend to appear in the adolescents' self-concept (academic and social abilities) and their feeling of connection with their parents, peers and larger ecological context may decline.

A lack of self-confidence can have a particularly negative impact at this stage of the young person's development by limiting experience, willingness to try and perseverance, and by increasing vulnerability to self-doubt and undue influence of peers (Mann, Hosman, Schaalma & De Vries, 2004). Confidence in social interactions is highly relevant to the ability to form social networks [START_REF] Cheng | Personality, peer-relations, and self-confidence as predictors of happiness and loneliness[END_REF]. It has been shown that both self-confidence and interpersonal trust negatively correlate with loneliness [START_REF] Rotenberg | Loneliness and interpersonal trust[END_REF][START_REF] Cheng | Personality, peer-relations, and self-confidence as predictors of happiness and loneliness[END_REF] and thus affect the person's psychosocial wellbeing.

Increased emotional reactivity during adolescence may result in poor risk management [START_REF] Pfeifer | Adolescent social cognitive and affective neuroscience: past, present, and future[END_REF], with substance use or self-harm as a possible consequences [START_REF] Dorard | Estime de soi, soutien social perçu, stratégies de coping, et usage de produits psychoactifs à l'adolescence[END_REF]WHO, 2017). Since adolescence is recognized as the peak age of onset for mental ill-health [START_REF] Green | Mental health of children and young people in Great Britain[END_REF][START_REF] Zarobe | The role of arts activities in developing resilience and mental wellbeing in children and young people a rapid review of the literature[END_REF], it should also be the targeted moment of preventive measures that enhance mental health, resilience and coping strategies [START_REF] Zarobe | The role of arts activities in developing resilience and mental wellbeing in children and young people a rapid review of the literature[END_REF]. During adolescence, both risk and protective behaviours start or are consolidated, which can be decisive factors of future adult health (WHO, 2017). Building life skills and enhancing competences can help promoting good mental health [START_REF] Barry | Implementing Mental Health Promotion[END_REF]WHO, 2018). Adolescents' health and well-being are considered drivers of change in the attempt to create healthier and more sustainable societies (WHO, 2017). The Global Strategy for Women's, Children's andAdolescents' Health (2016-2030) even identifies adolescents as being central to achieving the United Nations' Sustainable Development Goals (Every Woman Every Child, 2015). Considering adolescents as powerful societal assets, efforts should be made to ensure they can thrive and are in a position to positively transform the communities in which they live. Under this perspective, positive behaviours that need to be promoted among adolescents also include constructive forms of risk-taking, such as sport or drama. The need to go beyond a problem oriented focus and to address protective factors is addressed in the 5 C model of positive youth development [START_REF] Lerner | Positive youth development[END_REF]. According to this model, positive adolescent interventions should target the development of Competence, Confidence, Connection, Character and Caring [START_REF] Lerner | Positive youth development[END_REF]WHO, 2017).

Recognizing the importance of the new generation's capacity to evolve with ease in a constantly changing society and to collectively build solutions to the challenges of transition, this paper focuses on the enhancement of self-confidence and trust through a creative storytelling workshop to be carried out with a non-clinical adolescent population. With the aim to provide a framework to practitioners to guide the design and facilitation of life-skill enhancing workshops, this paper explicits how the design of this intervention applies pedagogical mechanisms that specifically build self-confidence and trust. Based on a pilot implementation, this paper provides preliminary results regarding the impact of the intervention on relevant components of self-confidence and trust.

2.

Intervention design

Pedagogical mechanisms promoting self-confidence and trust

The intervention was designed as a creative workshop using storytelling techniques. Both creativity and storytelling are promising drivers of self-confidence and trust. In addition, the storytelling approach cultivates social connection and self-accomplishment during adolescence.

Shaping a positive society in an increasingly unpredictable and challenging world requires to use one's creativity (Gardner, 1999;[START_REF] Shaheen | Creativity and Education[END_REF][START_REF] Smears | A risky business: creative learning in education exploring ways to develop student teacher confidence[END_REF]. Besides, active participation in arts activities has a role to play in well-being and quality of life [START_REF] Clift | Creative arts as a public health resource: moving from practice-based research to evidence-based practice[END_REF]. Several programmes targeted at enhancing youth's resilience and mental wellbeing use arts-based interventions to promote the development of a sense of belonging and purpose, social inclusion, relationship building and the improvement of social skills and self-esteem [START_REF] Zarobe | The role of arts activities in developing resilience and mental wellbeing in children and young people a rapid review of the literature[END_REF]. Studies on the benefits of participating in arts activities commonly reveal an increase in self-confidence and self-esteem in teenagers [START_REF] Ennis | Youth Arts and Mental Health: Exploring Connections in the TOP End[END_REF][START_REF] Hauseman | Youth-led community arts hubs: Self-determined learning in an outof-school time (OST) program[END_REF][START_REF] Kemp | Promoting the health and wellbeing of young black men using communitybased drama[END_REF][START_REF] Salmon | Implementing the rock challenge: Young people's perspectives on a drug-prevention and performing-arts programme[END_REF][START_REF] Wood | To the beat of a different drum": improving the social and mental wellbeing of at-risk young people through drumming[END_REF]. A literature review on drama interventions in non-clinical settings [START_REF] Daykin | The impact of participation in performing arts on adolescent health and behaviour: a systematic review of the literature[END_REF] concluded that performing arts had impacts on self-confidence and social skills [START_REF] Douglas | Vital youth: Evaluating a theatre in health education project[END_REF][START_REF] Lasic | Theatre and peer education: An innovative approach to health promotion[END_REF], as well as enhanced peer interaction and cooperation (Walsh- [START_REF] Walsh-Bowers | Improving early adolescents' peer relations through classroom creative drama: An integrated approach[END_REF].

Moreover, arts-based activities represent an opportunity for people to explore and learn about themselves, and may thus be a positive vector of identity development [START_REF] Zarobe | The role of arts activities in developing resilience and mental wellbeing in children and young people a rapid review of the literature[END_REF]. Beyond creativity related to arts, creative skills also apply to cognitive domains such as creative problemsolving [START_REF] Williamson | The creative problem solving skills of arts and science students-The two cultures debate revisited[END_REF] and creative learning [START_REF] Jeffrey | Creative teaching and learning: Towards a common discourse and practice[END_REF]. A review study concludes that offering creative learning environments has shown to increase not only academic achievement, but also confidence and resilience of youths [START_REF] Davies | Creative learning environments in education -A systematic literature review[END_REF].

The mental health benefits of storytelling interventions have been documented in different studies, observing positive developments such as increased self-efficacy and decreased depression [START_REF] Esterling | Empirical foundations for writing in prevention and psychotherapy: Mental and physical health outcomes[END_REF][START_REF] Gortner | Benefits of expressive writing in lowering rumination and depressive symptoms[END_REF][START_REF] Pennebaker | Telling stories: The health benefits of narrative[END_REF]. In particular, telling their own story in a group setting has proven to generate positive feelings among participants, mainly related to the opportunity to describe their own experiences and having their voices heard [START_REF] Difulvio | Digital storytelling as a narrative health promotion process: Evaluation of a pilot study[END_REF]. Writing a story and sharing it with others provides a sense of accomplishment and pride among participants, and increases in selfconfidence have been observed regarding the capability to express one's story [START_REF] Difulvio | Digital storytelling as a narrative health promotion process: Evaluation of a pilot study[END_REF].

Within a storytelling activity, positive psychology can serve as a framework to guide self-reflection, using the classification of strengths and virtues that enable human thriving [START_REF] Seligman | Positive psychology progress: empirical validation of interventions[END_REF]. Identifying one's signature strengths and using them in a new way has shown positive effects on wellbeing, specifically increasing levels of happiness [START_REF] Seligman | Positive psychology progress: empirical validation of interventions[END_REF].

Increased confidence in one's ability to perform and in oneself more generally is closely related to the ability to communicate with others and to relate to other people [START_REF] Kemp | Promoting the health and wellbeing of young black men using communitybased drama[END_REF]. Accordingly, increased social connection has previously been observed as a consequence of a storytelling workshop, with participants reporting feeling cared for and supported by the group and the facilitator (DiFulvio, Gubrium, Fiddian-Green, Lowe & Del Toro-Mejias, 2016).

Creating and sharing a story not only provides an opportunity of being seen and valued, but it also allows to experience small successes in an active learning process. Research suggests that experiencing small successes helps developing a growth mindset, which in turn builds selfconfidence by perceiving challenge as an opportunity to learn [START_REF] Dweck | Academic Tenacity: Mindsets and Skills that Promote Long-Term Learning[END_REF].

Experiential learning has proven benefits on knowledge acquisition but also on the development of memory, creativity and sensitivity to acquire knowledge [START_REF] Efstratia | Experiential education through project based learning[END_REF][START_REF] Boud | Using Experience For Learning[END_REF]. First approaches on experiential learning date back to Dewey, who considered education a process of living rather than a preparation for later life [START_REF] Dewey | My pedagogic creed[END_REF]. It has been argued that in order to prepare for life experiences, pedagogical methods should foster self-confidence, cooperation and determination [START_REF] Stetson | An Essay on Kurt Hahn Founder of Outward Bound 1866-1974[END_REF]. It has been observed that effective youth programmes provide a context that empowers the participants to become active learners [START_REF] Durlak | A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents[END_REF][START_REF] Larson | Intrinsic motivation and positive development[END_REF][START_REF] Vandell | Children's organized activities[END_REF].

In order to maintain the participants' attention and active participation, it is crucial to establish a balance between task types during the intervention [START_REF] Wick | I Can Stand Learning: A Controlled Pilot Intervention Study on the Effects of Increased Standing Time on Cognitive Function in Primary School Children[END_REF].

Besides, alternating different types of activity also fosters creative thinking, providing stimulation for creative ideas and reflection to evaluate ideas and make them evolve [START_REF] Cramond | Nurturing creative thinking[END_REF][START_REF] Fairweather | Infusing creative and critical thinking into the curriculum together[END_REF][START_REF] Poon | The feasibility of enhancement of knowledge and self-confidence in creativity: A pilot study of a three-hour SCAMPER workshop on secondary students[END_REF].

Both self-reflection and creative risk taking require time, and creating a dedicated space of safety and immersion is essential in order to allow for a deep and significant experience. Establishing safe and caring learning environments involving peer initiatives and community-building activities has been identified as one of the components that foster young people's social-emotional development, associated to the experience of feeling valued and generating intrinsic motivation [START_REF] Durlak | The impact of enhancing students' social and emotional learning: A meta-analysis of school-based universal interventions[END_REF].

Mouvement is a driving force of immersion, connection and esteem. Movement enhances attention to the present and focus [START_REF] Taylor | The five facilitation game changers[END_REF]. Besides, movement exercises that include rhythm help the participants to connect with each other and to bond [START_REF] Taylor | The five facilitation game changers[END_REF]. Moreover, movement exercises that involve mirroring develop empathy, and they respond to the human need to be seen and appreciated [START_REF] Taylor | The five facilitation game changers[END_REF], a likely vector of self-confidence.

Research on an intervention involving performing arts has shown that increased confidence was not only associated to overcoming challenges and having a purpose, but also from feeling valued and being treated like an adult [START_REF] Ennis | Youth Arts and Mental Health: Exploring Connections in the TOP End[END_REF]. Thus, a central aspect of creative facilitation is adopting a posture that is exempt of judgement and that encourages self-expression.

Facilitators need to develop trust-building skills in order to foster positive relationships of authentic trust with the participants. Research has shown that youth's trust in programme leaders amplifies programme benefits, by increasing participants' willingness to take risks and thus opening them to new experiences, and by helping increase youth's agency [START_REF] Griffith | Why trust matters: How confidence in leaders transforms what adolescents gain from youth programs[END_REF].

A workshop design that integrates the pedagogical mechanisms

In the present workshop, participants were given the opportunity to invent stories, to audio-record them, and to edit the recordings in order to create a finalized podcast. The creative process integrated a strong component of self-reflection and self-expression, facilitated through a storytelling approach. Participants gathered in groups of three based on a signature strength they had in common and assigned it to the hero of their story. They created a scenario where the hero put his/her strength into action, and a second one where the hero used it to help out a hero from a fellow group. This procedure was designed to give participants the opportunity to become aware of their own possibilities through the experience of an imagined character. They got their inspiration for their fictional stories from a reflection on their own strengths and weaknesses, and on the values they stand for. Self-reflection and self-expression were facilitated through back and forth moves between reality and fiction during the course of the workshop. A series of exercises led participants from becoming aware of themselves and acknowledging their strengths and weaknesses, to progressive self-disclosure in front of the group. The creative storytelling workshop gave the participants the opportunity to express what made sense to themselves and to actively listen to fellow participants, under the pretext to fuel their stories. The groups shared their creations at several stages of the workshop, leading them to get used to speaking to an audience and to take creative risks.

The creative storytelling workshop was carried out in a format of three sessions lasting three hours each. The workshop thread was structured in a way that gave the participants opportunities for small, stepwise successes: Three cycles of collective creation were each followed by sharing to celebrate progress (cf. figure 1 : improvised drama performance of the story, playing the audiorecorded story, playing the finalized podcast).

A decisive element of the workshop was to create a safe space, where adolescents could

participate fully and open up to the experience they were offered. Furthermore, positive workshop dynamics were created by alternating playful and serious episodes. These included creativityenhancing activities, such as a theatre play and imagination exercises, as well as cooperation games and exercises that activate the body or the senses, and may therefore play a role in developing trust. These creative and physical activities were interspersed between phases of selfreflection, presentation to the group and productive teamwork to create the stories.

The workshop structure featured an alternation of plenary sessions and small group work. During the plenary sessions, group dynamics could be initiated and positive interactions among participants could be experienced. Subsequent phases of creation in small teams allowed the participants to reproduce the dynamics they had experienced in the plenary sessions, while creating their stories in a more intimate and more autonomous setting. This structure aimed at giving the participants the opportunity to gradually open up to themselves and others, experience interactions with fellow participants in more or less controlled settings, while enjoying co-creating the stories.

Essentially, the storytelling workshop was designed to offer the opportunity to experience situations of personal progress and positive interaction with others, leading to the achievement of a common goal. Feelings of self-confidence and trust are expected to increase through this experience, rather than through overt discussions on self-confidence and trust.

The workshop was led by two experienced facilitators. Creating a safe and caring environment does not only depend on the exercises that are implemented, but also on the way they are presented. The facilitators need to provoke immersion of all participants by finding a sensitive balance in guiding them : they should neither leave room for hesitations to come up, nor should they push the participants into a reluctant execution of the exercises. The facilitators should lead the participants through a sequence of opportunities to be creative and to take creative risks progressively, always making sure that a safe and caring environment is maintained within the group [START_REF] Taylor | Creative Facilitation 1 Training Manual[END_REF]. In that, the way the facilitator interacts with the participants influences the posture of the participants towards the facilitator and among themselves.

As described above, the facilitators should actively create positive group dynamics in the plenary sessions and leave room for the participants to self-adjust and reproduce the interactions they are comfortable with in the small teams. Working in groups of three with no proactive intervention of the facilitators gives the participants a space of freedom that fosters autonomy and accountability.

In sum, the expected impact of this workshop design is based on experiential learning and raised awareness of self-worth and caring connection among participants. Rather than evoking the concepts of self-confidence and trust, the workshop builds on the framework of positive psychology, progressive risk taking and self-disclosure, and the cooperative experience of small successes towards a shared goal. The role of the workshop structure and of its facilitators is to create a safe environment that allows participants to lean into their edges in a playful yet reflected and personal way. It furthermore provides a progressively increasing experience of autonomy throughout the workshop activities.

3.

The pilot study 3.1. Method

Study design and participants

A pre-experimental design including a pre-and post-measurement was applied with 12 participants. The participants (seven girls, five boys) aged from 13 to 15 years (mean (M) = 13.5, standard deviation (SD) = 1.0) were recruited on a voluntary basis through convenience sampling from a non-clinical setting. They were offered to participate in a creative storytelling workshop, and the objective of raising self-confidence and trust was not disclosed before the final debriefing at the end of the workshop. None of the participants did know more than one fellow participant before the workshop. The study was organized in accordance with the Declaration of Helsinki. The parents of the participants signed a parental consent form.

Procedure of the creative storytelling workshop

The creative storytelling workshop was held in three half-day sessions on three consecutive days in April 2017. The two workshop designers took the role of facilitators. They followed a previously established workshop thread, including the exercices to create a safe space, the creative storytelling activities and the impact measurement tools for data collection. Figure 1 details the content of the three sessions. The detailed workshop thread is available on request.

Data collection & analysis

At the beginning of the first workshop session and at the end of the third workshop session, quantitative data was collected with a set of continuous scales named the barometer. In addition, qualitative feedback was gathered from participants at the end of the workshop.

The barometer

The barometer was developed to assess self-report data concerning participants' self-confidence and trust. This giant-sized print-out barometer is composed of eight items, consisting in onesentence statements with continuous answering scales, that are horizontally represented below each other on a wall. Four items refer to self-confidence and four items to trust (table 1). The items specifically refer to the components of self-confidence and trust that have been identified as relevant for the capacity to evolve in a changing society and to collectively respond to its challenges.

Participants were asked to place a dot-shaped sticker on each answering scale so as to rate how well the statement describes that in the present moment. Verbal anchors of the scales were "That's absolutely not me" (left) and "That's totally me" (right). The stickers were personalized in colour and pattern, in order to assign the data to each participant without making their answers easily identifiable by the other participants.

The barometer was presented as part of the workshop, as a welcome activity for the pre-treatment measure and a closing activity for the post-treatment measure. This way, putting the participants into a situation of judgement was avoided and the activity was also used to raise awareness on the value of diversity and on the evolutive character of each person's development.

Quantitative data was subsequently transcribed by coding the continuous scale into a 10-point Likert scale. Pre-and post-measures were compared for each item using paired samples t-tests with SPSS Statistics 24. Effect sizes (Cohen's dz) were calculated with G*Power 3.1.9.4.

Table 1. Items of the barometer Item wording

Self-confidence I am able to deal with a lot of situations, even complicated ones.

I can learn how to deal with situations that I am not yet comfortable with.

When a situation is difficult, I keep it up.

I feel comfortable speaking out, even in front of people who do not agree with me.

Trust I enjoy learning from others.

I trust others when I can't manage alone.

For the most difficult situations, I prefer to be with others.

In a difficult situation, I let others help me.

Feedback of the participants

A closing exercise involving individual self-reflection and self-expression was used to collect participants' feedback just before the end of the workshop. Participants were, in a first place, asked to individually write on post-its what the workshop had brought them, and in a second time to present their feedback to the group. More specifically, they had to answer to complete the three following sentences: " I have overcome [...] ", " I felt gifted in [...] ", and " I am leaving with [...] ".

Results

Paired samples t-tests revealed significant differences between the pre-treatment measure and the post-treatment measure for the items "I can learn how to deal with situations that I am not yet comfortable with" (t(8) = -5.515, p = .001, Cohen's dz = 1.34; Table 2), "I feel comfortable speaking out, even in front of people who do not agree with me" (t(10) = -5.051, p < .001, Cohen's dz = 3.19), "I trust others when I can't manage alone" (t(10) = -4.457, p = .001, Cohen's dz = 1.18), and "In a difficult situation, I let others help me" (t(9) = -2.953, p = .016, Cohen's dz = 0.79). Concerning the item "I am able to deal with a lot of situations, even complicated ones", the difference between the pretreatment measure and the post-treatment measure tends towards significance (t(9) = -2.25, p = .051, Cohen's dz = 0.72). However, no significant difference was found between the pre-treatment measure and the post-treatment measure for the items "When a situation is difficult, I keep it up" (t(9) = -0.126, p = .903, Cohen's dz = 0.02), "I enjoy learning from others"(t(9) = -1.741, p = .116, Cohen's dz = 0.56), and "For the most difficult situations, I prefer to be with others" (t(10) = -1.747, p = .111, Cohen's dz = 0.33). The feedback of the participants suggests that the workshop increased their self-confidence, especially in terms of communication, technical and creative skills. Indeed, they declared they overcome : "the fear of talking to people", their "shyness", and the challenge of "speaking in front of everyone" and they felt gifted "in the use of the software", "in recording", "as an actress" and "creatively". Participants though that the workshop led them to collaborate with others: they met the challenge of the group work and they overcome the "reluctance to trust people". They also reported having "met new people" and "made friends" during the workshop. Finally, the participants had fun participating in the workshop: they declared having "good times" and left "in a good mood", with "good memories" and "happiness".

Discussion

The aim of this paper was to present the design of a creative storytelling workshop based on pedagogical mechanisms that specifically build self-confidence and trust in a non-clinical adolescent population. Through pilot data, this paper provided first evidence for the workshop's effects on components of self-confidence and trust that are particularly important in the current societal context.

The results of the pilot study show that the experience provided to the participants during the workshop positively impacts several of the measured components of self-confidence and trust.

Indeed, quantitative results reveal a statistically significant improvement at the end of the workshop for the self-confidence items relating to confidence in learning (growth mindset) and speaking, as well as the trust items referring to the willingness to rely on others in challenging situations. An improvement which tends to be significant has been observed for the selfconfidence item referring to generalized self-efficacy. The feedback of the participants was in line with these results, since they declared having acquired or improved personal and relational skills.

The concepts of self-confidence and trust were specifically mentioned by teenagers although the objectives of the workshop had not been revealed to them. In terms of their perceived achievements during the workshop, they did not only refer to their tangible results (e.g. production of stories) but underlined the effect of the experience on their personal growth (e.g. overcome their shyness).

The results regarding the evolution of confidence seem to relate to the experience of going beyond their comfort zone and gaining confidence in their capability to manage new situations, to learn new skills and especially to speak publicly (including self-disclosure). The impact of the workshop on trust appears to operate through the experience of succeeding collectively in an unfamiliar or challenging situation, presupposing the ability to accept help from peers.

The missing impact of the workshop on the item "For the most difficult situations, I prefer to be with others." can be attributed to a ceiling effect. This finding underlines the importance of collaboration amongst youth, who are reluctant to face challenges alone. It seems, however, that relying on concerted efforts to achieve goals does not necessarily go in hand with an appreciation of peerlearning ("I enjoy learning from others."). On the one hand, this finding suggests that the workshop does not provide sufficient experience in this regard so as to develop the participants trust in this dimension. On the other hand, the item might not be specific enough to measure trusting to learn.

As argued in the introduction of this paper, adolescence is a period in life that is characterized by new challenges and changing life context. Being able to step out of their comfort zone with confidence is essential to youth, in order to be able to venture into unknown environments. An experiential learning approach as the one presented in this paper seems to be an appropriate and effective means to develop their self-confidence.

The results of this study indicate that providing a collaborative and challenging creative experience to youth is a promising approach to build their generalised trust. This conclusion is in line with the sphere-specific logic of trust formation argued by [START_REF] Freitag | Spheres of trust: An empirical analysis of the foundations of particularised and generalised trust[END_REF], stating that generalised trust is not only based on psychological predispositions but also on the radius of trust building experiences. It supports that generalised expectancy to trust is built through social interaction and individual learning rather than being genetically predisposed [START_REF] Rotter | Generalized Expectancies for Interpersonal Trust[END_REF]. Thirdly, the workshop integrates several mechanisms specifically devoted to developing confidence and trust. However, the workshop cannot be considered as an explicit intervention given that the objective to develop confidence and trust was only revealed at the very end, and rather masked on purpose behind the more tangible objectives of collective creation of the stories.

Whether making the objective to develop confidence and trust explicit to the participants would enhance or hamper the effectiveness of the workshop is arguable.

Limitations and further research needs

The first limitation of this study is its reduced sample size. As a pilot study, it proves above all the feasibility of the workshop and indicates its potential of impacting confidence and trust positively.

A second limitation concerns measurements which rely exclusively on self-report data, making it subject to biases such as social desirability. However, the use of coloured stickers which were randomly assigned to the participants just before putting them on the barometer hampers at first glance individual identification and thus reduces the likelihood of shame-triggered answers. In order to conclude on the psychometric qualities of the barometer, in particular the reliability and construct validity, further studies with larger samples are required. It is also recommendable to further assess the effect of this type of workshops with other types of data, such as body language observations, more extensive questionnaires or semi-structured interview techniques. In addition, long-term effects and impact on participant's daily life have not been measured in this study. It would be interesting to investigate how stable the effects on confidence and trust are, and how they need to be consolidated through follow-up interventions. Moreover, the observed effects on confidence and trust can only be attributed to an interplay of the pedagogical mechanisms operating in the workshop, but the role or importance of each mechanism was not specified (e.g. creativity, storytelling approach, workshop structure). Likewise, the attitude and skills of the facilitators are likely to be decisive for the success of the intervention. It is therefore of importance to be trained for this type of intervention, especially since the creative storytelling workshop uses a bottom-up pedagogy traditional educators are not necessarily familiar with. Studies on skill training not only concluded that the trainer's interpersonal skills are important but also noted that the trainer should help youth generalize their developed skills to their everyday life [START_REF] Dusenbury | Eleven components of effective drug abuse prevention curricula[END_REF][START_REF] Gresham | Best Practices in Social Skills Training[END_REF]. Ideally, the workshop would be embedded into a larger programme that, beyond offering opportunities to build confidence and trust, would allow establishing deeper relationships between the adolescents and the facilitators, and that would build youths' engagement in activities through a programme routine and long-term structure (Yohalem, Wilson-Ahlstrom, Fischer, & Shinn, 2007).

Conclusion

This paper presents a creative storytelling workshop as a promising positive intervention to build the components of self-confidence and trust among adolescents that will help them thrive in today's society. Practitioners can find guidance to create and implement life skill enhancing workshops (especially those targeted at self-confidence and trust) that provide an impactful experience to the participants, by considering the relevant elements and pedagogical mechanisms exposed in this paper.

5.

Figure 1 :

 1 Figure 1 : Content of the three workshop sessions

  The creative storytelling workshop satisfies three out of four criteria of recommended training practices (SAFE -sequenced, active, focused, explicit), established by[START_REF] Durlak | A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents[END_REF] to qualify after-school programmes that seek to promote personal and social skills in children and adolescents. Firstly, the workshop uses a connected and coordinated sequence of activities to achieve skill development. Training outcomes could probably be enhanced if the workshop was embedded in a wider programme of targeted activities. Secondly, the workshop is almost exclusively based on active forms of learning to help adolescents develop confidence and trust.

Table 2 .

 2 Responses (means and standard deviations) to the items of the barometer at the beginning (pre)

	and the end (post) of the workshop
	Item

Acknowledgements

This research is based on a project that received non-commercial funding from Renault Trucks to develop and carry out the workshop and measure its impact.

Declarations of interest statement: none 6.

CRediT author statement


Gregory Kucherov

Steven Skiena

Improving the Sensitivity of MinHash Through Hash-Value Analysis

Keywords: 2012 ACM Subject Classification Theory of computation → Bloom filters and hashing MinHash sketching, sequence similarity, hashing Digital Object Identifier 10.4230/LIPIcs.CPM.2023.20

MinHash sketching is an important algorithm for efficient document retrieval and bioinformatics. We show that the value of the matching MinHash codes convey additional information about the Jaccard similarity of S and T over and above the fact that the MinHash codes agree. This observation holds the potential to increase the sensitivity of minhash-based retrieval systems. We analyze the expected Jaccard similarity of two sets as a function of observing a matching MinHash value a under a reasonable prior distribution on intersection set sizes, and present a practical approach to using MinHash values to improve the sensitivity of traditional Jaccard similarity estimation, based on the Kolmogorov-Smirnov statistical test for sample distributions. Experiments over a wide range of hash function counts and set similarities show a small but consistent improvement over chance at predicting over/under-estimation, yielding an average accuracy of 61% over the range of experiments.

Introduction

MinHash sketching is an important algorithm for efficient document retrieval. It reduces a set S of size n to a smaller representation of size m ≪ n by applying m distinct hash functions h 1 , . . . h m to each of the n elements of S, and identifies the smallest hash code for each h i . This vector of minimum hash codes serves a sketch for the larger set S. A classical result [START_REF] Broder | On the resemblance and containment of documents[END_REF][START_REF] Broder | Identifying and filtering near-duplicate documents[END_REF] shows that the probability that smallest hash codes of two sets S and T are equal is identical to J(S, T ), the Jaccard similarity of S and T . Thus the fraction of matching MinHash codes represents an unbiased estimator of J(S, T ). Hash code values, by definition, are not supposed to mean anything. They represent mappings of an item x to a pseudorandom integer h(x) for purpose of fast identity matching and retrieval. The relative values of h(x) and h(y) for items x and y have no special properties beyond that of h(x) = h(y) likely implies that x = y for the conventional hash functions as employed in algorithms such as MinHash.

But in this paper, we report a curious observation associated with MinHash. Suppose the MinHash values for sets S = {s 1 , . . . , s n } and T = {t 1 , . . . , t n } equal both a particular value, namely:

a = n min j=1 h i (s j ) = n min j=1 h i (t j )
We shall show that the value of this matching MinHash value a conveys additional information about the Jaccard similarity of S and T over and above the fact that the MinHash values agree.

This observation holds the potential to increase the sensitivity of minhash-based retrieval systems. Our main results in this paper are:

We explain why observing a larger matching MinHash value a increases the expectation of high similarity between S and T . Specifically, the expected value of a common MinHash value a for two n-element sets with intersection size i is N/(2n -i + 1), presuming the underlying hash function selects an integer from [0...N ] uniformly at random. We analyze the expected Jaccard similarity of two sets as a function of observing a matching MinHash value a under a reasonable prior distribution on intersection set sizes, specifically the case where pairs of n-element sets have equal probability of intersection size i for 1 ≤ i ≤ n. Experimental results confirm a modest increase in the sensitivity of our hash-code weighted variant of MinHash over the original, over a range of set similarities and number of hash codes. We present a practical approach to using MinHash values to improve the sensitivity of traditional Jaccard similarity estimation, based on the Kolmogorov-Smirnov statistical test for sample distributions. Our techniques provide a supplemental signal suggesting whether the fraction of matching MinHashes is more likely to over-estimate or underestimate the true Jaccard similarity between two sets. Experiments over a wide range of hash counts (k) and set similarities show a small but consistent improvement over chance, specifically an average accuracy of 61% over the range of experiments.

We believe that this orthogonal view of measuring Jaccard similarity through the value of matching MinHash codes is novel, and will inspire further interest. Although the practical improvement we have demonstrated is not large, we believe that better interpretations of the underlying statistics may yield better results.

This paper is organized as follows. We begin with a survey of the literature of MinHash and related techniques in Section 2. We provide intuition as to why the value of the matching MinHash value matters through a thought experiment in Section 3. We present our analysis of the expected intersection size as a function of MinHash value for an appropriate prior distribution in Section 4, and ways to combine this information into an estimate of Jaccard similarity in Section 5. An alternate approach to interpret the values of MinHash codes using the Kolmogorov-Smirnoff statistical test is presented in Section 6. Finally, we conclude with some open problems raised by our work.

Prior Work

Broder [START_REF] Broder | On the resemblance and containment of documents[END_REF][START_REF] Broder | Identifying and filtering near-duplicate documents[END_REF] developed MinHash as a solution to identifying similar documents (represented as sets of shingles or substrings) within a large text corpus while avoiding the quadratic-time costs of explicitly comparing each pair of documents. A function h(x) is applied to each set element, mapping each element x to a pseudo-random integer. Each set S is represented by the minimum hash value among all its elements. The Jaccard similarity J(S, T ) of two sets S and T is defined as

J(S, T ) = |S ∪ T | |S ∩ T | .
For identical sets, J(S, T ) = 1 while for disjoint sets, J(S, T ) = 0. Broder [START_REF] Broder | On the resemblance and containment of documents[END_REF][START_REF] Broder | Identifying and filtering near-duplicate documents[END_REF] observed that the probability that two sets S and T generate the same MinHash value exactly equals the Jaccard similarity of the two sets, J(S, T ). The fraction of matching minimum hashes over k independent hash functions provides an unbiased estimator of J(S, T ), with the variance of this estimate equal to J(S, T )(1 -J(S, T ))/k [START_REF] Broder | Min-wise independent permutations[END_REF]. Surveys of MinHash sketching include Cohen [START_REF] Cohen | Min-hash sketches: A brief survey[END_REF].

MinHash is one of the most important algorithms for web search and duplicate detection [START_REF] Fetterly | A large-scale study of the evolution of web pages[END_REF][START_REF] Henzinger | Finding near-duplicate web pages: a large-scale evaluation of algorithms[END_REF][START_REF] Jindal | Opinion spam and analysis[END_REF], social networks [START_REF] Chierichetti | On compressing social networks[END_REF], and machine learning [START_REF] Chi | Context-preserving hashing for fast text classification[END_REF][START_REF] Li | Hashing algorithms for large-scale learning[END_REF]. More recently, it has been successfully applied to bioinformatics for sketching large DNA sequence datasets, starting from the seminal Mash software [START_REF] Ondov | Mash: fast genome and metagenome distance estimation using minhash[END_REF] and followed by related tools [START_REF] Brown | Sourmash: a library for minhash sketching of dna[END_REF][START_REF] Zhao | Bindash, software for fast genome distance estimation on a typical personal laptop[END_REF][START_REF] Daniel | Dashing: fast and accurate genomic distances with HyperLogLog[END_REF][START_REF] Ondov | Mash screen: high-throughput sequence containment estimation for genome discovery[END_REF]. Such applications motivate our efforts in this paper to increase the sensitivity of minimum hashing-based similarity measures.

Locality Sensitive Hashing (LSH) techniques seek to map similar data objects to the same hash codes with a higher probability than the dissimilar ones by adopting a family of randomized hash functions. Indyk et al. [START_REF] Indyk | Approximate nearest neighbors: towards removing the curse of dimensionality[END_REF][START_REF] Gionis | Similarity search in high dimensions via hashing[END_REF][START_REF] Datar | Locality-sensitive hashing scheme based on p-stable distributions[END_REF] introduced the notion of locality-sensitive hashing in the context of nearest-neighbor search and string similarity. MinHash can be viewed an instance of locality sensitive hashing. An extended survey of locality-sensitive hashing can be found in [START_REF] Wang | Hashing for similarity search: A survey[END_REF].

SimHash [START_REF] Moses S Charikar | Similarity estimation techniques from rounding algorithms[END_REF] is a LSH-based method which provides an unbiased estimator of the similarity between two vectors. Specifically, the probability that two vectors u and v generate the same SimHash value equals the Cosine similarity of u and v. Henzinger [START_REF] Henzinger | Finding near-duplicate web pages: a large-scale evaluation of algorithms[END_REF] performed a large-scale comparison of MinHash and SimHash on detecting similar web pages, finding that a hybrid of the two approaches yielded the best results. Srivastava and Li [START_REF] Shrivastava | In defense of MinHash over SimHash[END_REF] present analysis and experiments to suggests that MinHash is more sensitive than SimHash in regions of high similarity.

Each subset element is granted equal weight in traditional MinHash schemes, but this can be generalized in weighted MinHash, perhaps to reflect the TD-IDF values of each word. Weighted MinHash algorithms are surveyed in [START_REF] Wu | A review for weighted MinHash algorithms[END_REF].

Finally, we mention another related sketching scheme named HyperLogLog [START_REF] Flajolet | HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm[END_REF] primarily designed for the task of estimating the number of distinct items in a stream, but also capable of estimating the cardinality of the union of two sets and therefore their Jaccard similarity. Several works proposed unifying combinations of the two sketches [START_REF] Ertl | Setsketch: Filling the gap between minhash and hyperloglog[END_REF][START_REF] William | Hyperminhash: Minhash in loglog space[END_REF]. Note that HyperLogLog has some relationship with our work, as it employs the idea of estimating the cardinality of a random set from its minimum value. However, a direct application of this idea to MinHash has not been made, to our knowledge.

Thought Experiment: Why MinHash Values Matter

We present the following thought experiment to illustrate how the actual value of matching MinHash codes provides information about Jaccard similarity. For a set S, let M h (S) denote its MinHash value under a given hash function h, i.e. M h (S) = min s∈S {h(s)}. We use the notation M (S) when h is irrelevant (but assumed fixed across sets). Now consider following two "extreme" situations involving pairs of sets S and T , each with n elements: 1. S and T are identical. Hence by definition, the minimum hash values must match, so

a 1 = M h (S) = M h (T ), 2.
S and T intersect in only one element x, which happens to be the minimum value of both under h, so

a 2 = h(x) = M h (S) = M h (S).
Now, given just the two unlabeled values for a 1 and a 2 , can we correctly assign these codes to the appropriate case above with probability greater than 1/2? Assume hash function h selects an integer from the range [0 . . . N ] uniformly at random. Now suppose that two n-element sets with intersection size i share a common MinHash value m. Then m is the smallest of the 2n -i values in the union. Since the expected minimum of ℓ numbers drawn uniformly at random from [0..N ] is N/(ℓ + 1), the expected value of m is N/(2n -i + 1). In the first case above, i = n, so E[a] = N/(n + 1), while for the second case i = 1 and E[b] = N/(2n). Thus it is more likely that min(a 1 , a 2 ) corresponds to case (1) and max(a 1 , a 2 ) to case [START_REF] Broder | On the resemblance and containment of documents[END_REF].

The situation is illustrated in Figure 1, which shows the probability of observing a given MinHash value a for three different intersection sizes. The probability of observing a MinHash value of 0 with a possible range [0 . . . 1000] is almost twice as high for two 100-element sets with a 20-element intersection than when the sets are identical. More similar pairs of sets, with larger intersection sizes, have greater probability of large matching MinHash values.

Expected Intersection Size as a Function of MinHash Value

In this section, we analyze the expected intersection size of two sets based on observing a particular matching MinHash value. For two n-element sets S and T where |S ∩ T | = i, J(S, T ) = i/(2n -i). Thus analyzing the intersection size of S and T provides a result which can be alternately interpreted in the context of the Jaccard similarity of S and T for n-element sets.

Let S and T be two sets each of size n. We limit our attention to the case where S and T are non-disjoint, which is necessary for MinHash values to legitimately collide, so S ∩ T ̸ = ∅. Further, we assume that range of h from [0 . . . N ] is sufficiently large relative to n that we can discount the possibility of spurious collisions, namely that there does not exist s ∈ S and t ∈ T where h(s) = h(t) despite s ̸ = t.

Prior Distribution

Determining the expected intersection size as a function of matching hash values requires knowledge of a prior distribution on the value of the intersection size. In the analysis below, we base our analysis on a uniform prior distribution, that all intersection sizes between S and T are equally likely. Thus for every

i ∈ [1..n], P[|S ∩ T | = i] = 1/n.
The uniform distribution appears most natural to us as a general prior, which is why we analyze this case below. That said, the true prior distribution differs with application, particularly as to whether pairs of randomly selected sets are likely to have large or small intersection sizes. The analysis below can be repeated for any particular well specified prior distribution in an analogous fashion.

Analysis

The probability of two sets sharing the MinHash value equals the Jaccard similarity index, that is

P[M (S) = M (T ) | i = |S ∩ T |] = i 2n -i . ( 1
)
Because all intersections are equiprobable under our prior distribution, we have

P[|S ∩ T | = i | M (S) = M (T )] = i 2n-i n j=1 j 2n-j (2)
Note that given ℓ random numbers x 1 , . . . , x ℓ uniformly drawn from [1..N ], for a ∈ [1..N ], we have

P[min{x 1 , . . . , x ℓ } ≤ a] = 1 -P[x 1 > a & . . . & x ℓ > a] = 1 -1 - a N ℓ .
(

Then, the probability the MinHash is exactly a is given by

P[min{x 1 , . . . , x ℓ } = a] = 1 - a -1 N ℓ -1 - a N ℓ . ( 4
)
We now estimate the conditional probability

P[|S ∩ T | = i | M (S) = M (T ) = a].
We have

P[|S ∩ T | = i | M (S) = M (T ) = a] = P[(|S ∩ T | = i) ∧ (M (S) = M (T )) ∧ (a = M (S ∪ T ))] P[(M (S) = M (T )) ∧ (a = M (S ∪ T ))] = P[|S ∩ T | = i] • P[a = M (S ∪ T ) | |S ∩ T | = i] • P[M (S) = M (T ) | (|S ∩ T | = i) ∧ (a = M (S ∪ T )] n i=1 (P[(M (S) = M (T )) ∧ (a = M (S ∪ T )) | |S ∩ T | = i] • P[|S ∩ T | = i]) = P[(a = M (S ∪ T )) | (|S ∩ T | = i)] • P[(M (S) = M (T )) | (|S ∩ T | = i) ∧ (a = M (S ∪ T ))] n i=1 P[(M (S) = M (T )) ∧ (a = M (S ∪ T )) | |S ∩ T | = i] (5)
The last rewrite follows because P[|S ∩ T | = i] = 1/n is the same for all i. To further simplify Eqn. 5, observe that

P[M (S) = M (T ) | (|S ∩ T | = i) ∧ (a = M (S ∪ T ))] = P[M (S) = M (T ) | |S ∩ T | = i]
because the event of sharing common MinHash (M (S) = M (T )) is independent of its value (a) for a fixed intersection size. For the same reason, in the denominator,

P[(M (S) = M (T )) ∧ (a = M (S ∪ T )) | |S ∩ T | = i] = P[M (S) = M (T ) | |S ∩ T | = i] • P[a = M (S ∪ T ) | |S ∩ T | = i]
Eqn. 5 then rewrites to Using ( 4), (1), we obtain

P[a = M (S ∪ T ) | |S ∩ T | = i] • P[M (S) = M (T ) | |S ∩ T | = i] n i=1 (P[M (S) = M (T ) | |S ∩ T | = i] • P[a = M (S ∪ T ) | |S ∩ T | = i]) . ( 6
P[|S ∩ T | = i | M (S) = M (T ) = a] = i 2n-i 1 -a-1 N 2n-i -1 -a N 2n-i n j=1 j 2n-j 1 -a-1 N 2n-j -1 -a N 2n-j (7)
Using [START_REF] Chi | Context-preserving hashing for fast text classification[END_REF], we can compute the expected intersection size as a function of the shared MinHash value:

E[|S ∩ T | | M (S) = M (T ) = a] = n i=1 i • P[|S ∩ T | = i | M (S) = M (T ) = a] ( 8
)
As an illustration, Figure 2 shows probability distributions of intersection sizes without taking into account the common MinHash value (formula (2)) and knowing the common MinHash value (formula [START_REF] Chi | Context-preserving hashing for fast text classification[END_REF]). The figure demonstrates that larger common MinHash values provide an evidence for larger intersection sizes.

Hash Scoring for Sketch Similarity

In the classical MinHash scheme, the probability that two sets have matching MinHash is equal to the Jaccard similarity between the two sets. Thus, the fraction of matches taken over a number of trials provides an unbiased estimator of the Jaccard similarity. We have shown that the values of these matching MinHashes provides an orthogonal measure of similarity.

The question is what the best way to combine these measures is. We propose the following initial strategy. Traditional MinHash can be interpreted as averaging the values of 0/1 indicator variables, where a match of hash codes is represented by 1 and a mismatch by 0. We will replace the value associated with matching hashes by real values that over/underweight based on the value of shared MinHash. More specifically, a shared MinHash value will contribute with weight where the numerator is defined by Eqn 8, and

E[|S ∩ T | | M (S) = M (T ) = a] E[|S ∩ T |] , ( 9
)
E[|S ∩ T |] = n i=1 P(|S ∩ T | = i | M (S) = M (T )) (10)
is the expected intersection size independent of MinHash value, i.e. implied by the prior distribution of intersection sizes.

Experimental Results

We performed a modest experiment to evaluate the performance of this technique, with the results reported in Table 1. We limited the experiment to small sets (n = 20), but consider a broad range of hash function counts (1 ≤ k ≤ 20 = n) and set similarities defined by intersection sizes from 1 ≤ i ≤ 20 = n. Each cell represents the average difference in absolute error in estimating intersection size between MinHash with over/underweighting and traditional 0/1 counts, where each cell is averaged over 1,000 independent random trials. We note that the rightmost column in Table 1 (intersection size 20 out of a possible 20) corresponds to identical sets, where the traditional Jaccard (and intersection size) estimate is always correct, leaving our proposed method with no room for possible improvement. But 116/140 = 82.9% of the non-trivial cells show improvement over the traditional MinHash baseline.

C P M 2 0 2 3 20:8

Improving the Sensitivity of MinHash Through Hash-Value Analysis Figure 3 The Kolmogorov-Smirnov test quantifies the difference between two probability distributions by the maximum y-distance gap between the two cumulative distribution functions. On the left, two samples from the same normal distribution. On the right, comparison of samples from uniform and normal distributions drawn over the same 6

Sketch Evaluation using the Kolmogorov-Smirnov Test

We now propose an alternate approach to improve the Jaccard similarity estimate offered by the classical MinHash approach, namely the fraction of matching MinHash values in k trials. We seek to improve this estimate by analyzing the distribution of the values of the matching hashes from these trials to decide whether it is more likely to be over or under estimating the actual similarity. A key advantage of this approach over of that of Section 4 is that does not require a prior distribution on the actual intersection sizes.

Our approach is based on the Kolmogorov-Smirnov (KS) statistical test [START_REF] Kolmogorov | Sulla determinazione empirica di una lgge di distribuzione[END_REF][START_REF] Smirnov | Table for estimating the goodness of fit of empirical distributions[END_REF], which compares empirical cumulative distribution functions (CDFs) to assess whether two samples are drawn from the same underlying distribution. We will use it to compare the observed distribution of matching MinHash values against the theoretical distribution for the classical Jaccard estimate. The direction of the largest deviation suggests whether it is more likely an over or under estimate.

The Kolmogorov-Smirnov Test

In the KS-test, the empirical cumulative distribution function (CDFs) of the two different samples are plotted on the same chart. If the two samples are drawn from the same distribution, the ranges of x values should largely overlap. An empirical CDF F (x) of a sample is defined as the fraction of the sample ≤ x.

We seek to identify the value of x for which the associated values of the two CDFs differ by as much as possible. The distance D( F , Ĝ) between two empirical CDFs F and Ĝ is the difference of the y values at this critical x, formally stated as

D( F , Ĝ) = max x | F (x) -Ĝ(x)|
The more substantially that two samples differ in this fashion, the more likely it is that they were drawn from different distributions. Figure 3 (left) shows two independent samples from the same normal distribution. In contrast, Figure 3 (right) compares a sample drawn from a normal distribution against one drawn from the uniform distribution. The big gaps near the tails provide evidence that the two samples are drawn from different distributions.

The KS-test compares the value of D( F , Ĝ) against a particular target, declaring that two distributions differ at the significance level of α when:

D( F , Ĝ) > c(α) n 1 + n 2 n 1 n 2
where c(α) is a distribution-independent constant to look up in a table. In this paper, we use the ideas behind the KS-test for qualitative evaluation instead of precisely measuring statistical significance, and so will be interested in the direction of the deviation without this associated constant.

Application to MinHash Analysis

As explained above, the distribution of matching MinHash values differs as a function of the intersection size or (equivalently) Jaccard similarity between two sets of size n. Recall that for ℓ random numbers x 1 , . . . , x ℓ uniformly drawn from [1.

.N ], for a ∈ [1.

.N ], we have

F ℓ (x) = P[min{x1, . . . , x ℓ } ≤ x] = 1-P[x1 > x & . . . & x ℓ > x] = 1-1 - x N ℓ ≈ 1-e -xℓ/N . ( 11
)
This defines the CDF on matching MinHash values. Comparing two sets A and B, both of cardinality n with an intersection of size i, any common MinHash value represents the smallest of ℓ = 2n -i random values. Thus the distribution of matching MinHash values is defined by Eqn. 11, given an estimate for the union size ℓ. An important observation for us is that CDFs F ℓ are majorating one another, that is if

ℓ 1 > ℓ 2 , then F ℓ1 (x) > F ℓ2 (x) for any x.
Estimates for the union size ℓ and intersection size i follow from classical MinHash analysis. If m matching MinHash values are observed in k trials, m/k is an unbiased estimator of the Jaccard index i ℓ = i 2n-i = 2n-ℓ ℓ . Therefore, i and ℓ are estimated respectively by

î = 2nm k + m , l = 2nk k + m .
We can now employ the idea underlying the KS-test to evaluate how well the m observed MinHash values match the estimated distribution F l(x). In doing that, we analyze the sign of the critical deviation

D(F l, F ) = F l(x) -F (x) for x = argmax |F l(x) -F (x)|,
where F is the empirical CDF obtained from the sample of matching MinHash values. When D is positive, this suggests that the regular MinHash estimate l is an overestimate and therefore î is an underestimate for the true intersection size. Conversely, a negative D provides an evidence that î is an overestimate for the true intersection size. This reasoning is supported by the above-mentioned majorating property of CDFs, as it guarantees that the sign of D correctly defines whether the estimate l should be increased or decreased for the KS-test statistic to be reduced and therefore for the estimated CDF to fit better the observed MinHash values. We thus propose the sign of D as a secondary signal to improve the accuracy of î as an estimator for intersection size.

The running time of this test is O(m log m) because we must sort the observed matching hash values to compute the CDF. It is only necessary to compare the distributions at the m sample points to identify the extremal points, with each comparison efficiently done using the exponential form of Eqn. 11. The magnitude of the deviation directly maps to a confidence value in the direction of change, with p-values obtainable using tables of c(α) values from the standard KS-test. However, in the experiments below we propose to estimate correction direction from the sign of D independent of its magnitude.

C P M 2 0 2 3

Experimental Results

Table 2 summarizes the performance of our KS-based correction strategy over a wide range of hash counts (from k = 1 to k = 1000) and true Jaccard similarity (from 0.081 to 0.961).

For each Jaccard similarity level, we constructed 10,000 pairs of 1000-element sets, each pair constructed to the appropriate level of similarity. We then constructed k independent hash functions of these sets, and determined the number of matching MinHashes for these trials. We then performed the KS-test on the matching values to propose whether the actual Jaccard estimate should be higher or lower than the observed fraction of matches. We chose parameters of our tests (intersection size) so that to avoid the situation when the MinHash estimate exactly equals the true Jaccard similarity, making each case a fair binary trial. Of the 14 × 12 = 172 entries in Table 2, 144 of them (83.7%) are greater than 0.5, meaning the adjustment breaks in the correct direction more often than not. The average accuracy ratio taken over all trials is 61.0%, substantially better than the baseline of 50%.

When employing large numbers of hash functions k ≥ 100, our technique improves the estimate on average in 57 of 60 (95%) entries, and proves most beneficial in the middle regions where the Jaccard similarity is ≈ 0.5. This is curious, because larger k provides greater resolution on the fraction of matching hash values, thus reducing the quantization error of classical MinHash. But the KS-analysis also improves with more samples as k increases, and continues to refine the similarity estimate even as k = 1000. Presumably in the limit as k grows, the improvement over baseline will disappear, but it seems durable over the range of k that appear in general applications.

That our best (and worst) performance occurs for very small k reflects issues of quantization: for an intersection size of n/2 and k = 5, the best possible estimate must be wrong by at least 10%. As a statistical significance test, the KS-test was designed to be used with a meaningful number of samples per observed distribution. There are likely other statistical tests to do better with small (and maybe even large) values of k. Conclusions

We have demonstrated that the value of matching MinHash values provides additional information on the degree of similarity between pairs of sets. Our wins are small, but they are real. We believe that there exist better methods of integration to synthesize the mix of the number of matching hashes and their values into a more accurate measure of similarity and believe that this is a research direction worth pursuing. We note that even careful analysis of the values of the matching hash codes will be substantially less computationally expensive than that of obtaining the MinHash codes themselves, so these improvements will come at a little computational cost.

The MinHash values that do not match also contain some degree of signal concerning the similarity of two sets. Suppose the smallest hash values of two sets do not match, but are both unusually large, say a substantial fraction of the total range N . These large MinHash values signify that both sets exclude the same large fraction of possible elements from the universe, implying they must both be constructed from just a relatively small set of non-excluded elements. This conditioning increases the expected Jaccard similarity, despite the fact that the hash values do not match. We believe this signal to be very weak except in extreme cases, but its analysis may be part of a complete solution.

The theoretical success of MinHash depends strongly upon the elements of the sets being distinct. If multiplicity of elements should be taken into account, one should resort to the weighted variant of MinHash [START_REF] Wu | A review for weighted minhash algorithms[END_REF]. Extending our ideas to Weighted MinHash is another interesting direction of study for the future.

Figure 1

 1 Figure 1 Probability distributions that two sets of size 100 share a common MinHash value, as a function of the size of their intersection (respectively 20, 50, and 100). The probability of small matching MinHash values are increase for relatively dissimilar sets.

)Figure 2

 2 Figure 2 The probability of intersection size of two sets of size 100 sharing a common MinHash value. The red curve shows the probability of having a given intersection size (formula (2)). The other curves show the same probability conditioned on the value a of common MinHash (formula (7)), where the hash space is [1..1000]. Larger values of a favor larger intersection sizes.

Table 1

 1 Improvement (in terms of the average reduction of absolute error) in estimating set intersection size by summing hash-weighted counts vs. equal weighting to estimate Jaccard similarity for different numbers of hash functions (rows) and set intersection sizes (columns). Bolded entries represent improvement over traditional MinHash estimation, representing 116/140 = 82.9% of the non-trivial cells in the table.

	k / int	1	2	3	5	10	15	19	20
	1	.0068 .0140 .0205 .0375 .0837 .1150 .1550 -.1650
	2	.0067 .0138 .0395 .0583 .0626 .1020 -.1120
	3	.0092 .0250 .0258 .0466 -.0914 .0369 .0824 -.0833
	4	.0128 .0250 .0308 .0496 .0042 .0093 .0648 -.0615
	5	.0161 .0240 .0312 .0493 .0280 -.0799 .0684 -.0532
	6	.0163 .0251 .0377 .0537 -.0471 .0012 .0550 -.0489
	7	.0116 .0301 .0368 -.0277 -.0008 .0071 .0636 -.0384
	8	.0143 .0257 .0356 .0119 .0282 -.0072 .0615 -.0347
	9	.0159 .0201 .0350 .0200 -.0377 .0018 .0298 -.0325
	10	.0139 .0283 .0416 .0233 .0029 -.0411 -.0146 -.0290
	11	.0155 .0266 .0476 .0266 .0156 .0028 -.0166 -.0244
	12	.0147 .0262 .0218 .0350 -.0172 -.0023 -.0004 -.0231
	13	.0165 .0353 .0016 .0403 .0133 -.0044 .0035 -.0213
	14	.0176 .0283 .0157 -.0133 .0162 -.0035 .0070 -.0193
	15	.0168 .0280 .0125 .0077 -.0207 -.0226 .0148 -.0172
	16	.0164 .0291 .0142 .0173 .0081 -.0003 .0059 -.0155
	17	.0189 .0328 .0196 .0115 .0113 -.0045 .0084 -.0144
	18	.0163 .0293 .0193 .0207 -.0157 .0071 .0058 -.0144
	19	.0152 -.0060 .0208 .0304 .0043 .0036 .0092 -.0118
	20	.0146 .0094 .0213 .0289 .0059 -.0193 -.0122 -.0115

Table 2

 2 Performance (in terms of the fraction of correct direction predictions) of the KS-testbased over/under correction, as a function of the number of hash functions k (shown in left column), and the true Jaccard similarity/intersection size (shown in first/second row). Generally speaking, the improvement is greatest at extreme values of similarity (either high or low), and with smaller numbers of hash functions.

	Jaccard	.961 .869 .786 .739 .667 .538 .429 .333 .258 .176 .111 .081
	intersect	980	930	880	850	800	700	600	500	410	300	200	150	Avg
	1	1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
	2	.816 .696 .596 .534 .457 .339 .988 .994 .996 .998 .999 .998 .784
	3	.764 .618 .485 .419 .275 .552 .445 .340 .983 .992 .994 .998 .655
	4	.711 .556 .440 .651 .561 .388 .570 .475 .361 .982 .989 .993 .640
	5	.683 .519 .632 .572 .478 .551 .394 .551 .450 .974 .985 .993 .649
	10	.591 .577 .590 .537 .566 .505 .474 .452 .483 .561 .399 .966 .558
	20	.502 .540 .565 .573 .545 .564 .535 .528 .456 .460 .395 .514 .515
	50	.506 .550 .539 .567 .550 .560 .549 .544 .541 .520 .474 .396 .525
	75	.518 .535 .562 .552 .508 .553 .550 .492 .515 .506 .487 .431 .517
	100	.512 .552 .556 .561 .553 .578 .552 .552 .542 .525 .468 .454 .534
	200	.525 .549 .550 .557 .563 .568 .555 .557 .543 .527 .505 .491 .541
	300	.522 .551 .558 .556 .535 .561 .550 .530 .543 .536 .515 .506 .539
	500	.529 .550 .550 .559 .556 .574 .558 .553 .543 .534 .521 .511 .545
	1000	.520 .552 .556 .567 .561 .565 .557 .548 .550 .540 .512 .512 .545
	Average	.621 .596 .584 .586 .550 .561 .591 .580 .608 .690 .660 .697 .610

Funding Steven Skiena: Partially supported by a travel grant from Université Paris-Est, and NSF grant IIS-1927227.