Pauline Rasset
email: pauline.rasset@unicaen.fr

Benoı ˆt Montalan

Jessica Mange

Only human after all? a pre-registered study on gaze behavior and humanity attributions to people with facial difference

There is a great deal of indirect evidence suggesting that people with facial difference (FD) may be dehumanized. This research aimed to provide direct evidence of the dehumanization of people with FD based on the stigmatizing reactions they elicit. More precisely, previous findings revealed that the specific way people with FD are looked upon is related to the feelings of disgust they elicit. Since disgust fosters dehumanization, our aim was to confirm the modified pattern of visual attention towards people with FD and to determine whether it was also related to humanness perception. For that purpose, a preregistered eye-tracking study (N = 97) using a former experimental design extended to humanity attributions was conducted. This research replicates findings showing that the face of people with FD is explored differently in comparison with other human faces. However, the hypothesis that people with FD were given fewer humanity attributions was not supported. Therefore, the hypothesis of a "dehumanizing gaze" towards people with FD-beyond humanity-related attributions-is discussed in light of these findings.

Introduction

People living with a facial difference (FD; also known as "facial disfigurement") are stigmatized [START_REF] Rasset | Towards a better understanding of the social stigma of facial difference[END_REF], which leads them to feel dehumanized (i.e., denied a full humanness status; [START_REF] Haslam | Dehumanization and Infrahumanization[END_REF]). Indeed, they report feeling like they are not treated akin to their nonvisibly different counterparts (e.g., [START_REF] Bogart | People are all about appearances": A focus group of teenagers with Moebius Syndrome[END_REF]). Relatives of people with visible FD also report witnessing their next-of-kin being treated as diminished human beings (e.g., witness statements from parents of children with birthmarks, see [START_REF] Kerr | Memorable Messages Parents of Children with Vascular Birthmarks Receive from Others: Implications for Stigma and Identity[END_REF]). Although the concepts of stigma and dehumanization share some commonalities [START_REF] Goffman | Stigma: Notes on the management of spoiled identity[END_REF], to our knowledge, no previous study has directly investigated the dehumanization of people with FD seen from the perspective of their perceivers. Thus, the aim of this research work was to determine whether the stigmatization of people with FD could be related to their dehumanization.

Dehumanization of people with facial differences

Two broad types of dehumanization manifestations have been described: dehumanization as a content of social perception (e.g., the attribution of nonspecific human characteristics) and dehumanization as a form of perception (i.e., the reduction of people to their constituent parts) [START_REF] Bernard | Objectifying objectification: When and why people are cognitively reduced to their parts akin to objects[END_REF]. There are several models of dehumanization as a content of social perception with a general agreement on two basic dimensions of humanness attributions [START_REF] Li | Toward a comprehensive taxonomy of dehumanization: Integrating two senses of humanness, mind perception theory, and stereotype content model. TPM-Testing[END_REF]. More precisely, people can be dehumanized when they are denied uniquely human (UH) characteristics (e.g., refinement; i.e., characteristics that distinguish human beings from nonhuman animals) and/or characteristics that are at the essence of human nature (HN; e.g., moral sensitivity; i.e., characteristics that distinguish human beings from inert entities) [START_REF] Leyens | The Emotional Side of Prejudice: The Attribution of Secondary Emotions to Ingroups and Outgroups[END_REF][START_REF] Haslam | Dehumanization: An Integrative Review[END_REF]. In parallel, people can also be dehumanized in that they are perceived in a perceptually different manner than other human beings [START_REF] Hugenberg | The Face of Humanity[END_REF].

There is-to our knowledge-no evidence of lesser attributions of humanness to people with FD. However, some indirect findings support the assumption of a dehumanized perceptual processing of people with FD. First, a research team investigating the neural responses to faces with FD found diminished responses within a brain region implicated in social cognition [START_REF] Hartung | Behavioural and Neural Responses to Facial Disfigurement[END_REF], as also evidenced for highly dehumanized people (e.g., homeless people [START_REF] Harris | Dehumanizing the Lowest of the Low[END_REF]). Second, there is evidence suggesting that humanness perception is rooted in holistic face perception [START_REF] Hugenberg | The Face of Humanity[END_REF], which is impeded for faces with FD [START_REF] Boutsen | Do facially disfiguring features influence attention and perception of faces? Evidence from an antisaccade task[END_REF]. Third (and accordingly), research conducted with event-related potentials showed a reduced amplitude of the N170 -a component involved in holistic processing-when people are exposed to faces with FD in comparison to when they are exposed to faces without FD [START_REF] Huffmeijer | Changes in face-specific neural processing explain reduced cuteness and approachability of infants with cleft lip[END_REF]. Finally, attention to the eyes is related to humanizing modes of perception [START_REF] Fincher | Social antecedents and perceptual consequences of how we look at others[END_REF], but for faces with FD, gaze behavior is focused on the FD and drawn away from the eyes [START_REF] Rasset | Look me in the eyes! A pre-registered eye-tracking study investigating visual attention and affective reactions to faces with a visible difference[END_REF]. Interestingly, this specific pattern of gaze behavior seems to play a key role in the stigmatization of people with FD.

Gaze behavior as an indicator of the dehumanization of people with a facial difference?

Madera and Hebl [START_REF] Madera | Discrimination against facially stigmatized applicants in interviews: An eye-tracking and face-to-face investigation[END_REF][START_REF] Madera | To Look or Not to Look: Acknowledging Facial Stigmas in the Interview to Reduce Discrimination[END_REF] revealed that the focus on FD biases memory, which in turn fosters discrimination. More recently, Rasset and colleagues [START_REF] Rasset | Look me in the eyes! A pre-registered eye-tracking study investigating visual attention and affective reactions to faces with a visible difference[END_REF] showed that the interaction between the focus on the FD and the lesser attention given to the eyes was correlated with a heightened affective reaction of disgust. Concretely, they asked their participants to look at faces with and without a burn-like disfigurement on the cheek. They revealed that the more participants looked at the FD, the heightened their stigmatizing reaction of disgust was, but only when they looked little at the eyes area. Since disgust is a precursor of dehumanization [START_REF] Giner-Sorolla | Not Just Disgust: Fear and Anger Also Relate to Intergroup Dehumanization[END_REF], including the dehumanization of people with facial differences [START_REF] Landry | Filthy animals: Integrating the behavioral immune system disgust into a model of prophylactic dehumanization[END_REF], it is thus possible that this stigmatizing gaze could also be a dehumanizing gaze.

Thus, the aims of this research were twofold: 1) replicate previous eye-tracking research showing a stigmatizing gaze with a focus on the FD leading to the neglect of the eyes region [START_REF] Rasset | Look me in the eyes! A pre-registered eye-tracking study investigating visual attention and affective reactions to faces with a visible difference[END_REF], and 2) determine if this stigmatizing gaze could be related to the dehumanization of people with FD.

Method

For more details on the method, see the supporting information.

Ethical considerations

Ethical approval was obtained from the local institutional review committee of the laboratory of CRFDP-University of Rouen Normandy (N˚2020-06-A). Every participant gave twice 4bd75d4a8d3045b59fa2a75c791ffa81. The design and analysis plans for the experiment were preregistered on OSF and can be accessed with this link: https://osf.io/grytk.

PLOS ONE

(before and after the experiment) their written informed consent to participate in the study.

No identifying information was recorded.

Participants

One hundred thirteen participants volunteered to participate in the study without any financial counterpart. Twelve participants were withdrawn from the study because of calibration errors, and four were withdrawn because of disfigurement. The remaining 97 participants comprised 65 females, 30 males, and 2 nonbinary participants (M = 19.61 years; SD = 3.01). G*Power [START_REF] Buchner | Power 3.1 manual. Du ¨sseldorf[END_REF] was used to calculate an a priori sample size, using a standard parameter of α = .05, a power 1 -ß = .80, for a small effect of ρ = .25 [START_REF] Rasset | Look me in the eyes! A pre-registered eye-tracking study investigating visual attention and affective reactions to faces with a visible difference[END_REF]. This study was preregistered at https://osf.io/grytk. Data were collected from January to September 2021. Data were consulted for research purposes once they had been fully collected.

Procedure, material and measures

All participants were recruited directly on the campus by the experimenter and volunteered to participate. First, a Tobii X120 Eye-tracker (SR Research Ltd.) was calibrated. Each trial consisted of the following sequence: a fixation cross in the center of the screen displayed for 2 seconds, then a face displayed for 5 seconds, and then a question asking for humanness attributions.

The faces were presented either in their original condition or with an FD. Eight faces were used as stimuli: four different faces presented either in their original condition or with an FD. Participants were randomly exposed to one of two sets of four faces. In each set, participants could see two faces with FD and two faces without FD; each time, one face belonged to a woman, and the other to a man. For the two faces presenting an FD, one of them had an FD pattern on the left cheek, while the other had an FD on the right cheek. Each face was presented either in its original or in its "disfigured" condition depending on the set of faces.

Each face was presented six times in fixed random order, once for each attribute, thus bringing the total of trials to 24. Each trial lasted approximately 10 seconds. After providing sociodemographic information, they were then fully debriefed, gave a second a posteriori informed consent and were finally thanked.

The length and location of the participants' visual fixations on five separate nonoverlapping areas of interest (AoIs; i.e., eyes, nose, target cheek, other cheek, and lips) were recorded (based on [START_REF] Fincher | Social antecedents and perceptual consequences of how we look at others[END_REF]; see S1 Fig in S1 File). For each AoI, the total dwell time (i.e., the sum of all dwell times for the same AOI over a trial) and the time to first fixation (i.e., the time period from entering the AOI until the first fixation is made) were recorded.

Dual Model Questionnaires were used to measure humanness-related attributes (based on the work of Haslam et al., 2008 [22]). Based on a pilot study (see supporting information), only UH traits (α = .74) and HN traits (α = .61) were selected. Participants were asked to report "the extent to which most people seeing the person on the photograph may think that the person is" the written trait on a numeric scale of 1-7 (corresponding to: 1-'totally disagree', 4 -'neither agree nor disagree', 7 -'totally agree').

Statistical analysis

First, the hypothesis of an impact of disfigurement on visual attention was tested. For that purpose, a 2 (Type of face: original, with FD) X 5 (AoI: eyes, nose, target cheek, other cheek, lips) repeated-measures ANOVA was run on each eye tracking variable. Greenhouse-Geisser corrections were used to correct the analyses that violated the assumption of sphericity. Second, the assumption of the impact of the FD on humanness-related attributes was tested. For that purpose, 2 (Type of face: original, with FD) X 2 (Humanness-related attribute: UH, HN) repeated-measures ANOVA was run. Finally, the relationships between human attributes and visual attention were explored through correlations and linear regressions. For that purpose, for all measures, the scores were computed by considering the difference mean score of the faces with disfigurement minus the original faces mean score (for a similar procedure, see [START_REF] Stone | Emotional responses to disfigured faces and Disgust Sensitivity: An eye-tracking study[END_REF]).

Results

Impact of facial differences on gaze behavior

There was a main effect of AoI on the total dwell time and an interaction effect between type of face and AoI (see Table 1 for details). As expected, simple effects tests revealed that the target cheek was fixated longer when the face presented an FD than when it was presented in its original condition (see S2 Fig in S1 File). The opposite pattern emerged for the eyes, nose and lips. A larger effect size was observed for the eyes as opposed to the nose and the lips.

There were main effects of type of face and AoI on the time to first fixation and an interaction between type of face and AoI (see Table 1 for details). As expected, simple effect tests revealed that the target cheek was stared at quicker when the face had an FD than when it was presented in its original condition (see S3 Fig in S1 File). The opposite pattern emerged for the lips and nose, with a larger effect size being observed for the lips.

Impact of disfigurement on humanity attributions

There was neither a main effect of Type of face (F(1, 96) = 2.61, p = .10) nor a main effect of Humanness-related attribute (F(1, 96) = 2.03, p = .16). However, there was an interaction effect between those two variables (F(1, 96) = 10.72, p = .001, η 2 G = .01), showing that the participants attributed more HN traits to faces with a disfigurement (M = 4.56, SD = 0.96) than to faces in their original condition (M = 4.19, SD = 0.87, p Bonferroni < .05), whereas no such difference was observed for UH traits (faces with disfigurement: M = 4.44, SD = 0.94; in original settings: M = 4.48, SD = 0.85; p Bonferroni = .99). This pattern of results is inconsistent with our hypothesis of a dehumanization of people with FD.

Relationship between visual attention and humanity attributions

Neither correlation between total dwell time towards any areas of the face and the humannessrelated traits-, i.e., for UH: -.14 < r < .19, p >.06; for NH: -.11 < r < .15, p >. [START_REF] Boutsen | Do facially disfiguring features influence attention and perception of faces? Evidence from an antisaccade task[END_REF] -nor correlation between time to first fixation towards any areas of the face and the humanness-related traits-, i.e., UH: -.10 < r < .17, ps >.09; for NH: -.10 < r < .17, ps >.09 -reached significance (see S1

Discussion

This research replicates previous findings showing that faces with FD engender a significantly different pattern of gaze behavior in comparison to faces without FD [START_REF] Rasset | Look me in the eyes! A pre-registered eye-tracking study investigating visual attention and affective reactions to faces with a visible difference[END_REF]. Put differently, the presence of FD seems to disrupt the regular way in which people gaze at human faces [START_REF] Vo | Do the eyes really have it? Dynamic allocation of attention when viewing moving faces[END_REF]. However, there was no evidence of lesser attributions of humanness-related traits to faces with FD, which hampers our dehumanizing gaze hypothesis. In other words, this research confirms that faces with FD are significantly differently gazed at, but there was no evidence of their dehumanization. Thus, this discussion will explore some possible explanations, in line with this lack of evidence of dehumanization along with the limitations of this study in regard to the measures, the method, and the hypotheses.

First, it is possible that the measures of dehumanization as a part of social perception cannot capture the dehumanization of people with FD, or at least the scale we have chosen. Indeed, in the past, research has shown that self-reported evaluations of people with FD are biased by social desirability [START_REF] Stone | Emotional Responses to Disfigured Faces: The Influences of Perceived Anonymity, Empathy, and Disgust Sensitivity[END_REF], which may also have biased humanness attributions [START_REF] Enock | No convincing evidence outgroups are denied uniquely human characteristics: Distinguishing intergroup preference from trait-based dehumanization[END_REF]. This would explain why our findings revealed that people with FD were perceived as having even more HN traits. Moreover, these kinds of measures of dehumanization are closely related to stereotyping [START_REF] Over | Falsifying the Dehumanization Hypothesis[END_REF], and there are inconstancies among studies investigating stereotypes about people with FD [START_REF] Stevenage | The Influence of Disfigurement on Conversational Recall[END_REF][START_REF] Stevenage | Model applicants: The effect of facial appearance on recruitment decisions[END_REF]. Thus, future research should also consider using measures other than those consisting of attributing characteristics (for a complete review of dehumanization-related measures, see [START_REF] Kteily | Dehumanization: trends, insights, and challenges[END_REF]). Since people with FD are stigmatized [START_REF] Rasset | Towards a better understanding of the social stigma of facial difference[END_REF] and automatic strong stigma reactions can be lowered by more controlled reactions [START_REF] Pryor | A Dual-Process Model of Reactions to Perceived Stigma[END_REF], a special interest should be given to the measures allowing the investigation of dehumanization at an earlier stage of processing. More generally, measures to circumvent the social desirability bias would be advisable.

Second, we used faces displaying direct eye gaze, which may have biased humanity attributions. Faces are individuating stimuli [START_REF] Hugenberg | Faces are central to social cognition[END_REF]. Moreover, Khalid and colleagues (2016) [START_REF] Khalid | The Eyes Are the Windows to the Mind[END_REF] revealed that direct eye gaze signals an invitation to social interaction and thus promotes mind perception. Eye contact may have impeded social distancing, which would explain the unexpected overattribution of HN traits to people with FD. In other words, this direct eye gaze may have counterbalanced the negative effect of FD on mind perception. Moreover, we provided photographs of people with relatively mild forms of disfigurement, which could have been sufficient to foster a global change in gaze behavior towards people with FD but insufficient to dehumanize them. Since the more severe the disfigurement is, the more it elicits disgust [START_REF] Stone | Emotional responses to disfigured faces and Disgust Sensitivity: An eye-tracking study[END_REF][START_REF] Shanmugarajah | The role of disgust emotions in the observer response to facial disfigurement[END_REF], future studies should investigate the impact of a severity gradient on humanity attributions. In this research, we did not include a measure of disgust for theoretical (e.g., we were concerned that priming our participants with disgust might bias their visual attention) and technical reasons (e.g., a longer procedure might have exhausted participants-this would have been particularly true in the case of including multiple affects). However, given that previous research on disgust [START_REF] Landry | Filthy animals: Integrating the behavioral immune system disgust into a model of prophylactic dehumanization[END_REF] led us assume a dehumanization of people with FD that was not supported by our data, the impact of disgust on social perceptions of people with FD would merit future attention.

Finally, it is possible that this stigmatizing gaze towards people with FD is not underpinned by dehumanizing attributions. Thus, other related-but independent-constructs, such as objectification [START_REF] Bernard | Objectifying objectification: When and why people are cognitively reduced to their parts akin to objects[END_REF], may better correspond to the lived experience of people with FD, which deserves future attention. Regardless, people with FD feel dehumanized (i.e., metadehumanization [START_REF] Kteily | They see us as less than human: Metadehumanization predicts intergroup conflict via reciprocal dehumanization[END_REF]), which may lead them to perceive themselves as diminished human beings (i.e., selfdehumanization [START_REF] Bastian | Self-dehumanization. TPM-Testing[END_REF]). Thus, the impact of this stigmatizing gaze on their self-and meta-dehumanization should also be investigated in future studies.

Conclusion

There is a French expression for saying that a person is unrecognizable after an event that has profoundly altered his or her facial appearance: he or she 'no longer has a human face' ('Il/Elle n'a plus figure humaine' in French). However, future research is needed to confirm that this expression is psychologically true.

Facial

  difference and humanness PLOS ONE | https://doi.org/10.1371/journal.pone.0295617 December 12, 2023 2 / 8

F = 5 13 Note.

 513 Values for total dwell time and time to first fixation are indicated in seconds. Contrary to total dwell time, lower scores indicate more attention for time to first fixation. Time to first fixation for the eyes and nose must be considered carefully because, for some faces, the participants were asked to focus on these areas at the onset of each image presentation. https://doi.org/10.1371/journal.pone.0295617.t001

Table 1 . Means (standard deviations) of eye-tracking variables for each AoI depending on the type of face evaluated in the preregistered study.

 1 The results of repeated-measures ANOVA are also included.

	AoI	Target Cheek	Eyes	Nose	Other cheek	Lips	All
	Type of face
	Total dwell Time. Main effect of Type of Face: F(1, 96) = 0.74, p = .39, η 2	G < .001
	Original	0.18 (0.24)	2.26 (0.83)	0.80 (0.50)	0.18 (0.24)	0.33 (0.23)	0.75 (0.47)	Main effect of AoI
	Disfigured All	0.64 (0.33) 0.41 (0.29)	1.91 (0.87) 2,08 (0.85)	0.71 (0.48) 0.75 (0.49)	0.17 (0.18) 0.17 (0.21)	0.30 (0.21) 0.31 (0.22)	0.75 (0.48) 0.75 (0.47)	F(1.53, 146.75) = 266.88 p < .001 η 2 G = .69
									Interaction effect
	Simple effet test	F = 174.28	F = 26.97	F = 9.68	F = 0.80	F = 4.05		F(2.00, 192.10) = 54.94
		p < .001	p < .001	p < .01	p = .37	p = .05		p < .001
		d = 1.34	d = 0.53	d = 0.32	d = 0.09	d = 0.20		η 2	G = .06
	Time to First Fixation. Main effect of Type of Face: F(1, 96) = 43.74, p < .001, η 2 G = .01
	Original	4.09 (0.84)	0.87 (0.75)	1.20 (0.75)	4.08 (0.76)	3.30 (0.75)	2.70 (0.77)	Main effect of AoI
	Disfigured All	2.58 (0.92) 3.33 (0.88)	0.98 (0.86) 0.92 (0.81)	1.37 (0.96) 1.28 (0.86)	4.10 (0.79) 4.09 (0.78)	3.54 (0.90) 3.42 (0.83)	2.51 (0.89) 2.61 (0.83)	F(3.68, 257.65) = 362.59 p < .001 η 2 G = .69
	Simple effet test	F = 203.15	F = 2.96
		p < .001	p = .09
		d = 1.45	d = 0.17

  Table in S1 File for details).

PLOS ONE | https://doi.org/10.1371/journal.pone.0295617December 12, 2023

Acknowledgments

The authors thank Anne Lacherez for realizing the stimuli, Julie Brisson for providing both the experimental device and her know-how to be able to use it, and Celine Quint for the proofreading.

This study was supported by the Normandy Region [90 000 €, 2018; grant RIN tremplin 19E00851, 2019] to JM, and a grant from the Foundation Gueules Casse ´es [22 910 €, 2020] to PR. These funders did not play any role in the study design, data collection and analysis, decision to publish, or preparations of the manuscript. There was no additional external funding received for this study.

All data have been made publicly available via the Open Science Framework (OSF) and can be accessed at this link: https://osf.io/v7eab/?view_only=

Supporting information

S1 File. Supplementary file. Pilot study, full method section, and supplementary results.


Freddy Limpens
email: freddy.limpens@sophia.inria.fr

Alexandre Monnin
email: alexandre.monnin@malix.univ-paris1.fr

David Laniado
email: david.laniado@elet.polimi.it

Fabien Gandon
email: fabien.gandon@sophia.inria.fr

NiceTag Ontology: tags as named graphs

Current tag modelling does not fully take into account the rich and diverse nature tags, as signs, can take on. We propose an ontology of tags in which tags are modelled as named graphs. These named graphs are made of a resource linked to a "sign" which can be any resource reachable on the Web (an ontology concept, an image, etc.). The purpose of our model is to be able to describe tags in a very general manner, and as an immediate consequence, to describe tags as modelled by other tag models (SCOT, CommonTag, etc.). Our tag model can thus be seen as a bridge between the manifold conceptualizations and instantiations of tags's models currently available on the Web.

Introduction

Tags are nowadays a key feature of the social Web and a new form of expression that can serve many purposes: categorizing or classifying content, comment, vote, react, express, share, identify, etc. Social tagging and the resulting folksonomies can be seen as a new opportunity to involve users in a novel relationship with web content.

The goal of our model, drafted at the VoCamp in Nice 2009 5 , is to allow for modelling tag actions without being bound to a unique model either of the sign used to tag or of its semantics. In order to describe tags in the most flexible manner, we propose considering them primarily as a link between a tagged resource and a sign used to tag, which can take on many different forms and conceptualizations (an image, a literal, an ontology concept, etc.). These two entities are modelled with the rdfs:Resource class from the RDF specifications in order to leave the choice of the model of tags or tagged resources (in particular in conjunction with the IRW resource ontology designed by Harry Halpin and Valentina Presutti which allows for a fine-grained specification of the different strata of resources encountered on the Web). Then, the link between a tagged resource and a sign used to tag is represented using named graphs. As the declaration of named graphs is not natively supported in RDF, we have integrated the model from Carroll et al. [START_REF] Carroll | Named graphs, provenance and trust[END_REF] and the RDF/XML Source declaration syntax from [START_REF] Gandon | Rdf/xml source declaration, w3c member submission[END_REF]. This choice allows us to express taggings with the different current models of tags (such as SCOT6 , CommonTag7 , etc.) and, thus, to bridge them in a very efficient way, and to query several conceptualisations of tags at a time. In addition any description of the act of tagging itself (provenance, means or other contextual and pragmatic information) can be attached to the named graphs of the tag action.

The paper is organized as follows. In section two we discuss the motivations for proposing a new model of tags which tries to account for the wealth of expression hidden behind the simplicity of tags. In section three we detail our modelling of tags and our implementation of their declaration as named graphs, and give some examples before concluding in section four.

Nature and usages of tags

For quite a long time, the nature of tags has been partly obfuscated. Thanks to the work accomplished in the wake of the identity crisis faced by the semantic Web (due to the shift from a Web of documents towards a Web of things, URIs being used to identify both "things" and Web pages thus fostering ambiguity) by Patrick Hayes and Harry Halpin on Web proper names formerly known as URIs (now IRIs), reference was eventually formally distinguished from access8 -the name Patrick Hayes has given to this other relationship.

As much material devices as semiotic ones, tags exhibit a similar kind of duality. To conflate the two aspects would therefore be tantamount to overlooking the simple fact that symbolic bonds between words and things, for instance, do not the least require to be technically implemented in any way. No technical apparatus is necessary for a word to point to an object, no artifact will ever make up for this possibility; in other words, reference pertains to the domain of semantics (or pragmatics, thereof) -not to network engineering.

On the other hand, every tagging system is implemented according to the rules dictated by the needs of the website it is a part of (who's allowed to tag? what? how? etc.). Reference is thus de facto complemented on another level by the association, on the technical side of things (the Internet being a physical network were information is exchanged, this should not come out as a surprise) of a tag with a resource (or rather, if we consider delicious-like tagging, "the representation of a WebResource", as defined by Halpin and Presutti [START_REF] Halpin | An ontology of resources: Solving the identity crisis[END_REF]).

Its ins and outs concern interfaces design and, as we previously remarked, the tangible realities of networks and protocols (especially the Web architecture centred around the HTPP protocol and IRIs).

Hence, while the label of the tag itself is nothing more than the string of characters inscribed on it, the latter is also to be construed as a material support belonging to an informational network. While contriving access to a resource, this prop also allows users to add any required bit of texts to whatever resource. It then becomes possible to index, evaluate, share or find again objects that previously overstepped the customary limits of annotation.

The most striking contrast with previous existing bodies of practices and norms, from the point of view of professional indexers, must have been the shift from a priori, controlled indexing to these freer forms of annotations. An immediate consequence of this newly acquired freedom, in a nutshell, is that labels are no longer terms of a thesaurus, subject headings in library classifications or even words but all this at the same time and even much more than that. Including, among other things: triple and/or machine tags, URLs, smileys (iconic representations in general), images, messages written in special fonts like Windings, computer code, etc.

If the freedom to choose its own labels that is now offered to users was to be properly acknowledged, then there would remain only one conclusion to be drawn. Contrary to subject headings or descriptors whose semantics is rigidly established in line with a single model of interpretation or through a well-ordered lexicon -thanks to a small number of relations established and postulated in order to evacuate every remainder of ambiguity -tags, understood as inscribed labels, are able to comprise various entities, linguistic or not, thus forbidding all global theorizing on the semantics behind their use. In other words, the label of a tag is a blank space that is fit to accommodate any sort of inscribable entities. Any sign. Accordingly, it is therefore completely devoid of any fixed (denotational) semantics (See [START_REF] Monnin | Tags and folksonomies as artifacts of meaning[END_REF] for a thorough vindication of this point of view. The NEPOMUK ontology NAO 9 , back in 2007, also made a similar point).

3 Modelling tags with the NiceTag ontology

Tag actions as named graphs

Carroll et al. [START_REF] Carroll | Named graphs, provenance and trust[END_REF] noted that RDF does not provide mechanisms (apart from statement reification) for talking about graphs and relations between graphs. They introduced Named Graphs in RDF to allow publishers to communicate assertional intent and to sign their assertions. The fact that it is often useful to embody social acts with some record clearly resonates with the scenarios of social tagging. Several authors before them proposed to transform RDF triples into quads [START_REF] Beckett | Redland notes -contexts[END_REF][START_REF] Dumbill | Tracking provenance of rdf data[END_REF][START_REF]Intellidimension. Rdf gateway -database fundamentals[END_REF][START_REF] Macgregor | Representing contextualized data using semantic web tools[END_REF] appending to them an additional URIref or blank node or ID. The definition of [START_REF] Carroll | Named graphs, provenance and trust[END_REF] is deliberately simpler than [START_REF] Guha | Contexts for the semantic web[END_REF] and [START_REF] Sintek | Triple -a query, inference, and transformation language for the semantic web[END_REF] : "A Named Graph is an RDF graph which is assigned a name in the form of a URIref. The name of a graph may occur either in the graph itself, in other graphs, or not at all. Graphs may share URIrefs but not blank nodes." [START_REF] Carroll | Named graphs, provenance and trust[END_REF].

Extending the class rdfg:Graph defined in Carroll et al. [START_REF] Carroll | Named graphs, provenance and trust[END_REF], we define a subclass of named graphs called TagAction class and embodying the acts of tagging (see fig. 1). The triples contained in the named graph describe the link between a tagged resource and a sign, as described in fig 1 where two rdfs:Resource are linked with the property nicetag:hasSign. From this point, our model can support different ways of modelling tagged resources and signs used to tag.

Regarding the model of tagged resources, Halpin & Presutti [START_REF] Halpin | An ontology of resources: Solving the identity crisis[END_REF] addressed the fuziness typical of the notion of a "Resource" as defined by Berners-Lee et al. [START_REF] Berners-Lee | Uniform resource identifiers (uri): Generic syntax[END_REF] and its relation to URIs, and proposed a model for solving the identity crisis of resources on the Web. Their model is particularly useful to distinguish between taggings of non-information resources (as when tagging the Eiffel Tower itself, i.e. the physical object, even when doing so through a web page) from taggings of information resources (as when tagging a web page about the Eiffel tower). Thus, if one wants to tag the web page www.tour-eiffel.fr to comment on its usefulness for planning a trip, the tagged resource could be modelled with the class irw:WebResource from the IRW ontology 10 . The signs used to tag can be modelled with all the other currently available models of tags such as SCOT, NAO, Newman's Tag Ontology11 , or CommonTag. To account for the nature of the different possible tag actions, we defined subclasses of the TagAction class. These subclasses may be of interest to distinguish between tagging performed by machines (AutoTagAction), from tagging performed by humans (ManualTagAction), from more complex types of tagging as those involving machine tags (MachineTagAction). Tag actions may also be collective (CollectiveTagAction) or individual (IndividualTagAction). Finally, the TagAction class is declared as a subclass of the class Item from the SIOC 12 ontology in order to account for the shareable nature of tags, which can be seen as some sort of post in an online community platform. This, in turn, allows us to describe the place where tag actions are stored with the sioc class sioc:has container, and also the account (sioc:User) of the user (foaf:Person) of the tag with sioc:has creator. Of course the idea is also to extend that model as new types of tagging acts are identified.

Modelling tag usages

Current models of tagging aim at linking tags to well defined meanings in order to face the problem of polysemy of tags [START_REF] Passant | Meaning of a tag: A collaborative approach to bridge the gap between tagging and linked data[END_REF]. Still, polysemy is not the only ambiguity of tags: some meaning resides in the (so far implicit) kind of relationship between the resource and the sign. For example, the use of tag "blog" can assume at least two different meanings, with respect to the same definition of the word "blog" it can mean that a resource is about blogs, or that a resource is a blog. Moreover, some tags are intended for personal use and to only make sense for the applier. Inspired by previous studies, and in particular by Golder & Huberman [START_REF] Golder | Usage patterns of collaborative tagging systems[END_REF], we modelled the different possible uses of tags with sub-properties of the nicetag:hasSign property; we also grouped them in three broader classes: factual, subjective and personal, as proposed in [START_REF] Sen | Tagging, communities, vocabulary, evolution[END_REF]. Beyond this classification, we added two more branches to represent community and networking tags (see fig. 2).

The property isAbout represents the most common use of a tag, to identify the topic of an item, whereas hasKind is intended for all cases in which a tag is used to define what a resource is (e.g.: "forum", "video"); both are subsumed by hasFactualSign, the first subproperty of hasSign. The second branch, for subjective tags, comprises two subproperties: hasQuality, to associate a resource with an adjective or with any kind of sign expressing a quality (e.g.: "nice", "bullshit"), and emotionalReaction, for tags expressing an emotion stirred up by a resource; typical examples are exclamations and smileys (e.g.: "wow!", "ˆˆ"). The third direct subproperty of hasSign is hasPersonalSign, which covers all uses of tags intended to just make sense for the applier; this includes Golder & Huberman's classes task organizing (like "toread") and self reference (like "mystuff"). Similarly, we introduced the property hasCommunitySign for tags that have an intended audience of a community. For example, we used the tag "#vocampnice2009" to share resources about the VoCamp across multiple social Web applications.

As a last branch we've added the two properties suggestedTo and suggest-edBy, as subproperties of hasNetworkingSign, to model networking tasks. Some bookmarking systems already have a special syntax for this (e.g.: delicious "for:username" tags).

Using RDF/XML Source declaration to implement and use named graphs

In SPARQL, when querying a collection of graphs, the GRAPH keyword is used to match patterns against named graphs. However the RDF data model focuses on expressing triples with a subject, predicate and object and neither it nor its RDF/XML syntax provide a mechanism to specify the source of each triple. To serialize named graphs, Carroll et al. used TriX and TriG [START_REF] Carroll | Named graphs, provenance and trust[END_REF] but noted that RDF/XML is the deployed base. Therefore, we proposed in the W3C Member Submission "RDF/XML Source Declaration" [START_REF] Gandon | Rdf/xml source declaration, w3c member submission[END_REF] an XML syntax to associate to the triples encoded in RDF/XML an IRI specifying their origin ; it uses a single attribute to specify for these triples represented in RDF/XML the source they should be attached to. The IRI of the source of a triple is:

1. the source IRI specified by a cos:graph attribute on the XML element encoding this triple, if one exists, otherwise 2. the source IRI of the element's parent element (obtained following recursively the same rules), otherwise 3. the base IRI of the document.

The scope of a source declaration extends from the beginning of the start-element in which it appears to the end of the corresponding end-element, excluding the scope of any inner source declarations. Such a source declaration applies to all elements and attributes within its scope. If no source is specified, the URL of the RDF/XML document is used as a default source. Only one source can be declared as attribute of a single element.

The example in listing 1.1 shows how this applies to declare a tag as a named graph. Lines 4-7 declare the tag as a graph named http://mysocialsi.te-/tag#7182904 . Lines 8-11 reuses the name of the graph to qualify the tag as a tag created manually by "Fabien Gandon" the 7th of October 2009.

Loading this RDF in a compliant triple store one can then run SPARQL queries like the one in listing 1.2. Line 2 searches for named graphs and the triples they contain. Line 3 enforces these graphs to be manually generated tags.

Thanks to the flexibility of our model, we are able to express tags in many different possible flavors. A sign used to tag can be a mere character string, or an instance of the tag class from SCOT or CommonTag. In the the latter case, the meaning of the tag (given by ctag:means property) will be included within Listing 1.1. Declaration of a tag as a named graph using RDF/XML GRAPH ? tag { ? t ? a ? g } 3 ? tag rdf : type nt : Ma n ua lT ag A ct io n } the named graph of the tag action. As we mentioned earlier the IRW ontology [START_REF] Halpin | An ontology of resources: Solving the identity crisis[END_REF], tagged resources can be modelled with the class irw:Resource or its subclasses in order to distinguish properly what is being tagged. As a consequence, it is possible to retrieve in a single query all taggings expressed with our model, regardless the type of signs used to tag or the type of the tagged resource.

Conclusion

In this paper we have proposed a general and flexible model which represents tags, taking into account the "speech act" dimension of tagging, by means of named graphs. The essence of a tag in NiceTag is to embody in a record one or more RDF triples associating a resource with a sign and this core information can be enriched in several directions. To allow for the specification of the function of a tag, we have created several subproperties to cover different kinds of relationships between the sign and the tagged resource. The named graph containing the tag is itself an instance of the class TagAction, and can thus have properties associated with it (like the tagger, the tagging date, etc.). Moreover, it is possible to define the kind of a TagAction by choosing one of the subclasses we have defined. All these primitives (sign classes, function properties, tag action classes) are also designed to be extended at will. This way, thanks to the use of the RDF/XML Source Declaration syntax to assign a URI to a tag action, we obtain a full expressive richness to represent tags from a multiplicity of facets, avoiding the burden of RDF reification. Both the Named Graphs model and the RDF/XML syntax extension provide a highvalue for a small, incremental and backward-compatible change to the Semantic Web Recommendations. Combined with the tagging vocabularies this model provides us with a very flexible and extensible framework for social tagging interoperability.

Existing tagging ontologies can be integrated into this model on the side of the "sign", allowing, for instance, for the association of a tag with a well defined concept with MOAT 13 or CommonTag, or for the specification of semantic relationships between tags with SCOT. On the side of the resource being tagged, we have pointed out the problems raised by the manifold nature of URIs in the semantic Web, where it can be often unclear if a tag should be referred to a Web resource or to the real object or entity it represents, and we have suggested the use of the IRW ontology in combination with NiceTag to face the identity crisis.

Fig. 1 .

 1 Fig. 1. TagAction instances are declared as named graphs

Fig. 2 .

 2 Fig. 2. nicetag:hasSign sub-properties

http://vocamp.org/wiki/VoCampNiceSeptember2009

http://scot-project.org/scot/

www.commontag.org

Hayes and Halpin [9] convincingly underline the necessity to carefully dissociate the two dimensions in their discussion of URIs. Our main assumption is that it is essential to reckon what this discussion revealed and transpose its result to our endeavour to characterize tags. In a twofold fashion; first, as words or rather potentially meaningful strings of characters; then as a "material" reality granting access to a resource and tightly constrained through limitations attributable to the computerized systems it belongs to (be it the Web or a local application)

http://www. semanticdesktop.org/ontologies/nao

http://ontologydesignpatterns.org/ont/web/irw.owl

http://www.holygoat.co.uk/owl/redwood/0.1/tags/

http://moat-project.org/


Felipe C Belém

Benjamin Perret

Jean Cousty
email: jean.cousty@esiee.fr

Silvio J F Guimarães

Alexandre X Falcão
email: afalcao@ic.unicamp.br

Towards a Simple and Efficient Object-based Superpixel Delineation Framework

published or not. The documents may come

I. INTRODUCTION

Superpixels are commonly defined as groups of connected pixels which share a common property (e.g, color, texture). Superpixel segmentation methods create an image partition by delineating relevant borders. Ideally, they should preserve the borders of an object of interest, such that the object can be represented by the union of its superpixels. These methods are often used as an intermediary step in several applications: (i) medical image analysis [START_REF] Martins | A supervoxel-based approach for unsupervised abnormal asymmetry detection in MR images of the brain[END_REF]- [START_REF] Zhou | Superpixel segmentation of breast cancer pathology images based on features extracted from the autoencoder[END_REF]; (ii) pedestrian segmentation [START_REF] Yu | Pedestrian segmentation based on a spatio-temporally consistent graph-cut with optimal transport[END_REF]; and (iii) plant detection [START_REF] Zhang | Plant diseased leaf segmentation and recognition by fusion of superpixel, K-means and PHOG[END_REF].

Several works [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF], [START_REF] Stutz | Superpixels: An evaluation of the state-of-the-art[END_REF] elect a number of desirable properties for superpixel methods.

• Every pixel must be uniquely assigned to a single superpixel; • Superpixels must be a connected set of pixels;

• The object boundaries should be overlapped by a superpixel border; • Superpixels should be generated efficiently; and • The user should be able to control the number of superpixels. Moreover, these properties should be satisfied with as few superpixels as possible [START_REF] Liu | Entropy rate superpixel segmentation[END_REF]. Although some authors list other properties, such as compacity, one may note that the aforementioned ones are a consensus amongst all methods.

A three-stage pipeline is an approach commonly seen in many superpixel segmentation methods [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF], [START_REF] Li | Superpixel segmentation using linear spectral clustering[END_REF]- [START_REF] Belém | Superpixel segmentation using dynamic and iterative spanning forest[END_REF] As an example, Iterative Spanning Forest (ISF) [START_REF] Vargas-Muñoz | An iterative spanning forest framework for superpixel segmentation[END_REF] is a superpixel segmentation framework in which, benefiting from the properties of the Image Foresting Transform (IFT) [START_REF] Falcão | The image foresting transform: Theory, algorithms, and applications[END_REF] algorithm, all of its steps are independently defined. That is, modifications in one step do not require adjustments in the others. Dynamic ISF [START_REF] Belém | Superpixel segmentation using dynamic and iterative spanning forest[END_REF] is another example that follows the same pipeline. In DISF, the seeds are selected through oversampling and, through a few iterations, superpixels are delineated by the IFT algorithm as the seed set is reduced in step (c) based on superpixel properties. Although state-of-theart methods present accurate object delineation, they do not usually consider prior object information. For instance, it is not possible to control the superpixel displacement in order to improve object delineation in specific regions, often within or nearby the boundary of the objects of interest.

A recent category of superpixel methods permits the inclusion of object information during execution. Given such information, these algorithms may improve the delineation performance in relevant regions. The Object-based ISF [START_REF] Belém | Superpixel generation by the iterative spanning forest using object information[END_REF], [START_REF] Belém | Superpixel segmentation by object-based iterative spanning forest[END_REF] method is a three-stage IFT-based superpixel approach that falls into this category. In OISF, object information is represented by an object saliency map. OISF variants consider the map's intensities as indication of probable object location and extension. As exemplified in Figure 1, such maps often present good estimation of the object location, but performs poorly in terms of object delineation (even when the saliency map is created by a deep neural network [START_REF] Qin | U2-net: Going deeper with nested u-structure for salient object detection[END_REF]). Given that, OISF methods are highly dependable on the object information quality, such that errors in the saliency map may critically affect its delineation performance. Furthermore, OISF variants often present high computational cost.

In this paper, we propose Object-based DISF, a novel object-based superpixel segmentation framework which exploits the major features of DISF while incorporating prior object information as represented by an object saliency map. Similarly to DISF, ODISF variants perform oversampling to guarantee that relevant seeds are part of the initial seed set and delineate superpixels by using the IFT algorithm. The major contribution of ODISF is the novel object-based seed removal criterion, which favors seeds whose superpixels are placed nearby probable object boundaries. By strategically incorporating the object information only in the seed removal step, ODISF exploits the good estimation of the object location and is more robust to eventual delineation errors in the saliency map. Therefore, such pipeline not only provides accurate object delineation with low computational cost, but also manages to overcome the high dependency on the saliency map quality seen in OISF (see Fig. 1).

This paper is organized as follows. First, we discuss recent works in superpixel segmentation in Section II. Then, in Section III, we detail the mathematical framework used for presenting DISF and ODISF in Section IV. The experimental setup and results are shown in Section V and, finally, we draw conclusions and possible future work in Section VI.

II. RELATED WORKS

In this section, we present an overview of the state-of-the-art superpixel segmentation methods. For a deeper discussion, one may refer to notable surveys [START_REF] Stutz | Superpixels: An evaluation of the state-of-the-art[END_REF], [START_REF] Schick | An evaluation of the compactness of superpixels[END_REF]- [START_REF] Neubert | Superpixel benchmark and comparison[END_REF]. In Section II-A, we review methods which are completely unaware of any object information. Then, in Section II-B we discuss methods based on deep learning. Finally, we present algorithms that consider object information independently from their source (Section II-C).

A. Classic and Content-Sensitive Methods

We may broadly classify such methods in two groups: (i) clustering-based methods; and (ii) graph-based methods. The former comprises those that solve superpixel segmentation by pixel clustering (e.g., K-means, DBSCAN, GMMs). Simple Linear Iterative Clustering (SLIC) [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF], the most popular method, solves such task by an adaptive K-means approach, resulting in low computational cost and fair object delineation. Inspired by SLIC, Linear Spectral Clustering (LSC) [START_REF] Li | Superpixel segmentation using linear spectral clustering[END_REF] maps every pixel into a 10-dimensional space and, subsequently, applies K-means. LSC shows significant improvement over SLIC in delineation, with the expense of being slightly slower. Similarly to LSC, Intrinsic Manifold SLIC (IMSLIC) [START_REF] Liu | Intrinsic manifold SLIC: A simple and efficient method for computing content-sensitive superpixels[END_REF] runs K-means in a two-dimensional manifold, where area indicates the content density. The majority of these methods (except IMSLIC) cannot guarantee connected superpixels in a given desired number, being necessary to apply a post-processing step that compromises the number of desired superpixels. Finally, due to strict constraints, such as the restricted search scope in the adapted K-means, adjustments for effectively considering object information may become impractical.

The second group of methods solve superpixel segmentation as a graph partitioning problem. Entropy Rate Superpixels (ERS) [START_REF] Liu | Entropy rate superpixel segmentation[END_REF] generates superpixels by removing edges based on the entropy of a random walk in the graph. Although it presents high object adherence, it also presents high computational cost. Superpixel Hierarchy (SH) [START_REF] Wei | Superpixel hierarchy[END_REF] computes a hierarchy of superpixels with high boundary adherence and low computational cost using the Borůkva algorithm. Yet, such methods do not consider object information in their computation.

A major subset of graph-based algorithms segments the image through path concatenation using the Image Foresting Transform (IFT) [START_REF] Falcão | The image foresting transform: Theory, algorithms, and applications[END_REF] algorithm. Such methods are often fast and present accurate object delineation. In [START_REF] Vargas-Muñoz | An iterative spanning forest framework for superpixel segmentation[END_REF], the authors propose a three-stage superpixel segmentation framework named Iterative Spanning Forest (ISF), whose steps are independently defined. This flexibility favored the development of recent and more effective approaches. The Recursive ISF [START_REF] Galvão | RISF: recursive iterative spanning forest for superpixel segmentation[END_REF] is a hierarchical superpixel segmentation method that applies ISF at each layer of the hierarchy. Dynamic ISF (DISF) [START_REF] Belém | Superpixel segmentation using dynamic and iterative spanning forest[END_REF] generates superpixels dynamically through initial seed oversampling with subsequent removal along iterations based on a predefined criterion. Such methods, however, do not consider object information in their computation.

B. Deep Learning Methods

Recently, a significant number of studies propose deep learning solutions for superpixel segmentation. In [START_REF] Awaisu | Fast superpixel segmentation with deep features[END_REF], the authors present an algorithm, named Deep-FLIC, that considers deep features as input for an adapted version of the Fast Linear Iterative Clustering (FLIC) [START_REF] Zhao | FLIC: Fast linear iterative clustering with active search[END_REF] algorithm. Conversely, the authors in [START_REF] Tu | Learning superpixels with segmentation-aware affinity loss[END_REF] debate over the efficiency of incorporating deep features and propose a neural network for computing pixel affinities in the image, which are used as input for a superpixel segmentation method (e.g., ERS). Finally, we recall the Superpixel Sampling Network (SSN) [START_REF] Jampani | Superpixel sampling networks[END_REF] and the Superpixel with Fully Convolutional Network (S-FCN) [START_REF] Yang | Superpixel segmentation with fully convolutional networks[END_REF] as representatives of end-to-end trainable superpixel segmentation methods. Notably, deep learning methods are strongly dependent on the amount of annotated training examples, which are usually scarce in scientific applications. Even if a large set is provided, one may argue that the profits of supervision lacks more evidence. Finally, for a different object of interest, the proposed model may not assure the reported performance even through fine-tuning.

C. Object-based Methods

Object-based algorithms generates superpixels by considering a prior information regarding the objects of interest. Similarly to ISF, the Object-based ISF (OISF) [START_REF] Belém | Superpixel generation by the iterative spanning forest using object information[END_REF] is a threestage object-and IFT-based superpixel segmentation framework. In OISF, the object information is provided through an object saliency map. This map is used in each of the three steps, permitting the user to control the superpixel morphology and displacement with respect to the borders of the map. As drawbacks, OISF variants are highly dependent of the quality of the saliency map, and their computational cost is high.

III. THEORETICAL BACKGROUND

In this section, we briefly discuss the theoretical background. In Section III-A, we recall basic notions regarding images and graphs and, in Subsection III-B, we detail the Image Foresting Transform (IFT) [START_REF] Falcão | The image foresting transform: Theory, algorithms, and applications[END_REF], the core method of our approach.

A. Image Graph

Let an image I be a pair P, F in which P ⊂ Z 2 is the set of picture elements (i.e., pixels) and F(p) ∈ R m , for m ∈ N >0 , consists in a particular and representative sequence of features (e.g. color) of a pixel p. If m = 1, I is a grayscale image; otherwise, I is a colored image (e.g., CIELAB). Moreover, p can be represented a unique sequence of (spatial) coordinates x p , y p . Finally, considering the previous definitions, we may define an object saliency map O as a grayscale image P, O in which O(p) ∈ [0, 1] maps every pixel p to a representative value that indicates its likelihood of belonging to an object of interest.

Given I, it is possible to build an image graph G = V, E such that V ⊆ P is the set of vertices and E ⊂ V 2 is the set of edges. A classic approach for defining the edge set is through the Euclidean (spatial) distance between two distinct vertices u, v, with respect to a certain radius r ∈ R ≥0 . More specifically, E = { u, v | u -v 2 ≤ r}. In this work, we use the 8-neighborhood (i.e., r = √ 2). If u, v ∈ E, then u, v are adjacents. In this work, G is an undirected graph (i.e., u, v ≡ v, u ).

Consider a path ρ = v i k i=1 to be a sequence of distinct vertices such that, for i < k, v i , v i+1 are adjacents. If k = 1, then ρ is a trivial path. We may explicitly exhibit the origin (or root) v 1 and the terminus v k of ρ either by ρ v1 v k or simply by ρ v k . Finally, the following notation ρ s s, t indicates a path resultant from a concatenation between a path ρ s and an edge s, t .

B. Image Foresting Transform

Several state-of-the-art algorithms use the Image Foresting Transform (IFT) [START_REF] Falcão | The image foresting transform: Theory, algorithms, and applications[END_REF] for generating superpixels [START_REF] Vargas-Muñoz | An iterative spanning forest framework for superpixel segmentation[END_REF], [START_REF] Belém | Superpixel segmentation using dynamic and iterative spanning forest[END_REF], [START_REF] Belém | Superpixel segmentation by object-based iterative spanning forest[END_REF], [START_REF] Galvão | RISF: recursive iterative spanning forest for superpixel segmentation[END_REF]. The IFT is a framework for the development of image operators based on connectivity and, in this work, we focus on its seed-restricted variant. For a given set S ⊂ V of vertices (i.e., seeds), the IFT finds optimum-paths in an nondecreasing order of cost from any s ∈ S to every p ∈ V \ S.

First, let Π be the set of all possible paths in G. Then, a path-cost function assigns a non-negative path-cost value f * (ρ) ∈ Rto any path ρ ∈ Π. The f max is often chosen due to its effective performance in object delineation, and may be defined as follows (Eq. 1):

f max ( t ) = 0 if t ∈ S +∞ otherwise f max (ρ t t, s ) = max {f max (ρ t ), w * (t, s)} (1)
in which w * (t, s) ∈ R defines a cost to an edge t, s ∈ E.

An optimum-path ρ t is a path in which, for any other

τ t ∈ Π, f * (ρ t ) ≤ f * (τ t ).
Let P be an acyclic map in which assigns a vertex v ∈ V to its predecessor P(v) = u ∈ V in a unique path ρ v or to a distinctive marker ∈ V when v is the root of ρ v . P is said to be optimum if all of its paths are optimum. As one may note, we may assign every vertex v to its respective root R(v) defined in P through recursion. Therefore, for a given seed set S and a path-cost function f * , the IFT algorithm can output an optimum predecessor map P such that, for every seed x ∈ S, it is defined an optimum-path tree T x with paths that are more closely connected to x than to any other seed. In other words, the IFT minimizes a path-cost map C(v) = min ρv∈Π {f * (ρ v )} and, consequently, builds P through path concatenation. In this work, every superpixel whose seed is x ∈ S is an optimumpath tree T x . Even if f * is not smooth [START_REF] Ciesielski | Path-value functions for which Dijkstra's algorithm returns optimal mapping[END_REF], the superpixel segmentation can be effective.

IV. OBJECT-BASED DISF

In this section, we present the Object-based Dynamic and Iterative Spanning Forest (ODISF) method, illustrated in Figure 2, by making clear its differences with respect to the DISF approach [START_REF] Belém | Superpixel segmentation using dynamic and iterative spanning forest[END_REF]. In Section IV-A, we discuss two different strategies for seed oversampling and, subsequently in Section IV-B, we revisit the concept of dynamic edge-cost estimation for superpixel generation using the IFT framework. Finally, we detail our proposed object-based seed removal strategy for a given object saliency map in Section IV-C.

A. Seed Oversampling

For a given number N f > 0 of desired superpixels, the first step of seed-based methods is to estimate an initial seed set S. Differently from most methods [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF], [START_REF] Li | Superpixel segmentation using linear spectral clustering[END_REF], [START_REF] Belém | Superpixel segmentation using dynamic and iterative spanning forest[END_REF], DISF and ODISF start off from seed oversampling such that |S| = N 0 N f . This strategy significantly increases the probability of selecting, in the initial seed set, all relevant seeds for solving the problem.

Most methods [START_REF] Achanta | SLIC superpixels compared to state-of-the-art superpixel methods[END_REF], [START_REF] Li | Superpixel segmentation using linear spectral clustering[END_REF], [START_REF] Belém | Superpixel segmentation using dynamic and iterative spanning forest[END_REF] initially distribute seeds equidistantly in a grid pattern. Such strategy, hereafter named GRID, first estimates the expected superpixel size a for an image I = P, F as a = |P|/N f . Then, it establishes that seed pairs must be distanced by √ a. Finally, in order to avoid seeds placed over borders, every seed s ∈ S is shifted to a position whose gradient is the lowest within its neighborhood.

In [START_REF] Belém | The importance of object-based seed sampling for superpixel segmentation[END_REF], the authors evaluate the impact of object-based seed sampling strategies for superpixel segmentation and conclude that a high concentration of seeds within the object leads to better results in delineation. However, these strategies are computationally expensive. On the other hand, the combination of seed oversampling and an accurate local criterion for seed removal along iterations may result into similar effectiveness with considerable efficiency gains. For example, one can preserve seeds near the borders of a given object saliency map in the seed removal step. We explore this strategy and, moreover, analyze if a random selection of seeds (hereafter named RND) is equivalent to GRID when using seed oversampling.Unlike GRID, RND does not require computing the image gradient for seed perturbation, and it is straightforward to implement for non-rectangular masks.

B. Superpixel Generation

Both DISF and ODISF use the IFT algorithm with pathcost function f max and on-the-fly edge-cost estimation, as proposed in [START_REF] Bragantini | Graphbased image segmentation using dynamic trees[END_REF] for interactive object segmentation. Let µ F (T x ) = v∈Tx F(v)/|T x | be the mean feature vector of a growing optimum-path tree rooted in x. The dynamic edgecost function is defined as w(u, v) = µ F (T x ) -F(v) 2 for x = R(u) by the time the path ρ u is optimum during the IFT algorithm.

Whenever the object saliency values in O are derived from F, both should contain the same information about the boundaries of interest. In [START_REF] Belém | Superpixel segmentation by object-based iterative spanning forest[END_REF], the authors use a path-cost function with an object-based edge-cost estimation in which it is possible to control boundary adherence with respect to the borders in O and F. As Figure 4 shows, for imperfect saliency maps, this strategy can negatively affect superpixel delineation. In such a case, the border information in F should be sufficient for accurate superpixel delineation. In our framework, we prefer to constrain the use of the object saliency map for seed removal (Section IV-C).

C. Object-based Seed Removal

In DISF, due to seed oversampling, the number of iterations is such that N f seeds (superpixels) must result at the last iteration. At each iteration i ∈ N >0 , M(i) = max N 0 exp -i , N f seeds are selected from S for the delineation step in iteration i + 1, while the remaining are discarded. This process is repeated until N i = N f , resulting in Ω iterations. For most practical cases, Ω = 5.

One approach for selecting M(i) seeds is to assign a relevance value V * (s) to each s ∈ S based on the characteristics of its resulting superpixel T s . Two distinct trees T x , T y are said adjacent if ∃ u, v ∈ E such that u ∈ T x and v ∈ T y . Let B be the set of pairs T x , T y of adjacent trees within the forest. Then, we may define V 1 (s) as a combination of size and contrast of T s as presented in Eq. 2

V 1 (s) = |T s | |V| min ∀ Ts,Tr ∈B { µ F (T s ) -µ F (T r ) 2 } (2)
The major drawback of V 1 is not distinguishing whether a superpixel is near an object border. Obviously, the size of a superpixel does not assist in such determination. As one may note, the contrast information is also insufficient since low contrast regions are present within the whole image. Thus, inspired by [START_REF] Borlido | Towards interactive image segmentation by dynamic and iterative spanning forest[END_REF], we propose an object-based seed relevance criterion V 2 (Eq. 3)

G(T s ) = max ∀ Ts,Tr ∈B { µ O (T s ) -µ O (T r ) 1 } V 2 (s) = V 1 (s) max {µ O (T s ), G(T s )} (3)

V. EXPERIMENTAL RESULTS

In this section, we present the experimental setup and the results obtained by our approach. In Section V-A, we present the baselines, the datasets and the evaluation metrics considered for benchmark. Subsequently, in Sections V-C and V-B, we present and illustrate the performance of all methods.

A. Experimental Setup

Most works evaluate their proposals in a contour-driven dataset [START_REF] Arbelaez | Contour detection and hierarchical image segmentation[END_REF] in which all relevant borders are desired. However, as one may note, our proposal aims the accurate delineation of the object of interest. Therefore, we chose three datasets from distinct domains and with different object properties for evaluating the robustness of ODISF. The Extended Complex Saliency Scene Dataset (ECSSD) [START_REF] Shi | Hierarchical image saliency detection on extended cssd[END_REF] is a popular dataset for salient object detection and consists of 1000 natural images with diverse objects. The Liver [START_REF] Vargas-Muñoz | An iterative spanning forest framework for superpixel segmentation[END_REF] dataset is composed of 40 CT slices of the human liver, which imposes a great challenge for being a grayscale object with smooth borders. Similarly, the Parasite [START_REF] Belém | Superpixel segmentation by object-based iterative spanning forest[END_REF] dataset contains 72 images of colored objects (i.e., helminth eggs) with smooth borders in which an impurity may be attached. For all datasets, we randomly defined 70% as test set. For generating object saliency maps, we considered a recent deep neural network approach [9] 1 , suitable for small datasets, whose detection accuracy is on par with many other deep-learning approaches. We trained such estimator in the remaining 30% of the dataset.

We selected five state-of-the-art methods in which the last one is, to the best of our knowledge, the only objectbased method in literature: (i) SLIC [6] 2 ; (ii) ERS [START_REF] Liu | Entropy rate superpixel segmentation[END_REF] 3 ; (iii) 1 https://github.com/xuebinqin/U-2-Net 2 https://www.epfl.ch/labs/ivrl/research/slic-superpixels/ 3 https://github.com/mingyuliutw/EntropyRateSuperpixel LSC [START_REF] Li | Superpixel segmentation using linear spectral clustering[END_REF] 4 ; (iv) SH [START_REF] Wei | Superpixel hierarchy[END_REF] 5 ; and (v) OISF-OSMOX [10] 6 . All methods were selected due to their performance in superpixel delineation and, for that, the recommended parameter configuration was set. For our approach, we set N 0 = 8000 (as in [START_REF] Belém | Superpixel segmentation using dynamic and iterative spanning forest[END_REF]) and evaluate ODISF with two options for seed oversampling: ODISF-GRID and ODISF-RND. The code of ODISF is available online 7 . The performances were assessed by two classic metrics. Boundary Recall (BR) [START_REF] Stutz | Superpixels: An evaluation of the state-of-the-art[END_REF] measures the percentage of object boundaries correctly overlapped by a superpixel border (i.e., higher is better). On the other hand, Under-Segmentation Error (UE) [START_REF] Neubert | Superpixel benchmark and comparison[END_REF] measures the error of multiple ground-truth overlapping by superpixels (i.e., lower is better). Since we do not aim for compact superpixels, and due to the high correlation between different superpixel metrics [START_REF] Stutz | Superpixels: An evaluation of the state-of-the-art[END_REF], we argue that the both selected are sufficient for a proper evaluation. We evaluated all methods considering N f ∈ [START_REF] Jampani | Superpixel sampling networks[END_REF]750].

B. Quantitative Results

Figure 3 shows the results for all datasets. It is possible to see that ODISF surpasses all baselines with significant margin, especially in boundary adherence. Table I shows the average computational time of each method using a 64bit, Intel (R) Core(TM) i5-5200U PC with CPU speed of 2.20Ghz. Knowing that SLIC, LSC, and SH are O(N ) complex, considering N to be the number of pixels in the image, it is no surprise that they are the fastest baselines. When comparing to OISF, ODISF presents a significant speedup of 3.05 (for N f = 25) in Parasites, with on pair performance in UE. It is important to emphasize that, although both ODISF and OISF are O(N log N ), OISF-OSMOX exploits parallelism during sampling and computes differential IFTs (DIFTs) [START_REF] Falcão | Interactive volume segmentation with differential image foresting transforms[END_REF] for speed optimization. Therefore, since ODISF is an IFT-based method, our implementation is eligible for computing DIFTs and further improve its speed.

Similarly to [START_REF] Jerônimo | Graph-based supervoxel computation from iterative spanning forest[END_REF], we found no statistical difference between ODISF-GRID and ODISF-RND for both metrics and, furthermore, both achieve similar performance to object-based seed sampling approaches [START_REF] Belém | The importance of object-based seed sampling for superpixel segmentation[END_REF]. Thus, we may conclude that the oversampling step can be significantly simplified without prejudicing the ODISF performance. Interestingly, OISF, whose edge-cost estimation is objectbased, presents performance inferior to ODISF, which considers the input image features only. Although there are other differences in their connectivity function, the neural network may generate imperfect object saliency maps and the use of this map restricted to the seed removal policy seems to make ODISF more robust than OISF. We argue that our object-based seed removal policy is effective to preserve relevant seeds and robust to imperfect saliency maps that at least locate the object of interest.

C. Qualitative Results

Figure 5 illustrates the segmentation results obtained by each method for the Parasite and ECSSD datasets. The strategy used by LSC yields effective results in natural images but, as one may note, it can lead to critical errors in different domains. On the other hand, SH shows a consistent performance irrespective of the image domain, justifying its position as one of the top superpixel methods. The results obtained by OISF-OSMOX illustrates its high dependency on the quality of the map by presenting high adherence to the borders of the map, including both correct and incorrect estimations. Finally, it is possible to see that ODISF segmentation manages to best approximate the object borders while it prevents severe superpixel leakings present in the results of the previous methods.

VI. CONCLUSION AND FUTURE WORK

In this paper, we propose a novel object-based superpixel segmentation framework inspired in a recent superpixel method named Dynamic and Iterative Spanning Forest (DISF) [START_REF] Belém | Superpixel segmentation using dynamic and iterative spanning forest[END_REF]. Our proposal, Object-based DISF (ODISF), exploits the positive aspects of DISF, such as seed oversampling and dynamic edge-cost estimation, while it offers a more effective and robust seed removal criterion based on prior object information. Experimental results show that ODISF variants can surpass state-of-the-art algorithms in superpixel delineation by preventing segmentation errors, while being faster than its object-based counterpart [START_REF] Belém | Superpixel generation by the iterative spanning forest using object information[END_REF], [START_REF] Belém | Superpixel segmentation by object-based iterative spanning forest[END_REF].

For future endeavors, we intent to study different curves for establishing the number of seeds at each iteration. Similarly to [START_REF] Borlido | Towards interactive image segmentation by dynamic and iterative spanning forest[END_REF], we also desire to investigate the applicability of ODISF for interactive object segmentation based on userdrawn markers.

Fig. 1 :

 1 Fig. 1: Results of object-based superpixel segmentation methods considering 25 superpixels. (a) Original image with ground-truth borders overlayed in red; (b) Object saliency map [9]; Segmentation results (in cyan), drawn over (a), obtained by (c) OISF [10] and (d) our proposal.

Fig. 2 :

 2 Fig. 2: Flowchart of the proposed Object-based DISF considering N 0 = 1000 and N f = 5.

Fig. 3 :Fig. 4 :

 34 Fig.3: Results obtained for ECSSD, Liver and Parasites, respectively. For all methods, the default configuration was set.

  in which, analogously to µ F , µ O (T s ) = v∈Ts O(v)/|T s | is the mean saliency value of T s . In contrast to other objectbased methods, which promotes a simple concentration of superpixels within (or outside) the objects of interest, our proposed function favors those nearby a probable object border, promoting seed competition in crucial regions, or within a region with high object certainty (i.e., high saliency values).

Fig. 5 :

 5 Fig.5: Segmentation results (in cyan), obtained for each method, overlaying the ground-truth (in red and magenta). A number N f = 25 of superpixels was required, and the default parameter setting was used.

TABLE I :

 I Average processing time for superpixel segmentation in the Parasites dataset.

https://jschenthu.weebly.com/projects.html

https://github.com/semiquark1/boruvka-superpixel

https://github.com/LIDS-UNICAMP/OISF

https://github.com/LIDS-UNICAMP/ODISF

ACKNOWLEDGMENT

The authors thank the Conselho Nacional de Desenvolvimento Científico e Tecnológico -CNPq -(PQ 303808/2018-7, 310075/2019-0), the Fundac ¸ão de Amparo a Pesquisa do Estado de Minas Gerais -FAPEMIG -(Grants PPM-00006-18), the Fundac ¸ão de Amparo a Pesquisa do Estado de São Paulo -FAPESP -(2014/12236-1) and the Coordenac ¸ão de Aperfeic ¸oamento de Pessoal de Nível Superior -CAPES -(Grant COFECUB 88887.191730/2018-00) for the financial support. Lastly, this study was financed in part by the Coordenac ¸ão de Aperfeic ¸oamento de Pessoal de Nível Superior -Brasil (CAPES) -Finance code 001.