Olivier Pourret
email: olivier.pourret@unilasalle.fr

Pallavi Anand

Sandra Arndt

Pieter Bots

Anthony Dosseto

Zimin Li

Johanna Marin Carbonne

Jennifer Middleton

Bryne Ngwenya

Amy J V Riches

Johanna Marin

Diversity, Equity, and Inclusion: tackling underrepresentation and recognition of talents in

Keywords: DEI, under-representation, awards, recognition, gender, professional societies

come   L'archive ouverte pluridisciplinaire

Introduction

Geochemistry and cosmochemistry apply principles of chemistry to address a diverse set of scientific questions about major societal concerns as well as our habitable planet's operation, secular evolution, and origin. Geochemical concepts and principles govern many Earth, planetary, and environmental processes that are part of global-scale biogeochemical cycles over geological time. These processes concern present-day human interaction with our planet's land, oceans, atmosphere and biosphere, as well as fundamental research into the chemistry and dynamical evolution of the Solar System. The origins of life, and its coevolution with planetary surface environments, are additional areas to which our community contributes scientific knowledge. The science that we do also covers themes ranging from the impacts of climate change through paleoclimate reconstructions and understanding past Earth System processes, understanding the mechanisms of groundwater contamination, sustainable use of resources, as well as their sometimes direct relationships to policy and to public health, the management of ocean ecologies, and the sustainability and strength of world economies (e.g., United Nations Sustainable Development Goals and the European Green Deal). The breadth of our research themes concern major present-day global challenges with important societal implications, and can also guide humanity's aspirations on and off-Earth. For the disciplines of geochemistry and cosmochemistry to prosper and reach full potential, we must attract, develop and nurture a diverse community of talented people [START_REF] Cooperdock | Black Lives Matter: Promoting Diversity, Equity, and Inclusion in Geochemistry[END_REF].

The Geochemical Society (GS) and the European Association of Geochemistry (EAG) have a long history of promoting diversity and inclusion, especially during their joint meeting, the annual Goldschmidt Conference (e.g., [START_REF] Wood | Epilogue: The 15 th Annual V.M. Goldschmidt Conference[END_REF]. However, it was not until 2020 (following the murder of George Floyd and Black Lives Matter protests) that both GS and EAG created Diversity, Equity and Inclusion (DEI) committees (https://www.geochemsoc.org/about/committeesandpersonnel/diversity-equity-and-inclusion and https://www.eag.eu.com/about/dei/). These committees are composed of diverse members with the collective goals of improving two-way dialogue with the community, addressing challenges to inclusion within geochemistry, and promoting DEI (e.g., [START_REF] Riches | Uniting to Advance Diversity, Equity, and Inclusion in a Pandemic and Post-Pandemic World[END_REF]. The work of the two DEI committees also helps to examine and find solutions to problems such as the lack of equitable opportunities for career development and bias in society awards.

We have a responsibility as professional societies, and as key voices for the geochemical and cosmochemical communities, to encourage, inspire and support change.

Recently, both the GS and the EAG have modified their governance, policies and activities to promote inclusion and diversity (e.g., by creating DEI committees). The work of both GS and EAG DEI committees has been influenced not only by our need to advance excellence and reflect the values expressed by members and elected leaders, but also by the policy and practice of other leading geoscience societies (e.g., American Geophysical Union: White and Bell, 2019; European Geosciences Union: [START_REF] Holmes | Diversity and equality in honours and awards programssteps towards a fair representation of membership[END_REF], governmental organizations and funding agencies. Over recent decades, studies have shown -albeit centred on workforce data for the USA -that changes have resulted in increased numbers of white women in the profession, while the proportion of scientists from historically excluded ethnic and racial groups, and under-represented countries, have remained consistently low for ~40 years (e.g., [START_REF] Mukasa | Underrepresentation of women and minority awardees in geoscience societies[END_REF][START_REF] Bernard | No progress on diversity in 40 years[END_REF][START_REF] Morris | Combating racism in the geosciences: Reflections from a black professor[END_REF]. The GS and EAG DEI committees seek to gather data to inform initiatives aimed at establishing best practices for removing barriers while promoting diversity and inclusion. Here we present a three-year dataset (2018-2020) of the gender demographic make-up of Goldschmidt conference attendees, which provides an (imperfect) view of the demographics of our discipline. We also evaluate the gender breakdown of GS-EAG awardees as gender data is more complete than information for other historically under-represented groups. Analyses of these data allow us to make recommendations regarding future demographic data collection and suggest new approaches to reduce inequities in award allocations.

Assessing Membership Trends through Goldschmidt Conference Attendance

Professional society membership data are limited to broadly defined status (job title, institution…), whereas data on gender, gender identity, race, ethnicity, LGBTQ+ identity, and disability status are not presently collected when a person joins either society. Therefore, to assess gender diversity in the geochemistry and cosmochemistry community, we have examined Goldschmidt attendee data over the period 2018-2020 (Figure 1), the only period for which gender data were collected, in line with international data protection protocols that ensure the security of data. We know that geochemistry and cosmochemistry research is conducted by more people than are current members of the GS (about 4000 members) or EAG (about 3000 members), and of those members, only a fraction attend a given Goldschmidt conference. The selection is not random, and is based on factors such as access to financial resources or caring of others (e.g. children). Thus, using these data as a proxy for the demographics of global geochemists introduces a bias and filters out a large body of people, including many that the DEI efforts should be directed to. Nevertheless, these are the only demographic data presently available, so we will use them as an imperfect proxy for the global geochemical and cosmochemical community.

In this three-year period (Figure 1), 58% of attendees are men (70% professional, 30% students), 40% women (57% professional, 43% students) and 2% others; 65% of attendees are professionals (63% men, 35% women and 2% other) and 35% students (49% men, 49% women and 2% others). It is notable that while there is gender parity amongst student attendees, participation tilts heavily towards men in the professional category as previously documented throughout the Earth Sciences at professional levels [START_REF] Bernard | No progress on diversity in 40 years[END_REF][START_REF] Piccoli | A report on gender diversity and equality in the geosciences: an analysis of the Swiss Geoscience Meetings from 2003 to 2019[END_REF]. This result reflects the larger proportion of men in the profession, which has been attributed to lack of research time, gendered differences among funding outcomes, and/or tendencies for greater family/caring responsibilities among women. Interestingly, 2020 saw a higher attendance by students (both male and female) compared to professionals, and women compared to men. The 2020 conference was held online, as a result of the COVID-19 pandemic, while in previous years, Goldschmidt conferences were held in person (Boston, MA, USA in 2018;Barcelona, Spain in 2019). These data may reflect more equitable access to online conferences [START_REF] Niner | The pandemic push: can COVID-19 reinvent conferences to models rooted in sustainability, equitability and inclusion?[END_REF] because they do not require travel, are less costly, and are therefore more accessible to students, researchers from developing nations and women. In the latter case, women are, on average, in earlier career stages than men (e.g., [START_REF] Sonnert | Career Patterns of Women and Men in the Sciences[END_REF].

The Goldschmidt Conference delegate gender data echo a wealth of documented evidence in Science, Technology, Engineering and Mathematics, that the proportion of women in academia progressively decreases with advancing career stages. For example, the Royal Society of Chemistry finds that female chemists tend to leave academia at earlycareer stagesand that those who remain do not ascend to senior ranks in the same proportion as their male counterparts [START_REF] Gewin | Plugging the leaky pipeline for UK female chemists[END_REF]. Women comprise just 9% of UK chemistry professors, and 18% of Natural Sciences and Mathematics in the European Union (Picoli and Guidobaldi, 2021) implying that after graduation, the relative proportion of female natural scientists drops by 32%. This phenomenon of differential attrition, known as the "leaky pipeline" [START_REF] Alper | The Pipeline Is Leaking Women All the Way Along[END_REF], affects all research fields and all minorities (including LGBTQ+: [START_REF] Hughes | Coming out in STEM: Factors affecting retention of sexual minority STEM students[END_REF]racial: Dowey et al., 2021) but is particularly accentuated in geosciences [START_REF] Bernard | No progress on diversity in 40 years[END_REF]. These stark and deeply concerning failures with respect to the retention of diverse scholars are a problem for the entire community as this stifles scientific excellence and innovation.

Data for Society Awards

The GS and EAG award several prominent honors annually: the V.M. Goldschmidt, A. Treibs Award, given by GS for major achievements, over a period of years, in organic geochemistry. for the past ten years (data accessed on 04/29/2021). While we recognize that gender is not binary, we could not include non-binary people in our analysis due to lack of available information. Awardees were assigned a binary gender using first names and in some cases, based upon the authors' own perceptions and knowledge.

H.C. Urey Award

Of 221 awards in the above categories, 196 (89%) were bestowed upon men and 25 (11%) were awarded to women (Figure 2 a andc). It is the same for fellowships (316 awardees, with 274 men and 42 women; 87/13 ratio). Gender ratios of awards in the last decade (2011-2020) have improved slightly (Figure 2 b andd). Of 69 awards and 132 fellowships, 52 (75%) awards went to men and 17 (25%) went to women whereas 104 (79%) fellows were men and 28 (21%) were women. Two-thirds of all awards to women were bestowed in the last ten years (68%). The F.G. Houtermans award is the only category with equal gender representation in the last decade, perhaps because it targets relatively early career researchers, before the "leaky pipeline" takes effect in earnest. Though efforts to enlarge nomination pools and improve diversity are being undertaken by GS and EAG, the awards and fellowships bestowed by the societies have been imbalanced with respect to gender. In particular, we need to reward not just narrowly-defined research excellence among individuals but also community contributions, and must better take into account a diversity of career paths. The EAG have started to do this for all of their awards in 2020, the GS award criteria (other than Fellowships and Urey award) have yet to refine and broaden their criteria, but that is in process. In 2020, the Fellows honor criteria were revised by the two societies to recognize the broad spectrum of scientific achievements that advance geochemistry (https://www.geochemsoc.org/honors/awards/geochemistryfellows). Combined with society efforts to generate a larger nomination pool, this resulted in a much improved and diverse 2021 Fellows class, with for example, 43% female (6 out of 14) and based in four continents. Indeed, reforming awards is critical to the health, appeal, reputation and competitiveness of our discipline because awardees serve as prominent and inspiring role models as well as ambassadors. Thus, we need to ensure that these role models reflect the global societies that we serve and are relatable for all kinds of talented people whose aspirations may rise in consequence.

Representation in Society Leadership

Gender equity in committee representation has been a challenge for decades; however, the GS and EAG have taken some encouraging steps forward in this space over recent years. Gender ratios of GS board of directors and EAG council in the last decade (2011)(2012)(2013)(2014)(2015)(2016)(2017)(2018)(2019)(2020) have improved slightly (women to men: 43 to 57 and 46 to 54, respectively). As of 2021, the GS board of directors is composed of ten women and seven men, in comparison to four women and 12 men in 2010. The current EAG council also comprises ten women and six men, in comparison to 3 women and 12 men in 2010. Leadership positions in the GS and EAG also reflect improvements in gender equity: the last four GS Presidents have been women (although only one for the EAG). However, we note that there is a trend in societies to have women undertaking greater proportions of the leadership work without commensurate increases in awards and recognition. Other categories such as ethnicity, race, sexual orientation, (non-binary) gender identity, career path, neurodiversity, and physical ability have not yet been addressed to the same extent as binary gender has. This is primarily because there is currently a lack of data and data collection can be limited by national privacy laws. Nevertheless, the improvement in gender balance in leadership positions within the two societies is important, as it will promote long-term systemic change [START_REF] Mukasa | Underrepresentation of women and minority awardees in geoscience societies[END_REF].

As already highlighted by Mukasa in 2009, gender equity remains a significant problem for geochemistry and cosmochemistry. It is evident from our results (see section 2 and Figures 1 and2) that there persists absence among women in geochemical science leadership roles. As an example, the relatively modest proportion of women professionals attending the Goldschmidt Conference may have a direct link with inequality in awards; since conferences are places where networking occurs and this cultural factor could also affect nominations. To test this hypothesis, and any bias in nomination, there is a need for a sociological study that investigates the intellectual process of award nomination. Specific challenges exacerbated by the low number of women receiving awards include: women's progression and retention; the leaky pipeline of women in higher education; gender pay/promotion/strategic influence/infrastructure allocation inequities; and lower publication rates on average, especially as first author [START_REF] Pico | First Authorship Gender Gap in the Geosciences[END_REF]. Indeed, publication rates can be lower between women and men while some regional factors and institutional promotion criteria may currently emphasize such flawed publication metrics. Publication rate alone is not a measure of the novelty, difficulty, innovation, interdisciplinary nature or scientific impact of published works that should be considered as primary markers of excellence. We are mindful that this does not take account of differences in access or career path, contributions to excellent edited volumes or books, training and mentoring of superb personnel, curation of major collections, instrument and/or laboratory development and construction. Further, leadership of complex international initiatives or space missions, important new advances in curricula, leadership of scientific meetings, engagement with the public and policy makers, and wider contributions to build capacity and advance the field are undervalued when publication metrics are over-emphasized.

Recommendations

The results of our preliminary investigation of gender demographics lead to a series of recommendations to improve practices and inclusion:

(i) Collection of voluntary anonymous demographic data for the GS and EAG membership should be prioritized because such data are foundational to evaluating the present make-up of our community. We are aware that a whole host of privacy laws in multiple countries necessitates careful planning for their collection and the navigation of such legislation requires specialist guidance. These data will also provide a basis for identifying target areas to make a concerted effort in promoting diversity and inclusion, hence shaping the ongoing development of the GS and EAG DEI Strategic Plans. The number of scientists from underrepresented minoritized groups who are joining, remaining with, or leaving GS and EAG are currently unknown. Career stages (early / junior, mid, and senior) of professional members, not currently reported through society records, could provide much needed insights into membership trends and the composition of community leaders. Increased availability of demographic information, provided anonymously and voluntarily by participants, that is commensurate with international privacy laws and community practices, would help to evaluate the health of the societies (where we understand that the results will require strong community engagement and high participation rates to be meaningful);

(ii) Strong and visible diverse leadership free of academic misconduct and/or bullying (which has not been documented within the leadership of GS and EAG, but has been within our community; [START_REF] Anonymous | Sexual harassment must not be kept under wraps[END_REF][START_REF] Dalton | Journal will publish accused scientist's work[END_REF][START_REF] Xin | Chinese Geoscientist Accused of Misusing Funds[END_REF]. We suggest that past awardees (and current nominees) who have been proven guilty of misconduct relating to discrimination, harassment, bullying or criminal activities will lose their honors (or be omitted from consideration). Such an action will align with that recently proposed by other professional societies (e.g., National Academy of Sciences and Royal Society of Chemistry);

(iii) The names chosen for awards should evolve, with consideration to how current names reflect privileges in terms of access to opportunity, socioeconomic setting, and circumstances. Hence, potentially perpetuating unconscious cognitive bias, as well as subtleties of their aspiration and appeal among minoritized people. We are well aware that current names have existed for a long time and mean something to the community members in terms of recognition. As a first step, we might be able to add a neutral or female name to the Science and Innovation awards and to consult with the community to gather perspectives to inform the evolution of awards' naming conventions. As AGU and EGU did when choosing the name of their new awards, a greater diversity in award names would contribute to changing the perception that the successful scientist is a white male (Karacam, 2016), in consequence inspiring all genders and ethnicities to a career in science;

(iv) Increased two-way dialogue providing opportunities or platforms through which marginalized and minoritized groups can share their views with the societies. These exchanges can include friendly and critical discussion at conferences and during online Town Halls, via email contact points, and also through social media and publications in journals, magazines and blogs (e.g., parents caring for children with disabilities; [START_REF] Pourret | Redefine success after lockdowns[END_REF]. Embedding better ways to reach and engage with geochemists and cosmochemists across the world is a key target. Networking and Open Science are part of the solution [START_REF] Irawan | How Open Science may help us during and after the pandemic[END_REF];

(v) Revision of current nomination processes to enlarge and diversify nomination pools, account for inequity and reduce bias in award nomination, and better align with wider diversity, equity, and inclusion principles. Nominations for awards have historically come from a small number of people who are not representative of the society memberships and our depth of deserving talent. This discrepancy could be addressed by publishing materials that improve clarity for nomination procedures, possibly providing feedback to nominators to help them improve nomination packages (as modelled by AGU), and by polling or canvassing members on nominations. If 1000 GS members were polled, this would represent ~25% of the current total memberships. Such polling could be facilitated by recruiting volunteers. Polled members would be encouraged to nominate people across genders and ethnicities. Eventually, a more radical idea might be to say that no award will be given unless a minimum of five or ten nominations are submitted (or 30 for fellowships).

(vi) Evaluation and reform of award criteria and assessment procedures (i.e. work in progress by the joint GS-EAG Awards Task Force). Such reforms will help to recognize teams while lessening emphasis on troublesome metrics such as number of citations or journal impact factors. These revisions will also provide recognition of varied contributions to raise the image of the discipline as well as broadening impact and engagement so as to increase (e.g., geographic) diversity. The lack of equity in career development should also be addressed in award attribution. Women, people of racial minorities, and those identifying as non-binary may not have the same career opportunities as men and have to work against conscious and unconscious bias. Plus, scientists in developing nations or small institutions do not have access to the same facilities as colleagues, for instance, based in large employers in Europe and North America. In addition, minoritized people could face greater scientific cultural obstacles that are targeted by ongoing -but not completed -community practice and structural reforms. If these issues are to be addressed, we need to be able to identify ways to quantitatively define and level the playing field among measures of research success. This is a complex goal to tackle, but it could benefit from externally-funded collaborations involving experts in social and behavioral science; discussions regarding this approach are ongoing. Nevertheless, a more level playing field might be achieved by collecting data on research metrics of past awardees. Although one's research success should not be reduced to a number, a more balanced quantitative measure of research achievement would remove biases and allow for scaling to opportunity (e.g., when scaling to opportunity, a European male scientist with 1000 citations could be as deserving as a European female scientist with 500 citations, or an African male scientist with 200 citations).

We are well aware that citation metrics are flawed because they may favor whatever the dominant demographic may be in a particular field. But they are also flawed in that they only reflect the science that is popular, and many times, some of the most interesting work actually takes some time to become widely recognized and cited.

(vi) Increasing recognition of effective working partnerships and value of collaborative research groups to include technical staff and laboratory managers. This approach may reduce potential fuel for "toxic academic environments" or over-pressured internal competition. Indeed, we have a responsibility to identify in our award systems that research excellence is driven by collaboration, and so must recognize research teams. While individual recognitions are important, scientific discoveries are more often a team effort. This has been identified by awards such as the Nobel Prize (itself with its own DEI challenges) where a team of researchers are acknowledged for their scientific discovery. Collaborative working is the foundation of geochemical and cosmochemical research and it has led to numerous discoveries and innovations (e.g., clumped isotope interlaboratory calibration, [START_REF] Petersen | Effects of Improved 17O Correction on Interlaboratory Agreement in Clumped Isotope Calibrations, Estimates of Mineral-Specific Offsets, and Temperature Dependence of Acid Digestion Fractionation[END_REF] as well as scientific advances underpinned by sample collection involving international and diverse groups of researchers (e.g., International Ocean Discovery Program, space missions such as Stardust and Hayabusa2). Yet, while space agencies such as NASA and ESA do give group achievement awards, these collaborative efforts are currently overlooked by GS's and EAG's award systems in geochemistry and cosmochemistry. In addition, by recognizing team efforts and cooperative research cultures, divisive issues arising from competition within the community, institutional pressures, and exploitation/bullying of early career and/or under-represented groups, could potentially be reduced. A new award serving this purpose should be created by GS and EAG, and we advise that the name for this is generated through consultation with the wider membership.

(vii) More targeted research and statistical analysis to characterize the workforce in geochemistry and cosmochemistry globally. Such efforts will gain from new partnerships with practitioners of social and behavioral sciences and will benefit from meaningful financial support from funders across all nations and regions.

We must take more genuine action to build, promote, and retain a diverse student population, work on curricula and fieldwork requirements that are inclusive and provide opportunities for disadvantaged groups so as to unite in promoting equity (e.g., [START_REF] Anadu | Ten steps to protect BIPOC scholars in the field[END_REF][START_REF] Chiarella | Fieldwork and disability: An overview for an inclusive experience[END_REF][START_REF] Kingsbury | Nothing about us without us:" The perspectives of autistic geoscientists on inclusive instructional practices in geoscience education[END_REF]. Such efforts will enable us to build an increasingly diverse community in which a wide spectrum of people feel that they belong, thereby supporting scholars through student journey to early career to mid-and senior scientist stages while enriching our scientific excellence through the retention of diversity of thought and talent.

To increase the speed of transition, we propose new commitments aimed at reforming our policies and practices:

(i) Examining how we collaborate and value teams within our community, both those within and those that cross traditional subject matter, geographic, and institutional boundaries;

(ii) Encouraging community engagement with publications of this type and also with GS's and EAG's forthcoming strategic plans to advance diversity, equity, and inclusion;

(iii) Collecting DEI data, observations, and gathering expertise available wherever possible to inform centralized initiatives that seek to transform scientific workplaces and ecosystems for the better of all (e.g., via government / funder focused studies and programs).

(iv) Reports and publications derived from DEI data could help beyond the scope of improving inclusivity and equity in GS and EAG award procedures. These openly accessible studies could inform and influence the protocols of other international bodies while also encouraging the community in cooperative efforts to improve the representation in geochemistry and cosmochemistry among other international awards.

The entire community should help to build momentum and inspire more reform. We need to lift the bar in higher education, and among private and non-profit employers, for diversity and inclusion. Institutions should better recognize the inherent value and the full range of contributions made to their organization by colleagues among diverse workforces, and must make greater commitments to developing the best environments and policies for change. The geochemical and cosmochemical community needs to overcome institutional disadvantages and encourage as many individuals to contribute to scientific discovery and innovation as possible. We must accelerate the pace of change and broaden the discussion beyond gender.

Figure 1

 1 Figure 1 Gender and career stage distribution of Goldschmidt Conference attendees

F

  .W. Clarke, C.C. Patterson, A. Treibs, H.C. Urey, F.G. Houtermans medals and Science Innovation Award (see award descriptors below). The two societies also award Fellowships to outstanding scientists who have made major contributions to geochemistry (note that recipients of the V.M. Goldschmidt, C.C. Patterson, A. Treibs, H.C. Urey and Science Innovation Awards become Fellows automatically). Award Descriptors up to 2021: V.M. Goldschmidt Award, the GS's highest honor, presented annually for major achievements in geochemistry or cosmochemistry, consisting of either a single outstanding contribution, or a series of publications that have had great influence on the field. F.W. Clarke Award, presented annually by GS to an early-career scientist for a single outstanding contribution to geochemistry or cosmochemistry, published either as a single paper or a series of papers on a single topic. C.C. Patterson Award, awarded annually by GS for a recent innovative breakthrough in environmental geochemistry of fundamental significance, published in a peer-reviewed journal.

  , the EAG's highest honor, recognizing outstanding contributions advancing geochemistry over a career. F.G. Houtermans Award, recognizing exceptional contributions to geochemistry made by scientists within 12 years from the start of PhD by EAG. Science Innovation Award, recognizing scientists who have recently made a particularly important and innovative breakthrough in geochemistry by EAG, subject area differs from year to year, according to the following five-year cycle: Alfred Edward "Ted" Ringwood Medal honoring his work in petrology and mineral physics; Heinz Lowenstam Medal honoring his work in biogeochemistry; Nicholas Shackleton Medal honoring his work in climatology; Samuel Epstein Medal honoring his work in isotope geochemistry; Werner Stumm Medal honoring his work in low temperature and surface geochemistry.Geochemistry Fellows Honor, given to outstanding scientists who have made major contributions to geochemistry and recognising the broad spectrum of scientific achievements that advance geochemistry.

Figure 2

 2 Figure 2 Gender breakdown among selected award recipients, including (a) all-time award

Acknowledgments

We are deeply thankful to Marie-Aude Hulshoff, Kevin Johnson, Magali Ader, Heather Buss, Susan Little, Elena C. Maters, Elodie Saillet, Roberta Rudnick, Vickie Bennett, Derek Vance, Catherine Chauvel, Sumit Chakraborty, Sigurður Reynir Gíslason, Elizabeth Cottrell, Alexandra V. Turchyn and Janne Koornneef for their help in collecting the data, informal discussion and advice. This contribution was informed by exchanges with the entire membership of both GS and EAG DEI Committees. While these DEI committees retain a constructive degree of independence they are highly-appreciative of dialogue with the GS Board of Directors and the EAG Council.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial or non-profit sectors.

Conflict of interest


Wenbin Zou
email: wzou@szu.edu.cn

Zhengyu Zhang
email: zyzhang23@163.com

Yingqing Peng

Canqun Xiang

Shishun W Tian
email: stian@szu.edu.cn

Lu Zhang

Z Zou

Y Zhang

C Peng

S Xiang

Tian

A Strong Correlation Learning Framework for Region Proposal

Keywords: Region proposal, two-stage, strong correlation, SC-RPN

published or not. The documents may come   L'archive ouverte pluridisciplinaire

I. INTRODUCTION

O BJECT detection is one of the most fundamental and challenging tasks in computer vision, which is widely used in surveillance [START_REF] Furgale | Toward automated driving in cities using close-to-market sensors: An overview of the v-charge project[END_REF], biomedical analysis [START_REF] Shin | Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning[END_REF], digital map construction [START_REF] He | Text-attentional convolutional neural network for scene text detection[END_REF], and autonomous driving [START_REF] Conte | Meeting the application requirements of intelligent video surveillance systems in moving object detection[END_REF], [START_REF] Liao | Security event recognition for visual surveillance[END_REF]. As objects can exist at any positions with different scales in a given image, it is exhaustive to directly search everywhere. In order to reduce the searching area and improve the computation efficiency, region proposals are generated as the Region of Interest (ROI) where may contain objects. Modern two-stage object detectors usually begin with a region proposal approach and followed by a R-CNN head. For example, Faster R-CNN [START_REF] Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF], in the first stage, generates region proposals as coarse location by using region proposal network (RPN), and then predicts classification and more accurate location based on the region proposals in the second stage. Since the subsequent detection is based on region proposals, the performance of two-stage object detector is largely determined by the quality of region proposals produced by region proposal approach. Thus it is essential to improve the performance of region proposal approaches.

Generally, a region proposal approach consists of several modules, each of which plays its own role. Different combinations of these modules will affect the performance of region proposal approach. In this paper, we focus on the region proposal approaches with two-stage framework, whose location results are predicted in the first stage and refined in the second stage. By revisiting the training process of the pervious two-stage region proposal approaches, we find several weak correlation issues in the existing approaches, which lead to performance degradation. Specifically, these weak correlation issues can be roughly summarized into three categories:

• The correlation between location and classification of region proposals. In previous region proposal approaches, the classification score [START_REF] Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF] is the unique criterion to measure the location accuracy. Generally, the conventional binary cross entropy loss is applied to train the classification branch, which drives all the positive samples to learn their classification scores as high as possible without considering their location quality. Thus, the training of classification task and location task is independent of each other, which leads to the fact that the classification scores of region proposals cannot reflect their location accuracy correctly. As a result, some candidate region proposals with high location accuracy but low classification score are directly filtered out during Non-Maximum Suppression (NMS), which causes the performance degradation of region proposal. • The correlation of sampling strategy between the two stages. In the training phase, sampling strategy is utilized to select positive samples and calculate losses. The previous two-stage region proposal approaches rarely consider the importance of the correlation of sampling strategy between the two stages. In this paper, this correlation is called "sampling consistency". Here, the sampling inconsistency problems are manifested in three phenomena: [START_REF] Furgale | Toward automated driving in cities using close-to-market sensors: An overview of the v-charge project[END_REF] The positive samples selected in the first stage will be all inside the ground-truth box of the object, but some positive samples selected in the second stage are outside. [START_REF] Shin | Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning[END_REF] The number of the positive samples is stable in the first stage, but it becomes extremely unstable in the second stage.

(3) When combined with Feature Pyramid Networks (FPN) [START_REF] Lin | Feature pyramid networks for object detection[END_REF], the positive samples selected in the first stage are associated with the object size, but in the second stage, the positive samples become unrelated to the object size. • The correlation between region proposals and anchorbased representation. Bounding box representation has been shown effectiveness to describe region proposals. Actually, bounding box representation is a kind of anchorbased representation, which heavily relies on the setting of anchors. Since the preset anchors are the prototype of the predicted objects, higher performance can be obtained when the aspect ratio of the preset anchors are close to that of the predicted objects. Actually, even though several anchors with different aspect ratios are set up to meet as many kinds of object outlines as possible, it is still very difficult to fit all possible object outlines. Limited by the unavoidable shortcoming of anchors, anchor-based representation is inherently difficult to fit the outline of objects efficiently. However, region proposals need to fit the outline of objects well to facilitate the subsequent detection task. To solve the weak correlation problems mentioned above, we propose a strong correlation learning framework for region proposal, abbreviated as SC-RPN. It mainly contains three novel components: 1) Light-weight IoU-Mask branch; 2) Size-Aware Dynamic Sampling; 3) point-based representation.

In summary, the contributions of this paper can be summarized as follows: 1. Facing the weak correlation issue of classification score and location accuracy in region proposal approach, we propose an extra Light-weight IoU-Mask branch, which predicts IoU mask to rebuild the correlation between classification scores and location accuracy of region proposals with a small amount of additional computation. 2. We delve into the correlation of sampling strategy between different stages in two-stage region proposal approach, and find that it is crucial to ensure the sampling consistency. Thus we propose a Size-Aware Dynamic Sampling (SADS) to establish a stronger correlation between different stages, in terms of location, number and size. 3. We investigate the inherent shortcoming of anchor-based representation in region proposals. To handle this problem, we integrate a point-based representation into region proposal approach to improve the outline fitting ability of region proposal approach. 4. We demonstrate that the proposed SC-RPN, substantially outperforms the state-of-the-art region proposal approaches.

The SC-RPN is integrated into different kinds of mainstream object detectors and consistently achieves the best performance, which shows the effectiveness of our proposed framework.

II. RELATED WORK

Over the past few years, object detection has attracted extensive attentions and different variations of object detectors have been proposed, including one-stage object detectors (e.g., [START_REF] Redmon | You only look once: Unified, real-time object detection[END_REF], [START_REF] Redmon | Yolo9000: better, faster, stronger[END_REF], [START_REF] Liu | Ssd: Single shot multibox detector[END_REF]) and two-stage object detectors (e.g., [START_REF] Girshick | Rich feature hierarchies for accurate object detection and semantic segmentation[END_REF], [START_REF] Girshick | Fast r-cnn[END_REF], [START_REF] Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF]). Although the two-stage object detectors have achieved promising performance, recently researchers are still trying to exploit their potential from different aspects. Cao et al. [START_REF] Cao | Prime sample attention in object detection[END_REF] reweight the training samples according to a novel ranking method named IoU-HLR. A novel AP-Loss [START_REF] Chen | Ap-loss for accurate one-stage object detection[END_REF] is formulated to replace the classification loss, which alleviates foreground-background class imbalance issue. Chen et al. [START_REF] Chen | Piou loss: Towards accurate oriented object detection in complex environments[END_REF] propose a novel PIoU Loss to exploit both the angle and IoU for accurate oriented bounding box regression. Confluence [START_REF] Shepley | Confluence: A robust noniou alternative to non-maxima suppression in object detection[END_REF] selects optimal bounding boxes and removes highly confluent neighboring bounding boxes according to "Manhattan Distance" instead of the conventional IoU. AugFPN [START_REF] Guo | Augfpn: Improving multi-scale feature learning for object detection[END_REF] designs a more robust and powerful structure to further exploit the conventional FPN. In this paper, we focus on the first stage of two-stage object detectors, region proposal approaches, whose previous works can be roughly divided into three types: grouping proposal approaches, window scoring proposal approaches and CNN-based proposal approaches.

A. Grouping proposal approaches

In grouping proposal approaches, oversegmentation approaches are first adopted to generate superpixels for an image. Then the similar superpixels are grouped hierarchically with different merging strategies to obtain the proposals. Here, grouping proposal approaches are generally based on diverse low-level cues such as appearance color and superpixel shape. With manually similarity function, Selective Search [START_REF] Van De Sande | Segmentation as selective search for object recognition[END_REF], [START_REF] Uijlings | Selective search for object recognition[END_REF] generates proposals by greedily merging the most similar superpixels. Manen et al. [START_REF] Manen | Prime object proposals with randomized prim's algorithm[END_REF] innovatively propose a randomized superpixel merging strategy to address all the probabilities. Rantalankila et al. [START_REF] Rantalankila | Generating object segmentation proposals using global and local search[END_REF] utilize novel features that differ from Selective Search, then the generated regions are regarded as seeds to generate more proposals. CPMC [START_REF] Carreira | Constrained parametric min-cuts for automatic object segmentation[END_REF], [START_REF] Carreira | Cpmc: Automatic object segmentation using constrained parametric min-cuts[END_REF] directly uses seeds and unaries to cut the graph on pixels, which avoids the initial oversegmentation. A hierarchical segmentation is established from occlusion boundaries in [START_REF] Endres | Category independent object proposals[END_REF], [START_REF] Endres | Category-independent object proposals with diverse ranking[END_REF], then different seeds and parameters are used to solve graph cuts. The resulting proposals are ranked by a wide range of cues. Rigor [START_REF] Humayun | Rigor: Reusing inference in graph cuts for generating object regions[END_REF] uses multiple graph-cuts and fast edge detectors to speed up computing. In Geodesic [START_REF] Krähenbühl | Geodesic object proposals[END_REF], it starts with generating superpixels, then a set of precomputed geodesic distance transforms are selected as proposals. With multiple segmentation outputs, grouping proposal approaches can produce the proposals with high location accuracy, but they are also time-consuming and computationally expensive.

B. Window scoring proposal approaches

Window scoring proposal approaches firstly initialize a large number of candidate windows with different positions and scales in an image, then generate proposals by scoring and ranking each candidate window according to the probability that they contain objects. Objectness [START_REF] Alexe | What is an object?[END_REF], [START_REF] Alexe | Measuring the objectness of image windows[END_REF] regards the salient locations in the image as candidate proposals, these proposals are then sorted by multiple low-level cues. Rahtu et al. [START_REF] Rahtu | Learning a category independent object detection cascade[END_REF] initialize a large proposal pool which contains sampling regions and multiple randomly sampling boxes, and adopt a scoring strategy which is similar to Objectness.

Blaschko et al. [START_REF] Blaschko | Non maximal suppression in cascaded ranking models[END_REF] add more low-level features on the basis of Rahtu to distinguish the quality of proposals in the subsequent scoring work. In Bing [START_REF] Cheng | Bing: Binarized normed gradients for objectness estimation at 300fps[END_REF], a simple linear classifier is trained by edge information and runs as a sliding window approach to find high-scoring proposals. When Bing is applied in video sequences, an extra closed-loop proposal method [START_REF] Galteri | Spatio-temporal closed-loop object detection[END_REF] is proposed to exploit the sequential nature of videos which improves the quality of proposals. EdgeBoxes [START_REF] Zitnick | Edge boxes: Locating object proposals from edges[END_REF] begins with a coarse pattern of sliding window, then a subsequent refinement is applied to improve location accuracy. In RandomizedSeeds [START_REF] Van Den Bergh | Online video seeds for temporal window objectness[END_REF], each candidate window is scored by utilizing multiple randomised SEED superpixel maps, without any additional cues. Since this type of proposal approach does not return the segmentation results of proposals, they usually tend to be faster than grouping proposal approaches. However, in window scoring proposal approaches, all the sampling sliding windows are defined by hand-crafted scales and position, which leads to poor location accuracy.

C. CNN-based proposal approaches

As the flourish of convolutional neural network, CNN-based proposal approaches have been developed rapidly thanks to the powerful discrimination feature extracted by convolutional neural network. After obtaining the input feature, these approaches generally predict the coordinates of proposals for each local patch of feature. Multi-Box [START_REF] Szegedy | Scalable, high-quality object detection[END_REF], [START_REF] Erhan | Scalable object detection using deep neural networks[END_REF] trains a neural network to produce a certain number of proposals which contains coordinates and scores, all the proposals are then sorted according to their scores. RPN [START_REF] Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF] firstly generates the dense candidate proposals with Fully Convolutional Network [START_REF] Long | Fully convolutional networks for semantic segmentation[END_REF], and then filters out the high overlapping proposals with Non-Maximum suppression (NMS). DeepProposal [START_REF] Ghodrati | Deepproposal: Hunting objects by cascading deep convolutional layers[END_REF] uses sliding window to search proposals in CNN-based feature and trains a cascade linear classifiers to generate the high score proposals. Scale-aware prediction strategy is proposed in SPOP-net [START_REF] Jie | Scaleaware pixelwise object proposal networks[END_REF] that provides adaptive accurate prediction for objects of different sizes. Based on RPN, a two-stage manner is proposed in [START_REF] Zhong | Cascade region proposal and global context for deep object detection[END_REF] (denoted as Iterative RPN in this paper), which refines the scores and location of proposals stage by stage. As a two-stage region proposal approach, GA-RPN [START_REF] Wang | Region proposal by guided anchoring[END_REF] highlights the importance of feature alignment and adopts deformable convolution to align the feature before the second stage. Similar to GA-RPN, Cascade RPN [START_REF] Vu | Cascade rpn: Delving into high-quality region proposal network with adaptive convolution[END_REF] continues to focus on the feature alignment and proposes adaptive convolution to obtain better feature alignment. The recent CNN-based proposal approaches usually focus on the rule of feature alignment. Instead, in this paper, we pay closer attention to the weak correlation issues in two-stage region proposal approach, which distinguishes our work from others.

In summary, grouping proposal approaches can obtain high location accuracy, but they are more computationally expensive which results in lower speed, while window scoring proposal approaches are much more time-friendly but poor in location accuracy. Compared to these approaches based on hand-craft features, CNN-based proposal approaches achieve better speed/accuracy trade-offs, thanks to the CNN-based features with strong discrimination. To further improve the performance, a couple of previous CNN-based proposal approaches adopt two-stage framework to refine the result stage by stage. However, the performance of the two-stage region proposal approaches is still limited by the weak correlations between different modules. Therefore, in this paper, we delve into the existing weak correlation issues in two-stage region proposal approach, and some corresponding solutions are proposed to alleviate these issues.

III. THE PROPOSED FRAMEWORK

We propose a novel region proposal approach abbreviated as SC-RPN, which aims to alleviate the three weak correlation problems discussed above. Fig. 1 shows the overall pipeline of SC-RPN, the detailed process is shown as follows: 1) We firstly utilize FPN-based backbone to generate five feature maps, which are denoted as P 2 -P 6 , then each feature map is fed into each SC-RPN head. 2) In SC-RPN head, the initial location (denoted as init offset) is predicted from the feature map in the first stage. Then the feature alignment operation is adopted to generate classification feature map and regression feature map. 3) In the second stage of SC-RPN head, classification score is predicted from classification feature map, while the secondary location (denoted as secondary offset) and IoU mask are carried out from regression feature map. After that, the refined offset is obtained by combining init offset and secondary offset, and then the refined offset is converted into refined region proposals. 4) The mean of IoU mask and classification score is calculated as IoU-aware score, according to which the duplicate results in refined region proposals are filtered out in NMS. 5) Finally, an additional NMS is applied after the five SC-RPN heads, which filters out the duplicate region proposals again before the final results are output.

There are three novel components in SC-RPN: Light-weight IoU-Mask branch, Size-Aware Dynamic Sampling (SADS) and point-based representation. Specifically, IoU mask is predicted by the Light-weight IoU-Mask branch and region proposals are described by the point-based representation. In addition, the Size-Aware Dynamic Sampling (SADS) is used for generating the training samples and calculating the losses in the training phase, as shown in Fig. 2. Based on the forward propagation structure in Fig. 1, the training phase of SC-RPN is detailed below: 1) Firstly, the ground-truth (abbreviated as GT) and init region proposals are input into SADS. 2) After that, positive samples are output and all of them are regarded as regression samples in the first stage. 3) Then in the second stage, positive and negative samples are output, among which all the positive samples are regarded as regression samples in the second stage, while all the positive samples and the negative samples after random sampling are regarded as classification samples. 4) Finally, the difference between GT and the four predictions, including init region proposals, refined region proposals, IoU mask and classification score are computed as shown in the formula ( 4)- [START_REF] Lin | Feature pyramid networks for object detection[END_REF]. All the proposed components of SC-RPN are detailed in the following sections.

A. Light-weight IoU-Mask Branch

As mentioned above, regarding classification score as the unique criterion to measure the location accuracy is subop- timal. Thus we need a novel criterion to establish a strong correlation between location accuracy and classification score. In previous work, IoU-Net [START_REF] Jiang | Acquisition of localization confidence for accurate object detection[END_REF] directly predicts an exact IoU between ground-truth box and bounding box as location score, and then replaces the conventional NMS with IoU-guided NMS. However, predicting an exact IoU requires several fullyconnected layers in IoU-Net, which is too computationally expensive in region proposal approach. In addition, aiming to prevent the high location accuracy but low classification score region proposals from being removed in NMS, IoU-Net focuses on applying a new location score, but ignores the importance of classification score itself. In this work, instead of predicting an exact IoU for each region proposal, we directly refine the suboptimal distribution of classification score map to correct the wrong classification score of high-quality region proposals. Therefore, we propose a Light-weight IoU-Mask branch to predict IoU mask, which is the distribution map of IoU between ground-truth boxes and region proposals. In the testing phase, we calculate the mean of IoU mask and classification score as IoU-aware score, which is regarded as a novel criterion to measure the location accuracy.

ResNet50-FPN backbone

SC-RPN head

Light-weight Design. Light-weight IoU-Mask branch is designed to be parallel to the secondary regression branch. Compared to the complex structure of IoU-Net, Light-weight IoU-Mask branch consists of a 1×1 convolution layer and two sigmoid function layers, which is extremely computational friendly for region proposal approach. With a 1×1 convolution layer, Light-weight IoU-Mask branch predicts IoU mask, the distribution map of location score. Then the following sigmoid function layer can keep the IoU mask to the range of (0,1). Here, IoU mask is used to refine classification score and generate IoU-aware score. Since we do not want to change the distribution of classification score dramatically, we use the sigmoid function layer twice to get a smaller distribution mask, the classification score will be refined according to IoU mask, and the IoU-aware score can better reflect the location accuracy. In this situation, the best region proposal B will be retained.

B. Size-Aware Dynamic Sampling

Generally, in two-stage region proposal approach, the problems of sampling inconsistency between the two stages are manifested in location, number and size level, as shown in the following three phenomena:

1. In the first stage, since all the candidate samples are uniformly distributed, the selected positive samples will be all inside the ground-truth box of the object. After adjusting the location in the first stage, all the candidate samples are no longer uniformly distributed. As a result, some positive samples selected in the second stage are outside the groundtruth box of the object. This phenomenon is more common in objects with slender outlines, such as toothbrush and knife.

2. The number of positive samples is stable in the first stage, but it becomes extremely unstable in the second stage. The instability in the second stage is mainly caused by two factors: (1) Some easily identifiable objects may have a large number of positive samples, while objects that are difficult to identify may have very few positive samples. (2) High threshold leads to the shortage of positive samples in the early training, while low threshold results in an excessive number of positive samples in the late training.

3. When combined with Feature Pyramid Networks (FPN), positive samples are only selected from the appropriate feature map corresponding to the object size in the first stage, which reduces the learning pressure of each feature map. However, the positive samples selected in the second stage are completely unrelated to the object size and their corresponding feature map. These size-independent positive samples will conflict with the size-aware samples selected in the first stage.

Here, the above unreasonable sampling results mainly lie on the commonly used sampling strategy, Max IoU assigner, which is used in both the two stages. In Max IoU assigner, given a hand-craft overlapping threshold, the sample whose IoU with ground truth box higher than the threshold is regarded as positive sample, otherwise it is regarded as negative sample. Therefore, in order to address the suboptimal sampling strategy and ensure the sampling consistency, we propose Size-Aware Dynamic Sampling (SADS), a simple and efficient sampling strategy to select training samples. Specifically, it can be divided into two sampling strategies: Positive Region assigner in the first stage and Size-Aware Threshold assigner in the second stage, which are shown in Algorithm 1 and Algorithm 2, respectively. for each sample d ∈ D g do 8:

if IoU of d > m g and the center of d in g then 9:

P 2 = P 2 ∪ d 10:
end if

11:
end for 12: end for 13: N 2 = T -P 2 14: return P 2 ,N 2 How SADS works. In the first stage, we apply Positive Region assigner, an anchor-free sampling strategy, to select positive samples. Negative samples are not necessary because we only predict init offset in the first stage. Since there may be multiple ground-truth boxes on an image, we assign positive samples to each ground-truth box of each image. Concretely, for each ground-truth box g on an image, we firstly calculate area S g , and then get the corresponding positive feature map number i g according to the following formula:

i g = log 2 S g -9 2 (1)
as described in Line 4 to 5 of Algorithm 1, given a preset center ratio σ in [0,1], we regard the center region of groundtruth box as the positive region of F ig , and the size of the center region is controlled by center ratio σ. The training samples inside F ig are recognized as positive samples in the first stage.

In the second stage, Size-Aware Threshold assigner is adopted to select positive and negative samples. Firstly, we calculate the area S g and the corresponding positive feature map number i g for each ground-truth box g as we did in the first stage. If g is in moderate size, an extra positive feature map number j g is assigned for g. Here, feature map F jg is adjacent to feature map F ig :

j g = i g -1 + S g 2 2×ig+9 if S g ∈ [ 2 10 , 2 18 ] (2)
For each assigned positive feature maps, we firstly select k samples closest to the center of ground-truth box g based on L2 distance, and then calculate the IoU between their corresponding init region proposal and ground-truth box g. These IoU of selected samples are denoted as D g . After that, the mean of all the IoUs in D g is calculated and denoted as m g . Here, m g is regarded as a dynamic overlapping threshold for ground-truth box g. Finally, the sample whose IoU is greater than m g and located inside the center of ground-truth box g is selected as positive samples. On the contrary, the other samples are considered as negative samples. In addition, the sample with the highest IoU will be selected if a sample is assigned to multiple ground-truth boxes. Combined with Positive Region assigner and Size-Aware Threshold assigner in SC-RPN, positive samples with different sizes can be assigned to different feature maps for better learning. Besides, different dynamic overlapping threshold is assigned to different groundtruth box in different iteration.

Three corresponding solutions for the three sampling inconsistency phenomena. The proposed SADS provides some solutions for the three sampling inconsistency phenomena mentioned above. They are detailed as follows:

1. With Positive Region assigner and uniformly distributed candidate samples in the first stage, all the selected positive samples are guaranteed to be inside the ground-truth box. In the second stage, since all the candidate samples are no longer uniformly distributed, Size-Aware Threshold assigner directly considers samples outside the ground-truth box as negative samples. Therefore, all positive samples are strictly limited inside the ground-truth box, which ensures the sampling consistency throughout the training phase.

2. The instability of the positive sample number in the second stage is mainly caused by two factors: (1) Some easily identifiable objects may have a large number of positive samples, while objects that are difficult to identify may have very few positive samples. (2) High threshold leads to the shortage of positive samples in the early training, while low threshold results in an excessive number of positive samples in the late training. As for the former, Size-Aware Threshold assigner defines a set of candidate samples for each object and then selects positive samples for it, guaranteeing a balance of positive sample number between different objects. For the latter, since the dynamic overlapping threshold can reflect the quality of current candidate samples, the model can maintain a stable number of positive samples throughout the whole training process. Thus SADS can keep the quantity and quality of positive samples more reasonable.

3. When combined with FPN, Positive Region assigner in the first stage maintains the characteristic of size perception. Specifically, each object is assigned a positive feature map number based on their size, and then all positive samples are selected from their corresponding positive feature map. However, this characteristic is often ignored in the second stage. Here, Size-Aware Threshold assigner assigns two corresponding positive feature maps for most objects based on their size, and generates dynamic overlapping threshold to select positive samples. With Size-Aware Dynamic Sampling, sizeaware positive samples are produced to ensure the sampling consistency, which improves the training efficiency.

Selecting D g with L2 distance instead of positive region in the second stage. To select the sample closest to the center of the ground-truth box, positive region is adopted in the first stage while L2 distance is used in the second stage. Although they are both used to select the closest samples, there is a certain difference between these two methods. Given a fixed center ratio, even if two objects are assigned to the same positive feature map, the difference of area will lead to a large difference in the number of positive samples, which aggravates the phenomenon 2 mentioned above. On the contrary, given a fixed hyper-parameter k to define D g for ground-truth box g based on L2 distance, the number of positive samples will be more reasonable and stable since all the positive samples are selected from D g .

Calculating the mean of size-aware samples from two positive feature maps as dynamic overlapping threshold. In order to ensure the sampling consistency, Size-Aware Threshold assigner selects positive samples which are related to the object size. For each ground-truth box g, since only positive samples of i g feature map are trained in the first stage, we can easily find that the quality of region proposals predicted from i g feature map is higher than any other feature maps. In the second stage, if all the feature maps are used to calculate the dynamic overlapping threshold like ATSS [START_REF] Zhang | Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection[END_REF], the positive samples selected in this situation will no longer be associated with the size-aware positive samples selected in the first stage. On the contrary, if only one feature map is used for calculating, the threshold will be too high, which leads to the shortage of positive samples. Thus we apply two positive feature maps to calculate the dynamic overlapping threshold, ensuring a reasonable dynamic overlapping threshold and a moderate number of positive samples. Besides, different from calculating the overlapping threshold with both variance and mean, we only calculate the mean of D g as the dynamic overlapping threshold in Size-Aware threshold assigner, which is more concise to produce a reasonable threshold for g.

C. Point-based Representation

As discussed above, even if the number of anchors is increased, it is still very difficult to meet all possible object outlines. Furthermore, with the increasing number of anchors, higher accuracy will be obtained, as well as more parameters and computations. However, region proposals need to fit the outline of objects well, which conflicts with the inherent shortcoming of anchors. Hence we draw the conclusion that such a weak correlation between region proposals and anchorbased representation is suboptimal.

Inspired by Reppoint representation [START_REF] Yang | Reppoints: Point set representation for object detection[END_REF], we integrate a point-based representation into SC-RPN to describe region proposal instead of anchor-based representation. Specifically, for each feature point on the feature map, we firstly predict init offset that utilizes 9 groups of offset to adaptively find the object boundary. After that, to get a more accurate location, we adopt the point alignment operation and then predict secondary offset. Finally, refined offset is generated by combining init offset and secondary offset, then the maximum and the minimum value of the refined offset are converted into the boundary of the refined region proposal. Different from anchor-based representation, point-based representation automatically finds the boundary point of the object and forms a semantic set of points, which gets rid of the cumbersome anchor setting. Thus, point-based representation can capture the object information in a more detailed way and enhance the outline fitting ability in SC-RPN. The entire process of point-based representation is shown in Fig. 5.

D. Learning

As an end-to-end approach, SC-RPN is trained under the guidance of a standard multi-task objective function. The total loss L total is generated by adding up the losses of all the branches, which is defined as follow:

L total = λ cls L cls + λ mask L mask + λ init L init + λ ref ined L ref ined , (3)
L cls = 1 M cls ( i∈pos,neg L BCE (c i , ĉi )), (4)
L mask = 1 M cls ( i∈pos,neg L BCE (m i , mi )), (5)
L init = 1 N reg ( i∈pos L IoU (B i , Bi )), (6)
L ref ined = 1 M reg ( i∈pos L IoU (B i , B i )), (7)
here, N reg is the number of regression samples in the first stage, M cls and M reg are the number of classification and regression samples in the second stage. Furthermore, ĉ, m, B and B denote the predictions of the classification branch, Light-weight IoU-Mask branch, initial regression branch and secondary regression branch, respectively. The predictions with no hat represent their corresponding ground-truth.

IV. EXPERIMENT RESULTS

A. Experiment Settings 1) Dataset. Most of the experiments are based on MS COCO 2017 detection dataset [START_REF] Lin | Microsoft coco: Common objects in context[END_REF]. Specifically, both region proposal approaches and object detectors are trained on train split which contains 115k images. The performance of region proposal approaches and ablation experiments are tested on val split which contains 5k images. The performance of object detectors is tested on test-dev split which contains 20k images.

2) Implementation Details. All the region proposal approaches consist of two stages except for RPN. ResNet50-FPN is used as the backbone network. Without changing the aspect ratio, the input images are resized to the scale of 1333×800 for both training and testing. No data augmentation is used except for standard flipping. The center ratio σ is set to 0.2 for selecting positive samples in the first stage. In the multitask loss function, we assign different weights based on the importance of each loss. Specifically, we use λ cls = 2.0, λ init = 0.5, λ ref ined = 10.0 and λ mask = 1.0 to balance each loss. The NMS post-processing is applied for each head, whose overlapping threshold is set to 0.8. With SGD optimizer, all the detectors are trained with 2 GPUs and a total batch size of 4 for 12 epochs. We use an initial learning rate of 0.005 and divide the learning rate by 10 after 8 and 11 epochs. The runtime is measured on GTX 1080Ti GPU. All the code are implemented with mmdetection [START_REF] Chen | Mmdetection: Open mmlab detection toolbox and benchmark[END_REF].

3) Evaluation Metrics. The performance of region proposal approach is measured with Average Recall (AR), which is the average of recalls across IoU thresholds (from 0.5 to 0.95 with a step of 0.05). AR 100 , AR 300 , and AR 1000 mean AR for 100, 300, and 1000 region proposals per image. Computed for 100 region proposals, AR for small, medium, and large objects are denoted as AR S , AR M , AR L . Detection results are reported with the averages mAP of IoUs from 0.5 to 0.95 of standard COCO metric.

B. Results

1) Region Proposal Performance. As Table I shows, we compare SC-RPN with the state-of-the-art region proposal approaches, including SharpMask [START_REF] Pinheiro | Learning to refine object segments[END_REF], GCN-NS [START_REF] Lu | Toward scale-invariance and positionsensitive region proposal networks[END_REF], At-tractioNet [START_REF] Gidaris | Attend refine repeat: Active box proposal generation via in-out localization[END_REF], ZIP [START_REF] Li | Zoom out-and-in network with map attention decision for region proposal and object detection[END_REF], RPN [START_REF] Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF], Iterative RPN [START_REF] Zhong | Cascade region proposal and global context for deep object detection[END_REF], GA-RPN [START_REF] Wang | Region proposal by guided anchoring[END_REF] and Cascade RPN [START_REF] Vu | Cascade rpn: Delving into high-quality region proposal network with adaptive convolution[END_REF]. The weak correlations limit the performance of the previous region proposal approaches. By using the proposed Light-weight IoU-Mask branch, pointbased representation and Size-Aware Dynamic Sampling, the correlations between these modules are greatly enhanced. Without bells and whistles, SC-RPN achieves an improvement of 14.5% in terms of AR 1000 compared to the RPN. Even under different region proposal numbers and object sizes, SC-RPN consistently outperforms all the existing region proposal approaches.

2) Detection Performance. To further investigate the ability of generating high-quality region proposals and its potential to improve the detection performance, we integrate SC-RPN into two common object detectors, including Fast R-CNN and Faster R-CNN. In Fast R-CNN, the pre-computed region proposals are produced by SC-RPN. While in Faster R-CNN, we utilize SC-RPN to generate region proposals instead of RPN in the first stage and train the whole model end to end. The previous works have proved that aiming to train a detector successfully, several adjustments should be made when replacing RPN with other high-quality region proposal approaches. Therefore, following [START_REF] Wang | Region proposal by guided anchoring[END_REF], the overlapping threshold in R-CNN is set to 0.65 and the region proposal number is set to 300. The detection performance are reported in Table II. Besides, the detection performance of Iterative RPN, GA-RPN, Cascade RPN are cited from the previous paper [START_REF] Vu | Cascade rpn: Delving into high-quality region proposal network with adaptive convolution[END_REF]. With RPN, Fast R-CNN yields 37.0 mAP while Faster R-CNN yields 37.1 mAP. However, integrating SC-RPN into Fast R-CNN and Faster R-CNN can boost the performance to 40.2 mAP and 40.9 mAP, respectively. Here, it can be found that the improvement of SC-RPN for Faster R-CNN is greater than that of Fast R-CNN, which shows that an end-to-end manner can better exploit the potential of SC-RPN and achieve higher detection performance.

C. Ablation Studies

Component-wise Performance. In order to demonstrate the effectiveness of each component of SC-RPN, we show an overall component-wise performance in Table III, which omits different components progressively. Firstly, RPN with three preset anchors is regarded as baseline, yielding AR 1000 of 58.3. Then the performance falls to 55.8 when we use only one anchor, implying that the model cannot fit the object outline well. Even after predicting location twice, the AR 1000 is still similar to the baseline. After that, we apply the conventional alignment operation and IoU loss, the performance improves to 65.7 and 66.3, respectively. The incorporation of point-based representation increases the AR 1000 to 66.7, implying that abandoning the use of anchors can produce more high-quality region proposals. When the Size-Aware Dynamic Sampling is added, the AR 1000 incrementally surges to 72. The incorporation of SAT in the second stage can surge the performance to 69.6, indicating the importance of ensuring sampling consistency. Furthermore, when PosR is added in the first stage, the AR 1000 increases to 72.4, implying that selecting positive samples with the preset anchors will limit the model performance.

The performance dependency of different hyper-parameters in SADS is shown in Table V. At the top of the Table V, firstly, we evaluate the effectiveness of Positive Region assigner with different hyper-parameters σ, which denotes the center ratio of positive region in the first stage. The experiment results show that the performance is similar to each other when σ is set to 0.1, 0.2 or 0.3. After that, the performance decreases with the increase of σ, implying that training the model with a small positive region can achieve a more reliable result.

Then the performance dependency results of different hyperparameters k in Size-Aware Threshold assigner are reported at the middle of the Table V. Here, k denotes the number of the closest samples selected per feature map in Size-Aware Threshold assigner. As shown in the Table, a small k value results in the shortage of positive samples and performance degradation. On the contrary, a big k value leads to the excessive number of positive samples, which has a negative impact on model performance. Thus, adopting a moderate value, k = 9, can maintain a reasonable number of positive samples and obtain the best model performance.

Finally, we explore the effectiveness of different number of feature map N selected in Size-Aware Threshold assigner, and the experiment results are shown at the bottom of the Table V. The results show that when the number of the selected feature maps is insufficient, too few positive samples are assigned for proper training due to the high dynamic overlapping threshold. Meanwhile, the excessive number of selected feature maps introduces too much noise and limits the model performance.

When two feature maps are selected in SADS, the model gets the best performance, achieving 61.3, 67.9, and 72.4 in terms of AR 100 , AR 300 , and AR 1000 , respectively.

To further demonstrate the effectiveness of SADS, the IoU distribution of region proposals with and without SADS are shown in the Fig. 6. Here, x coordinate denotes IoU between region proposals and ground-truth boxes, while y coordinate denotes the statistics number of region proposals tested on MS COCO 2017 val split. The figure shows that the incorporation of SADS significantly reduces the number Combined with experiment results in Table VI and Table VII, we can easily observe that in the case of high location accuracy, testing with IoU mask can achieve higher performance than testing without IoU mask. But in the case of low location accuracy, testing with IoU mask can hardly obtain improvement. In addition, the overall improvement of Lightweight IoU-Mask branch in high location accuracy case is more significant than that in the case of low location accuracy. Since the supervision information of Light-weight IoU-Mask branch is derived from the refined region proposals, the higher the location accuracy is, the higher the quality of supervision information will obtain. Therefore, Light-weight IoU-Mask branch can effectively exploit the potential of high-quality model. In Fig. 7, we visualize the changes in the IoU distribution of region proposals after using IoU mask. All the region proposal results are tested on MS COCO 2017 val split. Fig. 7 (a) and Fig. 7 (b) describe the changes in the IoU distribution of the top 300 region proposals and the top 1000 region proposals, respectively. These two figures consistently show a trend: when we apply IoU mask for testing, the number of region proposals with high IoU increases significantly. This phenomenon verifies that using IoU mask can refine the classification scores of region proposals and prevent the highquality region proposals from being removed in NMS. based representation can be divided into two parts: point alignment and point description. Here, the point-based representation consistently outperforms the anchor-based representation under different region proposal numbers in terms of AR, implying that abandoning the use of anchors can better fit the object outline and achieve higher performance.

In order to further demonstrate the effectiveness of pointbased representation, we delve into the performance of bounding box representation and point-based representation on objects with different aspect ratios. We firstly regard the intersection-over-union (IoU) between refined region proposals and ground-truth box as the IoU of region proposals. Then the region proposals whose IoU higher than "high-quality threshold" are regarded as high-quality region proposals. As shown in Fig. 8, x coordinate denotes different aspect ratio of objects, while y coordinate denotes the statistics number of high-quality region proposals. Fig. 8 (a) and Fig. 8 (b) denote the high-quality threshold is set to 0.7 and 0.8, respectively. All the region proposal results are tested on MS COCO 2017 val split. The figure reveals that under point-based representation, objects with different aspect ratios have more high-quality region proposals. In other words, since pointbased representation is not sensitive to the object outline, it can better fit the object outline and generate more high-quality region proposals for all objects. 4) Extension With More Object Detectors. More detection results of SC-RPN combined with other object detector pipelines are reported in Table IX. Here, we investigate Double-Head R-CNN [START_REF] Wu | Double-head rcnn: Rethinking classification and localization for object detection[END_REF] and Cascade R-CNN [START_REF] Cai | Cascade r-cnn: Delving into high quality object detection[END_REF] with different proposal approaches. Both the baseline of Double-

V. CONCLUSION

In this paper, we propose a novel two-stage strong correlation learning framework, abbreviated as SC-RPN, to generate high-quality region proposals. In order to tackle the weak correlation between location and classification of region proposals, we subtly design an extra Light-weight IoU-Mask branch to refine the classification score, which prevents the high-quality region proposals from being filtered in NMS. Besides, considering the shortcoming of anchors to represent region proposals, point-based representation is adopted in SC-RPN to generate region proposals with strong fitting ability. Furthermore, to address the weak correlation of sampling strategy between the two stages, Size-Aware Dynamic Sampling (SADS) is applied to ensure the sampling consistency during the training. Finally, we report the overall performance of the proposed SC-RPN, which surpasses all the state-of-the-art region proposal approaches. The effectiveness of the proposed approach and its components are also validated in ablation experiments. Even when combined with several object detector pipelines, SC-RPN achieves the state-of-the-art performance in terms of AP under different settings of thresholds and object sizes. As for future work, we will extend the concept of weak correlation and the proposed approach to other related tasks, e.g., instance segmentation and object tracking.

Fig. 1 .

 1 Fig. 1. The overall pipeline of SC-RPN. IoU mask is predicted by the Light-weight IoU-Mask branch and region proposals are described by the point-based representation. In addition, Size-Aware Dynamic Sampling (SADS) is used for generating the training samples and calculating the losses, which is only used in the training phase.

Fig. 2 .Fig. 3 .Fig. 4 .

 234 Fig. 2. The training phase of SC-RPN with Size-Aware Dynamic Sampling (SADS). In the first stage, positive samples are output and all of them are regarded as regression samples. Then in the second stage, positive and negative samples are output, among which all the positive samples are regarded as regression samples, while all the positive samples and the negative samples after random sampling are regarded as classification samples.

Algorithm 1 P 1 = 1 Algorithm 2 5 :

 11125 Positive Region assigner Input:G: all ground-truth boxes on a certain image F i : the i th feature map σ: the center ratio of positive region Output: P 1 : a set of positive samples in the first stage 1: for each ground-truth box g ∈ G do 2: S g ← calculate the area of g 3: i g ← assign positive feature map number based on S g 4: R g ← assign positive region on F ig based on σ 5:P g ← assign samples inside R g as positive samples 6: P 1 ∪ P g 7: end for 8: return P Size-Aware Threshold assigner Input: G: all ground-truth boxes on a certain image F i : the i th feature map T : all training samples T i : all training samples on F i k: the closest samples number selected per feature map Output: P 2 : a set of positive samples in the second stage N 2 : a set of negative samples in the second stage 1: for each ground-truth box g ∈ G do 2: S g ← calculate the area of g 3: i g ← assign positive feature map number based on S g 4: j g ← assign adjacent positive feature map number based on S g and i g D g ← select k samples closest to g from both T ig and T jg based on L2 distance 6: m g ← calculate the mean IoU of D g 7:

Fig. 5 .

 5 Fig. 5. The entire process of point-based representation. For better visualization, we use the original image instead of the feature map.

  here, L cls is the classification loss and L mask is the loss of Light-weight IoU-Mask branch. L init and L ref ined are the regression loss of the first stage and the second stage, respectively. In addition, λ cls , λ mask , λ init and λ ref ined denote the weight of L cls , L mask , L init and L ref ined , respectively. Finally, classification branch and Light-weight IoU-Mask branch are both driven by the conventional binary cross entropy loss (denoted as L BCE ), the regression branch of the two stages are both supervised by IoU loss [47] (denoted as L IoU ). The loss functions of all branches are defined below:

Fig. 7 .

 7 Fig. 7. The changes in the IoU distribution of region proposals with and without IoU mask. Here, (a) and (b) describe the changes in the IoU distribution of the top 300 region proposals and the top 1000 region proposals, respectively. Both (a) and (b) verify that IoU mask can refine the classification scores of region proposals and prevent the high-quality region proposals from being removed in NMS.

Fig. 8 .

 8 Fig. 8. The performance of bounding box representation and point-based representation on objects with different aspect ratios. Here, (a) and (b) denote the high-quality threshold is set to 0.7 and 0.8, respectively. Both (a) and (b) show that with point-based representation, the model can better fit the object outline and generate more high-quality region proposals for objects with different aspect ratios.

  4, showing the effectiveness of ensuring the sampling consistency. Finally, applying IOU mask gets the AR 1000 of 72.8, indicating that IoU-aware score can reflect the location accuracy correctly. Overall, SC-RPN achieves 17.1%, 15.5%, and 14.5% higher than that of RPN in terms of AR 100 , AR 300 , and AR 1000 . 1) Ablation Studies on Size-Aware Dynamic Sampling (SADS). Table IV shows the experiment results of different sampling strategy combinations in two stages. The proposed Size-Aware Dynamic Sampling is made up of Positive Region assigner in the first stage and Size-Aware Threshold assigner in the second stage. Here, PosR and SAT denote Positive Region assigner and Size-Aware Threshold assigner, respectively. When Max IoU assigner is applied to both the first and the second stage, the model yields the AR 1000 of 66.7.

TABLE I PERFORMANCE

 I OF REGION PROPOSAL APPROACHES ON MS COCO 2017 VAL SPLIT.

	Approach	Backbone	AR100	AR300	AR1000	AR S	AR M	AR L	Time(s)
	SharpMask [50]	ResNet50	36.4	-	48.2	-	-	-	0.76
	GCN-NS [51]	VGG-16(Sync BN)	31.6	-	60.7	-	-	-	0.10
	AttractioNet [52]	VGG-16	53.3	-	66.2	31.5	62.2	77.7	4.00
	ZIP [53]	BN-inception	53.9	-	67.0	31.9	63.0	78.5	1.13
	RPN [6]		44.6	52.9	58.3	29.5	51.7	61.4	0.04
	Itertive RPN [41]		48.5	55.4	58.8	32.1	56.9	65.4	0.05
	GA-RPN [42]	ResNet50-FPN	59.1	65.1	68.5	40.7	68.2	78.4	0.06
	Cascade RPN [43]		61.1	67.6	71.7	42.1	69.3	82.8	0.06
	SC-RPN(ours)		61.7	68.4	72.8	42.9	69.9	82.8	0.07

TABLE II PERFORMANCE

 II OF OBJECT ON MS COCO 2017 TEST-DEV SPLIT.

	Approach	Proposal Approach	# proposals	AP	AP50	AP75	AP S	AP M	AP L
		RPN		37.0	59.5	39.9	21.1	39.4	47.0
		Cascade RPN	1000	40.1	59.5	43.7	22.8	42.4	50.9
		SC-RPN(ours)		40.2	59.7	44.0	22.8	42.6	51.3
	Fast R-CNN [12]	RPN Iterative RPN		36.6 38.6	58.6 58.8	39.5 42.2	20.3 21.1	39.1 41.5	47.0 50.0
		GA-RPN	300	39.5	59.3	43.2	21.8	42.0	50.7
		Cascade RPN		40.1	59.4	43.8	22.1	42.4	51.6
		SC-RPN(ours)		40.2	59.5	44.2	22.2	42.5	51.8
		RPN		37.1	59.3	40.1	21.4	39.8	46.5
		Cascade RPN	1000	40.5	59.3	44.2	22.6	42.9	51.5
		SC-RPN(ours)		40.8	59.6	44.6	23.1	43.2	51.7
	Faster R-CNN [6]	RPN Iterative RPN		36.9 39.2	58.9 58.2	39.9 43.0	21.1 21.5	39.6 42.0	46.5 50.4
		GA-RPN	300	39.9	59.4	43.6	22.0	42.6	50.9
		Cascade RPN		40.6	58.9	43.6	22.0	42.8	52.6
		SC-RPN(ours)		40.9	59.5	45.0	22.5	43.3	52.4

TABLE III COMPONENT

 III -WISE PERFORMANCE OF SC-RPN.

	Baseline	One anchor	Two-stage	Alignment	IOU loss	Point-based rep	SADS	IOU mask	AR100	AR300	AR1000
									44.6	52.9	58.3
									44.7	51.2	55.8
									42.9	51.7	58.9
									55.1	61.7	65.7
									55.7	62.3	66.3
									56.1	62.7	66.7
									61.3	67.9	72.4
									61.7	68.4	72.8
				Over Improvement				+17.1	+15.5	+14.5

TABLE V PERFORMANCE

 V DEPENDENCY ON DIFFERENT HYPER-PARAMETERS IN SADS. HERE, σ DENOTES THE CENTER RATIO OF POSITIVE REGION IN POSITIVE REGION ASSIGNER. k DENOTES THE NUMBER OF THE CLOSEST SAMPLES SELECTED PER FEATURE MAP AND N DENOTES THE NUMBER OF THE SELECTED FEATURE MAPS IN SIZE-AWARE THRESHOLD ASSIGNER. THE EXPERIMENT RESULTS OF EACH HYPER-PARAMETER ARE REPORTED WHEN THE OTHER HYPER-PARAMETERS ARE OPTIMAL AND CONSTANT. As we can see from the TableVI, training with regression samples in low location accuracy achieves the highest improvement, while training with classification samples in high location accuracy achieves the best performance. This phenomenon is due to the fact that there are larger difference between the quality of positive samples and negative samples in high location accuracy case, compared to that in low location accuracy case. Thus, training

				hyper-parameter	AR100	AR300	AR1000
							0.1	61.1	67.8		72.4
							0.2	61.3	67.9		72.4
				σ			0.3	61.2	67.8		72.3
							0.4	60.8	67.6		72.1
							0.5	60.2	67.2		71.8
							3	54.7	63.3		69.7
							6	56.8	64.4		70.4
				k			9 12	61.3 60.8	67.9 67.7		72.4 72.4
							15	58.9	66.5		71.9
							18	58.1	65.8		71.2
							1	60.6	67.3		71.8
							2	61.3	67.9		72.4
				N			3	60.9	67.7		72.4
							4	60.2	67.3		72.3
							5	59.4	66.7		72.2
	)
	4 Number of region proposals ( x10	0 50 100 150 200 250 300	-19%	+18%	+38% +52% +63% +69% +68% +76% +55% w/o SADS w/ SADS
		0.0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1.0
									IoU
	Fig. 6. The IoU distribution of region proposals with and without SADS. The
	incorporation of SADS significantly reduces the number of region proposals
	with low IoU and increases the number of region proposals under
	different IoUs.
	2) Ablation Studies on Light-weight IoU-Mask branch.
	The performance comparisons on training IoU-
	Mask branch with different training samples and in different
	location accuracy cases are shown in Table VI. With and
	without SADS stand for the cases of high location accuracy
	and low location accuracy, respectively.

  Light-weight IoU-Mask branch with classification samples in high location accuracy case contributes to a stronger discrimination of the model. However, training with too many negative samples will have an adverse effect in either case.The performance comparisons on predicting IoU mask with different feature maps and in different location accuracy cases are shown in TableVII. Here, cls feature map and reg feature map denote classification feature map and regression feature map, respectively. Fused feature map is obtained by combining classification feature map and regression feature map. With and without SADS stand for the cases of high location accuracy and low location accuracy, respectively. TableVIIshows similar experiment results in different location accuracy cases. Since the direct fusion of two feature maps with different properties may cause information confusion, predicting IoU mask with fused feature map only yields a slight improvement. When IoU mask is predicted by classification or regression feature map, the model obtains similar performance in either case. Finally, the model in high location accuracy case achieves 61.7, 68.4, and 72.8 in terms of AR 100 , AR 300 , and AR 1000 , respectively.

		TABLE VI
	PERFORMANCE COMPARISONS ON TRAINING LIGHT-WEIGHT IOU-MASK
	BRANCH WITH DIFFERENT TRAINING SAMPLES AND IN DIFFERENT
	LOCATION ACCURACY CASES. WITH AND WITHOUT SADS STAND FOR
	THE CASES OF HIGH LOCATION ACCURACY AND LOW LOCATION
		ACCURACY, RESPECTIVELY.
	train with SADS	training samples	test with IoU mask	AR100	AR300	AR1000
		without IoU mask	×	56.1	62.7	66.7
	×	regression samples regression samples and	× ×	56.5 56.5 56.2	63.0 63.0 62.7	67.0 67.0 66.6
		all negative samples		56.2	62.7	66.6
		classification samples	×	56.3 56.4	62.9 62.9	66.8 66.8
		without IoU mask	×	61.3	67.9	72.4
		regression samples	×	61.5 61.6	68.2 68.3	72.7 72.7
		regression samples and	×	61.2	68.0	72.5
		all negative samples		61.2	68.0	72.5
		classification samples	×	61.5 61.7	68.2 68.4	72.4 72.8

TABLE VII PERFORMANCE

 VII COMPARISONS ON PREDICTING IOU MASK WITH DIFFERENT FEATURE MAPS AND IN DIFFERENT LOCATION ACCURACY CASES. HERE, CLS FEATURE MAP AND REG FEATURE MAP DENOTE CLASSIFICATION FEATURE MAP AND REGRESSION FEATURE MAP, RESPECTIVELY. FUSED FEATURE MAP IS OBTAINED BY COMBINING CLASSIFICATION FEATURE MAP AND REGRESSION FEATURE MAP.

	train with SADS	feature map	test with IoU mask	AR100	AR300	AR1000
		without IoU mask	×	56.1	62.7	66.7
	×	fused feature map cls feature map	× ×	56.1 56.2 56.3 56.3	62.7 62.7 62.9 63.0	66.7 66.7 66.9 66.9
		reg feature map	×	56.3 56.4	62.9 62.9	66.8 66.8
		without IoU mask	×	61.3	67.9	72.4
		fused feature map	×	61.4 61.6	68.1 68.3	72.5 72.5
		cls feature map	×	61.6 61.7	68.2 68.4	72.6 72.7
		reg feature map	×	61.5 61.7	68.2 68.4	72.4 72.8

  3) Ablation Studies on Point-based Representation. The experiment results of different components in point-based representation are shown in Table VIII. In this work, point-

TABLE IX PERFORMANCE

 IX OF MORE OBJECT DETECTORS ON MS COCO 2017 TEST-DEV SPLIT.

	Approach	Proposal Approach	# proposals	AP	AP50	AP75	AP S	AP M	AP L
	Double Head R-CNN [54]	RPN SC-RPN (ours)	1000	39.6 40.6	59.9 60.3	43.2 44.1	23.2 23.3	42.5 43.6	49.3 51.0
	Cascade R-CNN [55]	RPN SC-RPN (ours)	1000	40.7 41.7	59.2 60.2	44.3 45.3	23.0 23.9	43.3 44.4	51.3 52.9

TABLE X PERFORMANCE

 X OF REGION PROPOSAL APPROACHES ON IMAGENET-DET DATASET.Head R-CNN and Cascade R-CNN are two-stage frameworks and adopt RPN in the first stage. Combined with SC-RPN, Double-Head R-CNN and Cascade R-CNN surge the mAP to 40.6 and 41.7, respectively. It can be concluded that the proposed SC-RPN significantly outperforms the RPN in terms of AP under different settings of thresholds and object sizes. 5) Performance on ImageNet-DET Dataset. We present the performance of different region proposal approaches on ImageNet-DET (ILSVRC 2015)[START_REF] Russakovsky | Imagenet large scale visual recognition challenge[END_REF] dataset in Table X. To make a fair comparison, all the experimental implementation details are consistent with the experiments on MS COCO 2017 detection dataset. Regarded as the most important performance indicator, the proposed SC-RPN achieves the highest performance in AR 1000 , surpassing all the existing region proposal approaches. Furthermore, in terms of AR under different settings of thresholds and object sizes, the proposed SC-RPN achieves the best performance except AR 100 . And we can easily find that SC-RPN gains slighter improvement on ImageNet-DET dataset compared to that on MS COCO 2017 detection dataset. The average object number on each image of ImageNet-DET dataset is several times lower than that of MS COCO 2017 detection dataset, in other words, MS COCO 2017 detection dataset is much more complex than ImageNet-DET dataset. However, the proposed training method SADS in SC-RPN, which brings the most significant improvement, is proposed to set independent dynamic overlapping threshold for each object on an image. When training with a simple dataset, the potential of SADS isn't fully exploited, nor is the SC-RPN. Therefore, SC-RPN can achieve stronger improvement on complex dataset (e.g., MS COCO 2017 detection dataset).

	Approach	Dataset	Backbone	AR100	AR300	AR1000	AR S	AR M	AR L
	RPN			41.1	49.2	55.0	29.8	51.7	61.1
	GA-RPN Cascade RPN	ImageNet-DET	ResNet50-FPN	63.6 67.3	67.5 71.6	69.7 74.3	43.0 46.6	64.1 68.3	77.3 82.4
	SC-RPN (ours)			67.1	71.7	75.0	46.8	69.4	82.7

ACCEPTED MANUSCRIPT

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIP.2021.3069547, IEEE Transactions on Image Processing Wenbin Zou received the M.E. degree in software engineering with a specialization in multimedia technology from Peking University, China, in 2010, and the Ph.D. degree from the National Institute of Applied Sciences, Rennes, France, in 2014. From 2014 to 2015, he was a Researcher with the UMR Laboratoire d'informatique Gaspard-Monge, CNRS, and Ecole des Ponts ParisTech, France. He is currently an Associate Professor of College of Electronics and Information Engineering, Shenzhen University, China. His current research interests include saliency detection, object segmentation, and semantic segmentation.

Zhengyu Zhang received the B.E. degree in electronic and information science and technology from Guangzhou University, Guangzhou, China, in 2018. He is currently pursuing the M.E. degree with Shenzhen Key Laboratory of Advanced Machine Learning and Applications, Institute of Artificial Intelligence and Advanced Telecommunication, College of Electronics and Information Engineering, Shenzhen University, Shenzhen, China. His research interests include object detection, semantic segmentation and image quality assessment. Yingqing Peng received the B.E. degree in communication engineering from Wuyi University, Jiangmen, China, in 2018. She is currently pursuing the M.E. degree with Shenzhen Key Laboratory of Advanced Machine Learning and Applications, Institute of Artificial Intelligence and Advanced Telecommunication, College of Electronics and Information Engineering, Shenzhen University, Shenzhen, China. Her research interests include semantic segmentation, object detection and image enhancement. Canqun Xiang received the B.S. and the integrated M.S. degrees from the Department of Electric and Information Engineering, Hunan Institute of Science and Technology, Hunan, China, in 2015 and 2018, respectively. He is pursuing a doctoral degree at College of Electronics and Information Engineering, Shenzhen University, Shenzhen, China. His research interests include capsule network and image processing in various applications. Shishun Tian received the B.Sc. degree from Sichuan University, Chengdu, China, the M.Sc. degree from Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, China, and the Ph.D. degree from National Institute of Applied Sciences, Rennes, France in 2012, 2015 and 2019 respectively. He is currently an assistant professor of College of Electronics and Information Engineering of Shenzhen University, Shenzhen, China. His research interests include image quality assessment, visual perception and machine learning.

Lu Zhang is an associate professor at National Institute of Applied Sciences (INSA) of Rennes in France. She received the B.S degree from Southeast University and the M.S. degree from Shanghai Jiaotong University in China in 2004 and 2007, respectively. From October 2009 to November 2012, she was a PhD student of the LISA and CNRS IRCCyN labs in France, working on the model observers for the medical image quality assessment. She received the Excellent Doctoral Dissertation of France awarded by IEEE France Section, SFGBM, AGBM and GdR CNRS-Inserm Stic-Santé in 2013. Then she worked on the quality of experience (QoE) in telemedicine before she joined INSA in September 2013, as a member of the VAADER research group of the IETR lab. She is a member of the international VQEG (Video Quality Experts Group). She works on human perception understanding, image analysis, saliency detection and image quality assessment.