Evangelia Adamou

Experimental Methods to Study Cultural Differences in Linguistics

.

. This chapter offers an overview of studies that successfully overcome these hurdles and discusses future avenues for experimental research in diverse linguistic and cultural settings.

Introduction and Definitions

A relatively recent development in linguistics is the use of experimental methods in different cultural settings. It takes place within a larger trend in psychology that questions the robustness of theories and generalizability of its findings following the observation that research predominantly relies on participants from so-called 'Western, educated, industrialized, rich, and democratic' (WEIRD) societies [START_REF] Henrich | The weirdest people in the world?[END_REF]. To set the scene for this discussion, let us start by looking more closely at the two keywords of this chapter: 'experimental' and 'cultural.' 'Experimentation' is a way for scientists to isolate, prepare, and manipulate their object of investigation and its environment to test and update a scientific theory, that is, a collection of laws and generalisations that aims to describe, explain, and predict a set of phenomena (for an overview of this topic in philosophy of science see [START_REF] Boyd | Theory and observation in science[END_REF]. In comparison, 'observation' is when scientists note details about their object of investigation, either under natural or experimental conditions. Observation and experimentation may require the use of instruments and therefore the quality of measures obtained, and their analysis is key to the success of the scientific endeavour. Since experiments are about control, the lab is the typical place to conduct them. A controlled environment may be crucial in physics, where scientists may also need to use experimental equipment that is only available in the lab (e.g., a particle accelerator). In addition, so-called 'field experiments,' conducted outside the lab, are common in many scientific disciplines like geochemistry and economics (e.g., where scientists test policy-related solutions in the real world). Similarly, linguistic experiments can be conducted in the field by slightly adjusting the experimental protocol (see sections 5 and 6 for more details). This is particularly important in order to include populations who live away from university laboratories.

Let us now turn to consider the deceptively intuitive notion of 'culture.' In a classic definition, the notion of culture encompasses those aspects of behaviour that are not genetically determined but are acquired through social learning [START_REF] Cavalli-Sforza | Cultural Transmission and Evolution: A Quantitative Approach[END_REF].

Culture is reflected in myths, legends, religion, body adornment, rules, daily routines, and use of production tools [START_REF] Brown | Human universals, human nature and human culture[END_REF]. In addition, every society has different forms of social organisation: it has rules for social groups, age grading, family, kinship systems, play, division of labour, exchange, cooperation, and reciprocity [START_REF] Brown | Human universals, human nature and human culture[END_REF]. Contemporary anthropologists came to critically appraise the notion of culture itself, viewed as 'essentialized, localized, territorialized, bounded, highly shared, and symbolically coalesced' (Hirschfeld, 2018, p. 233). [START_REF] Appadurai | Modernity at Large: Cultural Dimensions of Globalisation[END_REF] proposes instead a view of cultural spaces as fluid, heterogenous, fragmented, and constantly negotiated. Similarly, [START_REF] Claidiere | The role of attraction in cultural evolution[END_REF] opt for the dynamic notion of 'cultural representations.' This means that researchers should not just aim to investigate differences between WEIRD and non-WEIRD populations but should also take into consideration diversity and variability within these populations (e.g., by including ethnically diverse and migrant populations, people who may not have a college degree or who come from economically marginalised communities).

Historical perspectives

The idea that cultural differences should take front stage in experimental psychology is not new. [START_REF] Boas | Some problems of methodology in social sciences[END_REF], a major figure in American anthropology, also criticized the exclusive focus of psychologists on the individual, abstracting away from the social environment. Yet, [START_REF] Arnett | The neglected 95%: why American psychology needs to become less American[END_REF] find that 96% of participants in behavioural research publications from 2003-2007 were from WEIRD countries, although these represent roughly 12% of the world's population. Follow-up studies by [START_REF] Nielsen | The persistent sampling bias in developmental psychology: A call to action[END_REF] and [START_REF] Rad | Toward a psychology of Homo sapiens: Making psychological science more representative of the human population[END_REF] report no substantial change. In addition, [START_REF] Rad | Toward a psychology of Homo sapiens: Making psychological science more representative of the human population[END_REF] note that participant samples are frequently unidentifiable and when they are, they rely on undergraduates and online samples with no information about participant characteristics such as socioeconomic status, education, and ethnicity.

Linguistics is equally concerned by these cultural biases. Psycholinguistics is a traditionally experimental subfield that studies how word meaning, sentence meaning, and discourse meaning are represented in the mind during speech production and comprehension. [START_REF] Anand | Widening the net: Challenges for gathering linguistic data in the digital age[END_REF] find that only 10 languages represent 85% of psycholinguistics publications and conferences, i.e., English, German, Japanese, French, Dutch, Spanish, Mandarin, Korean, Finnish, and Italian. In child language acquisition, a subfield that has a tradition of relying on both experimental and observational data, [START_REF] Kidd | How diverse is child language acquisition? First Language online first[END_REF] arrive to a similar conclusion: in the past 45 years, only 1.5% of the world's 7000 languages is represented by at least one article (N=103 languages), with English and other well-studied Indo-European languages representing the great majority of publications. To my knowledge, there are no studies on the representation of different languages in the various subfields of experimental linguistics, like experimental phonology, semantics, pragmatics, syntax, and morphology [START_REF] Hemforth | Experimental linguistics[END_REF]. Experimental phonetics, an interdisciplinary field of linguistics, physiology, and acoustics, most likely stands out for having the longest tradition of crosslinguistic research.

Critical issues and topics

Why is it important to study cultural differences in linguistics? First, we must acknowledge the significance of the empirical database issue. There is considerable language diversity, with variety at every level, from linguistic sounds, to grammar, to meaning, and lexicon that still needs to be described [START_REF] Evans | The myth of language universals: Language diversity and its importance for cognitive science[END_REF]. As a result, the search for linguistic universals has been repeatedly challenged by novel empirical evidence. For example, a proposed universal that was abandoned is 'baby talk,' the specific prosody that English, French, or German caregivers use to talk to infants. Cross-cultural and cross-linguistic studies demonstrate that baby talk is in fact a cultural convention (see Quiché Mayan in [START_REF] Bernstein-Ratner | Higher pitch in BT is not universal: acoustic evidence from Quiché Mayan[END_REF]. Another abandoned universal is the Possible Word Constraint, according to which a single consonant or cluster of consonants cannot form a word. [START_REF] Aissati | Finding words in a language that allows words without vowels[END_REF], however, found that this constraint does not apply in Tashelhit Berber, an Afroasiatic language that also allows clusters of consonants to be words. For more examples, see [START_REF] Evans | The myth of language universals: Language diversity and its importance for cognitive science[END_REF] and [START_REF] Kidd | How diverse is child language acquisition? First Language online first[END_REF].

Second, the failure to find universals in such cultural and linguistic diversity should not come as a surprise if we admit that, although humans share a common biological basis, environment and experience determine which linguistic characteristics become conventionalized.

Experience-driven models try to capture the complex and adaptive nature of language:

Language has a fundamentally social function. Processes of human interaction along with domain-general cognitive processes shape the structure and knowledge of language. Recent research in the cognitive sciences has demonstrated that patterns of use strongly affect how language is acquired, is used, and changes. These processes are not independent of one another but are facets of the same complex adaptive system (CAS).

Language as a CAS involves the following key features: The system consists of multiple agents (the speakers in the speech community) interacting with one another. The system is adaptive; that is, speakers' behaviour is based on their past interactions, and current and past interactions to get her feed forward into future behaviour. A speaker's behaviour is the consequence of competing factors ranging from perceptual constraints to social motivations. The structures of language emerge from interrelated patterns of experience, social interaction, and cognitive mechanisms (Five Graces Group, 2009: 1-2).

In sum, if we want to better understand how the interaction environment and linguistic experience shape languages and cognition, we must move beyond student participant samples from the Global North and unidentifiable samples of participants recruited online [START_REF] Blasi | Over-reliance on English hinders cognitive science[END_REF].

Current Contributions and Research

This section offers an overview of some studies that have been successfully conducted with diverse populations and summarizes their findings.

Experimental studies on language and cognition in diverse cultural settings

A survey of experimental work to study cultural differences and their effects on language can only start with reference to the controversial 'linguistic relativity' or 'Sapir-Whorf hypothesis' [START_REF] Whorf | The relation of habitual thought and behavior to language[END_REF]. The linguistic relativity hypothesis holds that our language habits shape the way we think of the real world. Nowadays, a weak version of the linguistic relativity hypothesis is largely accepted, acknowledging that language allows for faster performance and better discrimination of the real world and that it enhances memorisation. The degree of detail in the linguistic categorisations, in a given language, reflects the cultural interests and communicative needs of its speakers. 4.1.1 Quantity [START_REF] Frank | Number as a cognitive technology: Evidence from Pirahã language and cognition[END_REF] study cognition of quantity among the Amazonian Pirahã, people who speak a language that does not have a special lexicon for number. The researchers performed two tasks, one of elicitation of numerals to confirm that the Pirahã do not use number language (N=6 participants), and one matching task with real life objects to test numerical cognition (N=14 participants). The study finds that, despite the absence of number words, speakers of Pirahã perceive and match exact quantities. The study also finds that lack of number words affects accuracy in memorisation of large numbers of objects. 4.1.2 Colour. [START_REF] Gibson | Color naming across languages reflects color use[END_REF] worked with the Tsimane', a hunter-gatherer Amazonian population from Bolivia (N=28-58 participants in the various tasks). They conducted various colour naming tasks using chips and picture stimuli, as well as a memory colour task. They then compared the results to two control groups, a group of Bolivian Spanish speakers from the area (N=20-25 participants) and a group of English speakers in the US (N=29-30 participants). Analysis of the results suggests that the Tsimane' respond to their communicative needs for natural objects through an elaborate botanical vocabulary, not through colour names. However, the study also finds that introduction of coloured artefacts boosts the relevance of colour since their colour is not predictable. The number of colour names is therefore likely to increase in industrial societies when speakers are confronted to new communicational needs that require colour disambiguation.

Space

A standard task to test memorisation strategies of spatial relations in the field is 'Animals in a Row' [START_REF] Levinson | Space in Language and Cognition[END_REF]. Participants in this task are asked to memorize the placement of three, small, toy animal figures positioned in a row in front of them. They are asked to recall this initial placement and reconstruct it a few meters away after a 180° rotation. Overall, responses to the Animals in a Row task provide results confirming the linguistic relativity hypothesis, that is, that language habits predict the preferred memorisation of spatial relations.

However, [START_REF] Meakins | Bilingualism, language shift and the corresponding expansion of spatial cognitive systems[END_REF] report that the Gurindji participants in Australia gave a majority of geocentric responses to the task (i.e., memorizing the animal figures with respect to a cardinal point) whether they spoke Gurindji, a Pama-Nyungan language with an elaborate system of cardinal terms (N=30 participants), or Gurindji Kriol, a mixed language resulting from an English-lexified creole and Gurindji that has not retained the Gurindji cardinal term system (N=77 participants). [START_REF] Calderón | How to speak "geocentric" in an "egocentric" language: A multimodal study among Ngigua -Spanish bilinguals and Spanish monolinguals in a rural community of Mexico[END_REF] follow up on this observation by investigating an Indigenous population of Mexico, the Ngiguas. In the small rural community where the study was conducted, only some older adults still speak Ngigua (Otomanguean), while the younger generations only speak Spanish. 17 Spanish monolinguals and 17 Ngigua-Spanish bilinguals responded to a localisation task to allow for the study of both speech and co-speech gesture.

Analysis of the results shows that Spanish monolinguals use geocentric co-speech gestures to support geocentric representations of space, as well as Spanish cardinal terms in an innovative manner. This confirms that transmission of traditional ways of representing space is possible in small, rural communities even when the Indigenous language is no longer spoken.

Experimental studies among diverse bilingual populations

Bilingualism is another subfield where the study of cultural differences is necessary to understand human language and cognition. Bilingualism is a dynamic and adaptive neurocognitive phenomenon across individuals and across the lifespan of a single individual [START_REF] Pliatsikas | Understanding structural plasticity in the bilingual brain: The dynamic restructuring model[END_REF]. It follows that if we only examine Western, middle-class, Universityeducated bilingual populations, we are missing out on the huge variety of bilingual experiences [START_REF] Adamou | The Adaptive Bilingual Mind: Insights from Endangered Languages[END_REF].

Language switching costs

For example, in the psycholinguistic literature, there is ample experimental evidence indicating slower reaction times when participants are asked to name pictures by alternating from one language to another. These are known as 'language switching costs' related to lexical access. However, researchers increasingly show that language switching costs can be reduced or even disappear when the experimental tasks are closer to participants' real-life experience with codeswitching (see [START_REF] Gullifer | When language switching has no apparent cost: Lexical access in sentence context[END_REF] for a study with Spanish-English bilinguals from the US).

In line with this work, [START_REF] Adamou | There are no language switching costs when codeswitching is frequent[END_REF] conducted the first experimental study of sentence processing in a typologically rare form of codeswitching used by Romani (Indic)-Turkish (Altaic) simultaneous bilinguals from Greece. This Romani-Turkish variety shows an interesting split in morphology: Turkish nouns inflect in Romani for case, number, and gender, like other Romani nouns do, but Turkish verbs systematically combine with the Turkish person, tense-aspect-modality morphology, and valency morphemes, contrasting with Romani verbs that combine with Romani verbal morphology. In summary, the findings from a picture-matching and a word monitoring experiment show that Romani-Turkish bilinguals anticipate codeswitching based on prior experience (N=37 and 49 participants). More specifically, when the input aligns with their expectations, there are no processing costs. This study paves the way to experimental studies with speakers of the few fully-fledged mixed languages that have been identified in the world. Mixed languages result from the systematic combination of two languages and arise under specific social circumstances where a community of speakers forges a mixed cultural identity.

Priming

Priming refers to the observation that processing one stimulus (the prime) unconsciously affects the processing of a subsequent stimulus (the target). This mechanism is welldocumented in language, including for 'structural priming' where the activation of one structure affects the production and the comprehension of a subsequent structure. A metaanalytic study also shows that structural priming operates among monolinguals and, to a lesser extent, among bilinguals [START_REF] Mahowald | A meta-analysis of syntactic priming in language production[END_REF]. In theory, cross-language structural priming among bilinguals could be a mechanism that drives contact-induced change in the long run [START_REF] Loebell | Structural priming across languages[END_REF]. However, such ongoing change is not easily observed in languages like English, with a strong written tradition and with prescriptive norms that are strengthened by formal education. [START_REF] Kootstra | Crosslinguistic structural priming as a mechanism of contact-induced language change: Evidence from Papiamento-Dutch bilinguals in Aruba and the Netherlands[END_REF] Similarly, [START_REF] Adamou | A unified approach to the study of language contact: Cross-language priming and change in adjective/noun order[END_REF] tested the role of cross-language priming in adjective (ADJ)noun (N) order among 90 bilinguals of Romani (Indic) and Romanian (Romance) from Romania. The study reveals significant cross-language priming effects, whereby bilinguals favour the use of N-ADJ order in Romani immediately following a N-ADJ sentence read in Romanian whereas the inherited ADJ-N Romani order only benefits from priming when speakers read a Romani sentence with an ADJ-N order.

Experimental studies on typologically rare linguistic phenomena in diverse cultural settings

Typologically rare linguistic phenomena offer a unique opportunity to test theoretical claims and expand our knowledge of what is possible in human language.

Relative clauses

In a classic article, [START_REF] Keenan | Noun phrase accessibility and universal grammar[END_REF] propose that, cross-linguistically, subjects are more accessible to relativisation than direct objects, indirect objects, and oblique objects; this preference is known as the Accessibility Hierarchy. Since then, there has been a large body of work with typologically diverse languages confirming the original explanation that the Accessibility Hierarchy 'directly reflects the psychological ease of comprehension' (Keenan & Comrie, 1977, 88).

More recently, a series of experimental studies was conducted in languages that use fully ambiguous relative clauses (RCs). The rationale is that these ambiguous RCs can offer an unconfounded result for universal subject preference and an experiment has the advantage of neutralizing any disambiguating semantic and pragmatic cues that can be otherwise found in natural conversations.

The first experimental study tested subject preference in two ergative Mayan languages, Ch'ol and Q'anjob'al, using a picture-matching comprehension experiment [START_REF] Clemens | Ergativity and the complexity of extraction: A view from Mayan[END_REF].

63 Ch'ol and 100 Q'anjob'al speakers participated in the study in Mexico and Guatemala respectively. The results for the ambiguous RCs in both languages confirm universal subject preference favouring an ergative-subject interpretation (68% in Ch'ol and 74% in Q'anjob'al).

The study also shows that participants were faster in the subject responses than in the object ones offering support to the ease of comprehension explanation.

Another experimental study tested subject preference in ambiguous RCs for both postnominal and prenominal RCs in Chamorro, an Austronesian language [START_REF] Borja | Constituent order and parser control processes in Chamorro[END_REF]. Results from 135 participants show object preference in prenominal RCs, but subject preference for postnominal RCs. The study also finds a subject processing advantage in both prenominal and postnominal RCs. [START_REF] Adamou | Subject preference in Ixcatec relative clauses (Otomanguean, Mexico)[END_REF] adapted this experiment for Ixcatec, a critically-endangered Otomanguean language of Mexico with less than ten speakers. The analysis shows that 63% of the Ixcatec ambiguous RCs are interpreted as subject RCs (N=7 participants). Results from reaction times show that subject RC interpretations are numerically faster than object RC interpretations, although this difference does not reach significance. This lack of significance may be due to the small size of the sample, or it could suggest that Ixcatec comprehenders do not go through an initial stage of subject interpretation before proceeding to an object interpretation in agreement with spoken corpus preferences where transitive subject RCs are as frequent as object RCs.

Nominal tense

Nominal tense is a linguistic phenomenon that has recently been reported for a small number of languages across the world to refer to the use of grammatical morphology on argument nominals with an independent temporal interpretation from that of the clause [START_REF] Nordlinger | Nominal tense in a crosslinguistic perspective[END_REF]. This is a highly controversial phenomenon in that it challenges our view of nouns as time stable and, specifically, our experience with languages that encode tense only through verbal but not nominal morphology.

Descriptive work identified that deictic suffixes carry temporal information on their own in Pomak, a non-standardized Slavic variety spoken in Greece. But, in practice, what does it mean for speakers to have nominal tense in their language? In [START_REF] Adamou | An experimental approach to nominal tense: Evidence from Pomak (Slavic)[END_REF], an experimental study using a response-time experiment with 40 Pomak participants demonstrates for the first time that speakers of a language with nominal tense can decide whether a noun phrase is past or future without any additional information from an adverb or a verb and in the absence of pragmatic and semantic cues. Beyond the relevance of the results for Pomak, this study introduces an experimental method that can easily be used to test the existence of nominal tense in other languages.

Main Research Methods

Experiments are quantitative methods. Typically, to obtain sufficient statistical power, the number of participants and experimental trials is high. However, Navarro-Torres et al. (2021) stress that rich characterisation of the sample may be as or even more important than a poorly characterized large sample. For example, when considering the impact of bilingualism on cognition, a binary characterisation 'bilingual' vs. 'monolingual' is not informative enough since we know that fine-grained distinctions in age of acquisition, proficiency, daily use, and frequency of codeswitching are moderating factors.

In an experiment, scientists identify a 'dependent variable,' something that is the focus of the study, and test whether it is affected by an 'independent variable.' For example, the dependent variable can be the reaction times of participants in a response task and the independent variable the position of the button on the right or left. At present, complex statistical models allow for the examination of more variables as well as their interactions.

Standard experimental techniques include the use of 'control' or 'comparison' groups that differ on the variable that is hypothesized to be the main cause for an observed behaviour (e.g., when studying bilingual populations, speaking language A and B, it is common to include a monolingual control group who only speaks language A and then compare responses in language A for the two groups); randomisation of trials, so that the order of appearance of the stimuli would not influence the results; inclusion of stimuli as 'fillers,' that is, items that the experimenter is not interested in and that are used in order to prevent participants from identifying the goal of the study (note that researchers sometimes analyse responses to fillers to see if the participants performed the task as expected); and 'norming' (i.e., rating) of the stimuli prior to the experiment by non-participants to make sure they are as well-formed as the experimenter thought they would be. Some experimental designs that have been successfully conducted in the field are judgement tasks, picture-matching tasks, priming tasks, and several semi-experimental tasks, including picture naming and speech production tasks.

Judgment tasks

Acceptability judgment tasks are a reliable tool in linguistic research [START_REF] Sprouse | A comparison of informal and formal acceptability judgments using a random sample from Linguistic Inquiry 2001-2010[END_REF], particularly, when working with a large sample of participants [START_REF] Gibson | The need for quantitative methods in syntax and semantics research[END_REF].

Participants in these tasks judge the acceptability or naturalness of linguistic stimuli using a five-point or a seven-point scale (small numbers are for negative judgments and high numbers for positive judgments). To facilitate the memorisation of the scale for participants who have little experience with computers or this kind of formal testing, it is possible to use stickers on the keyboard: a smiley face on the key for the best score and a frown for the worst. To make sure participants understand the task and are allowed to negatively evaluate some stimuli, practice sentences can serve as anchors for the highest and lowest points in the scale with feedback by the experimenter prior to the experiment. Moreover, instead of asking the participants whether the stimuli are 'correct' or 'acceptable,' experimenters can ask whether they are 'natural' in the community (e.g., [START_REF] Adamou | An experimental approach to nominal tense: Evidence from Pomak (Slavic)[END_REF].

Picture-matching tasks

Picture-matching experiments are inspired by the visual world paradigm which was developed by [START_REF] Dahan | Continuous mapping from sound to meaning in spoken-language comprehension: immediate effects of verb-based thematic constraints[END_REF] for eye tracking. In picture-matching experiments, participants view a pair of pictures and are asked to match one of the pictures with the content of a spoken sentence that they hear. The pictures can be drawings (e.g., [START_REF] Borja | Constituent order and parser control processes in Chamorro[END_REF][START_REF] Clemens | Ergativity and the complexity of extraction: A view from Mayan[END_REF], pictures found on the Internet (e.g., [START_REF] Adamou | There are no language switching costs when codeswitching is frequent[END_REF], or photographs of people, objects, and situations that the community is familiar with and that are specifically taken for the study (e.g., [START_REF] Adamou | Subject preference in Ixcatec relative clauses (Otomanguean, Mexico)[END_REF][START_REF] Calderón | How to speak "geocentric" in an "egocentric" language: A multimodal study among Ngigua -Spanish bilinguals and Spanish monolinguals in a rural community of Mexico[END_REF]. The task is generally presented on a laptop. However, to make sure that the older adult speakers of Ixcatec (in their 80s) would be comfortable enough to perform the task, [START_REF] Adamou | Subject preference in Ixcatec relative clauses (Otomanguean, Mexico)[END_REF] presented the stimuli printed in A4 size. The stimuli were manually randomized through reshuffling in the beginning of each session. The sessions were filmed, and reaction times were measured from the onset of the audio stimuli to the moment where participants pointed to the picture. Later, [START_REF] Calderón | How to speak "geocentric" in an "egocentric" language: A multimodal study among Ngigua -Spanish bilinguals and Spanish monolinguals in a rural community of Mexico[END_REF] conducted this experiment with older adult Ngigua speakers and found that it can also work well on a laptop. Lack of familiarity with a computer does not seem to be an impediment when the instructions are clear (also see [START_REF] Adamou | There are no language switching costs when codeswitching is frequent[END_REF] for a similar experience).

Priming tasks

A typical design for a priming production experiment consists of a sentence trial and a picture trial. Participants first read or listen to a sentence (sentence trial) and then describe a picture (picture trial). A priming effect is observed when the linguistic structure chosen in the picture trial follows the structure of the sentence trial (e.g., [START_REF] Kootstra | Crosslinguistic structural priming as a mechanism of contact-induced language change: Evidence from Papiamento-Dutch bilinguals in Aruba and the Netherlands[END_REF][START_REF] Adamou | A unified approach to the study of language contact: Cross-language priming and change in adjective/noun order[END_REF].

Word monitoring is a different kind of priming experiment. This is a task where participants listen to a sentence and are asked to press a button as soon as they hear a specific word (target). The time it takes to press the button for the target provides information about the ease of processing of the preceding word (prime). To make sure that participants are not just searching for the target word without processing the sentence, a comprehension question can follow each trial (e.g., [START_REF] Adamou | There are no language switching costs when codeswitching is frequent[END_REF].

Semi-experimental tasks

In addition to the experimental tasks listed above, there are several semi-experimental tasks that can be useful. Semi-experimental tasks are different from experimental tasks in that they are not always structured around dependent and independent variables, trials are not randomized, and there are no fillers and control groups.

For example, verbal fluency tasks allow researchers to assess a bilingual speaker's language dominance at the moment of the study. In the semantic version of the task, speakers are asked to produce in one minute as many words as possible belonging to a specific semantic category (e.g., body parts, animals, fruits, and vegetables) in one language and then in the other (see [START_REF] Calderón | How to speak "geocentric" in an "egocentric" language: A multimodal study among Ngigua -Spanish bilinguals and Spanish monolinguals in a rural community of Mexico[END_REF]. The language in which the greatest number of words is produced, is the dominant language, at the moment of the study. In the phonological version of the task, speakers need to produce as many words as possible beginning with specific letters or sounds.

However, this version should not be used with participants with low literacy levels as it is known to be a good indicator of illiteracy (in alphabetic scripts) and is modulated by the level of education [START_REF] Petersson | Language processing modulated by literacy: a network analysis of verbal repetition in literate and illiterate subjects[END_REF].

Other popular semi-experimental tasks are naming tasks in which speakers are asked to name real life objects or objects depicted in drawings or illustrated in pictures, for example, to study colour names [START_REF] Gibson | Color naming across languages reflects color use[END_REF]. In an information structure phonetic study in Ixcatec, reallife objects were shown in a first session (e.g., fruits); photographs of these objects were shown in a second session. This combination allowed for enough repetitions while avoiding a habituation process that could interfere with information structure status [START_REF] Adamou | Prosodic and morphological focus marking in Ixcatec (Otomanguean)[END_REF].

More generally, speech production tasks are semi-experimental. For example, to investigate spatial language and cognition, participants were asked to provide a description of the localisation of two buildings while being filmed [START_REF] Calderón | How to speak "geocentric" in an "egocentric" language: A multimodal study among Ngigua -Spanish bilinguals and Spanish monolinguals in a rural community of Mexico[END_REF]. This allows to analyse both speech and co-speech gesture for all participants while controlling their position with respect to cardinal points to differentiate between spatial representations that are geocentric (based on cardinal points) and egocentric (based on the speaker's viewpoint).

Recommendations for practice

This section offers a brief practical guide to build and conduct an experiment in diverse cultural and linguistic settings following eight steps:

Step 1: Identify a research question

The point of departure in an experimental study should always be the identification and formulation of a clear research question. For researchers who are community outsiders, familiarity with the community and the language under study are essential to this endeavour as to not directly transpose research questions from well-described languages and populations to lesser-described ones, but to identify the most relevant research questions. In line with the community-based research frame, researchers should recruit research assistants from the community who will actively participate in the elaboration of the experimental design as well as its implementation. Appropriate credit and compensation are expected, including offering co-authorship in scientific papers. Reminder: as in any workplace, maintaining respectful and ethical professional relationships in the field is paramount. When applicable, it is possible to integrate the research study in a local association's activities and negotiate its goals with the association's representatives and members. Researchers who are members of the community are also concerned by the ways to best communicate and prepare their research albeit in a different way (Cruz Cruz, 2020).

Step 2. Clearly formulate your research question and predictions

In experimental work, it is important to start by formulating clear research questions and predictions. This is always done before conducting the experiments. Preregistrations of the experimental protocol, including information about the hypothesis, the data that will be collected, and their analysis are possible using the Open Science Framework (OSF) (https://osf.io/).

Step 3. Choose your experimental design

A great number of experimental paradigms are available in the literature. In practice, some leeway is allowed in adapting these designs to the field (see section 5 for some examples).

Most research institutions require that research protocols be approved by research ethics committees before implementation. Research involving vulnerable groups such as minorities and groups who are economically disadvantaged are treated with particular caution. The Statement Ethics of the American Anthropological Association is a good starting point for a general reflection on ethical issues (see http://ethics.americananthro.org/ethics-statement-0preamble/), but also see Pérez González (2021) for an Indigenous academic perspective and [START_REF] Gaby | Toward linguistic justice for Indigenous people: A response to Charity Hudley, Mallinson, and Bucholtz[END_REF] for a discussion of how linguists should relate to Indigenous peoples and their languages.

Step 4. Prepare your stimuli and build your experiment

In psycholinguistic experiments, many tasks are built around written stimuli as they present several advantages (e.g., no confound by differences in prosody; better control of the length of the stimuli). However, aural stimuli are better adapted to the study of less-described languages as these are rarely written and taught at school. When recording the stimuli, it is important to make sure not to introduce artefacts due to dialectal differences, and therefore work with speakers from within the community. A good idea is to conduct a 'norming study' prior to the experiment. In a norming study, participants not involved in the experiment rate the various stimuli on a five or seven-point scale for well-formedness (as in a judgment task; see above).

Researchers should eliminate the stimuli that receive a mean rating below a given threshold (e.g., three in a five-point scale). Length of the stimuli should be controlled for to ensure comparability and eventually allow for calculating reaction times. Fillers need to be used to distract the attention of participants from the main research question. Finally, stimuli should be randomized for each participant. Regarding visual stimuli, it is best to use coloured photographs rather than black and white line drawings given that little-educated participants have difficulties recognizing the latter [START_REF] Reis | Color makes a difference: Twodimensional object naming in literate and illiterate subjects[END_REF]. One can either use existing visual stimuli or create their own, culturally adapted stimuli (see [START_REF] Adamou | Subject preference in Ixcatec relative clauses (Otomanguean, Mexico)[END_REF][START_REF] Adamou | Prosodic and morphological focus marking in Ixcatec (Otomanguean)[END_REF][START_REF] Borja | Constituent order and parser control processes in Chamorro[END_REF][START_REF] Calderón | How to speak "geocentric" in an "egocentric" language: A multimodal study among Ngigua -Spanish bilinguals and Spanish monolinguals in a rural community of Mexico[END_REF]. To build an experiment, a free open-source experiment builder is Open Sesame [START_REF] Mathôt | OpenSesame: An open-source, graphical experiment builder for the social sciences[END_REF].

Step 5. Select your participants

Before conducting experiments in the field, it is important to have had the opportunity to conduct extensive participant observation to comprehend the sociolinguistic background and decide who could take part in the study (e.g., age range, languages spoken, etc.). At the beginning of the experimental session, experimenters can conduct a short sociolinguistic interview to assess education, literacy, dialectal variation, and multilingualism among the participants.

Prior to their participation in the experiment, participants are asked to sign a consent form or simply record their approval. This is when researchers can explain what participants will be asked to do during the experiment; that they can leave before the end of the experiment if they feel uncomfortable; why the researchers are conducting this study; what are the benefits for the community; whether participants will receive compensation and who funds the research;

where the collected data will be stored and published; how the researchers will ensure respect for the privacy and confidentiality of participants' personal information. In addition to individual consent, researchers often need to obtain community consent through the appropriate institutions (e.g., the community's general assembly) as well as national and/or local authorities.

In psycholinguistic research, it is standard practice to compensate participants for their time, but in the field, it is important to first understand cultural norms regarding financial compensation. Also keep in mind that it is not ethically acceptable to exercise pressure or allure potential low-income participants by exclusively offering financial incentives.

Alternatives to individual financial compensation may be a small gift to participants or a financial contribution to a local association.

Step 6. Collect your data

Always allow extra time for data collection in the field as participants may be busy with everyday life tasks and not be available for the experimental study as planned. Even if you cannot conduct the experiment in the lab, try to conduct it in a similar location for all the participants: a quiet room in the house of participants, a classroom at the local school, a room in a collective structure that is accessible to all the participants. Unlike standard practice in most lab research, where the presence of the researcher during the experiments is not required, in the field it is best that the researcher be present to make sure that everything goes according to plan. Moreover, the researchers who will conduct the experiment in the field must be familiar with the computer programme (e.g., Open Sesame) that they will use to check at the beginning of every session if it is recording the data properly and adjust as needed.

Step 7. Analyse your data Experimental linguists typically learn to use statistical methods for data analysis. This is an important investment in time and constant training is needed to follow developments in the field of statistics. Alternatively, collaborating with a researcher who is familiar with statistics is an excellent way to ensure state-of-the-art analyses.

Step 8. Interpret your results

Once the statistical analyses are available, researchers can confront the results with the predictions formulated at the beginning of the research project.

Future directions

Experimental linguistics is a fast-growing field. But will cross-linguistic and cross-cultural representation follow this development?

Challenges

There are several challenges to the use of experimental methods outside the lab. First, practical reasons are at play. As with any research conducted in the field, impediments include accessibility to remote locations, access to electricity, and adverse climate conditions [START_REF] Hellwig | Linguistic diversity, language documentation and psycholinguistics: The role of stimuli, Language Documentation and Conservation[END_REF]. Other practical reasons may include difficulties to transfer high-quality portable equipment to the field. However, these problems are increasingly being solved thanks to recent technological progress allowing the use of laptops, cameras, recorders, and mobile eye-trackers even in isolated places. In addition, with travel restrictions due to the Covid-19 pandemic (starting in 2020), many researchers turned to web-based research, a special type of field research that allows reaching large and heterogeneous populations using smartphones, personal computers, and the Internet [START_REF] Boase | Mobile methods: Explorations, innovations, and reflections[END_REF][START_REF] Dufau | Smart phone, smart science: How the use of smartphones can revolutionize research in cognitive science[END_REF]. Prior experience shows that response times are accurately measured when they are recorded locally and then shared via the web [START_REF] Enochson | Collecting psycholinguistic response time data using Amazon Mechanical Turk[END_REF].

Second, linguists who work in diverse linguistic and cultural settings, and language communities themselves, may have different research priorities. For example, when studying an undescribed language, the primary goal is to produce a grammatical sketch and a dictionary. When working on an endangered language, the priority is language documentation and revitalisation. Experimental work can therefore be introduced once primary goals are met.

Experimental methods are not necessarily antagonistic with these goals since they allow researchers to work with a great number of participants in a given community, whether these are speakers of the language under study or not (in the latter case they can collaborate as part of a control or comparison group; Adamou, 2021).

Finally, a major hurdle is that experiments are designed for Western, University-educated participants and that there is still little knowhow to design experiments that can work for socially and culturally diverse populations. For example, Sauerland (2018) shares his experience with the Amazonian Pirahã community. During a weeklong stay, he recruited 16 speakers to respond to a truth value judgment task that aimed to test syntactic recursion following a heated debate about whether Pirahã exhibited recursion or not. Pirahã participants were asked to listen to the pre-recorded statements of a Pirahã speaker, e.g., 'I have been to the moon,' reported accurately or inaccurately by a second Pirahã speaker, e.g., 'X said 'I have been to the sun.' Participants in the experiment had to decide whether the second speaker heard well or said the truth. The experiment did not work as expected and the author admits that one reason might be that the speakers who recorded the utterances were wellrespected senior members of the community and that it may have been culturally and socially inappropriate for the participants in the study to challenge what the elders said by qualifying their statements as 'false.' Another interesting example of both a failed and a successful experiment is related in [START_REF] Mulak | Word learning in the field: Adapting a laboratory-based task for testing in remote Papua New Guinea[END_REF]. A team of researchers, including researchers who were familiar with the community, conducted a word learning experiment with 34 Nungon speakers from Papua New Guinea. Whereas the first experiment yielded null results, the second experiment was successful after adjusting the instructions to make them clearer, reducing the cognitive load of the task, and framing the study within a real-world context.

Ways forward

As a community, we can support diversity in experimental linguistics by explicitly encouraging the submission of papers from under-described languages in conferences and following through on the publication of this research in international journals. As individual researchers, we can change our practices, even when we do not work in a cross-cultural perspective, to include a thoughtful characterisation of our sample and carefully discuss its potential for generalisation.

Last but certainly not least, we must keep in mind that cross-cultural and cross-linguistic experimental research raises specific ethical issues. Whereas researchers in most lab-based experiments are not familiar with their participants, familiarity with the communities that participate in experimental work in the field should be the starting point. Otherwise, we run the risk of turning experimental research in the field into an extractive kind of science. As in other research fields, intense collaboration with the various communities and reciprocity should be the guiding principles in this endeavour.

Further Reading

Adamou, E. ( 2021). The Adaptive Bilingual Mind: Insights from Endangered Languages.

Cambridge: Cambridge University Press. This book demonstrates how experimental psycholinguistic methods can be successfully applied outside the lab, including with speakers of endangered languages. In addition to the quantitative, experimental results, the book offers ample qualitative linguistic and ethnographic observations. Majid takes the example of olfactory language to argue that not only do we need crosscultural and cross-linguistic experimental research, but that we also need to adopt a transdisciplinary approach that includes linguistic elicitation, text analysis, conversation analysis, ethnographic observation, case studies, surveys, and computational approaches, among others. Meakins, F., Green, J. and Turpin, M. (2018). Understanding Linguistic Fieldwork. London: Routledge. This is an up-to-date student's guide to linguistic fieldwork presenting a variety of methods. The volume discusses examples from colonial and postcolonial settings, including Australia, the United States of America, Canada, South America, and Africa. Rappaport, J., (2020). Cowards don't make history: Orlando Fals Borda and the origins of

  are the first to experimentally demonstrate the relevance of crosslanguage priming in structural change in a bilingual population speaking Papiamento, a Spanish and Portuguese-based Creole, and Dutch (Germanic). The researchers tested priming in dative structures among Papiamento-Dutch bilinguals living in the Netherlands (N=37 participants) and in Aruba (N=25 participants). They found that Papiamento speakers who are in daily contact with Dutch (in the Netherlands) exhibit higher levels of priming by Dutch dative structures than those who have less intense contact with Dutch (in Aruba).

  Gillioz, C. and Zufferey, S. (2020). Introduction to Experimental Linguistics. John Wiley & Sons. This is an accessible and detailed introduction to experimentation in linguistics, from concepts to methods and data analyses. It draws on examples from the various fields of linguistics. Majid, A., 2021. Olfactory language requires an integrative and interdisciplinary approach, Trends in Cognitive Sciences. https://doi.org/10.1016/j.tics.2021.03.004

  participatory action research. Durham, NC: Duke University Press. This is a historical overview of Participatory Action Research methods focusing on the collaboration between the sociologist Orlando Fals Borda with Indigenous and peasant populations in Colombia in the early 1970s. It should help researchers reflect on what it means to conduct research in the field.


Ainhoa Zapirain

Nicolas M Édoc

Mohammad Ghoniem

A Hybrid Multi-Layer Network Visualization for Exploring Overlapping Biclusters

Keywords:

Figure 1: Visualization of overlapping biclusters extracted from a news topic related to the UN and the concerns of the peacekeepers mission in South Sudan. (a) The overview displays 112 biclusters placed column-wise in each term × bicluster matrix. They represent bicluster communities nested in a node-link diagram to help identify their commonalities and specificities in the two dimensions. (b) Zooming into two communities, we can observe which biclusters and terms are associated to the selected document.

INTRODUCTION

Large text collections contain valuable knowledge that can be hard to infer without appropriate visual analytics software. To support exploratory analysis of unstructured text collections, visual text analytics combines interactive visualization and NLP/text mining methods. Most of these methods use the popular "bag of words" model which simplifies the representation of textual data in a term * e-mail: ainhoa.zapirain@list.lu † e-mail: nicolas.medoc@list.lu ‡ e-mail: mohammad.ghoniem@list.lu × document matrix. Biclustering [START_REF] Prelić | A systematic comparison and evaluation of biclustering methods for gene expression data[END_REF] exploits this model to group rows and columns simultaneously according to the duality of the relationships between the two dimensions of the matrix. Recently, this technique has been applied on the result of topic models to extract topic variants, helping investigative journalists drill-down into topics of interest and delineate various facts and opinions shared in multiple documents [START_REF] Médoc | Visual Exploration of Topic Variants Through a Hybrid Biclustering Approach (english translation)[END_REF].

In this work we aim to address the challenge of designing a meaningful visualization of numerous overlapping biclusters, which depicts their commonalities and specificities. In a matrix visualization, reordering rows and columns reveals biclusters in contiguous blocks. However, when biclusters overlap, the rearranged blocks are forced to split which impedes understanding their composition and thus their common items. Other solutions based on node-link diagrams must be carefully designed to avoid visual clutter in dense areas of the network. To address these issues, we propose a new hybrid approach based on multi-layer network visualization allowing the analyst to focus on the commonalities and specificities of biclusters. We also provide the protocol of a controlled experiment we aim to conduct in future work.

RELATED WORK

In the literature there are multiple approaches to represent overlapping biclusters. In gene expression data, Santamaría et al. [START_REF] Santamaría | A visual analytics approach for understanding biclustering results from microarray data[END_REF] use a force directed layout to lay out the items of the same biclusters close together and surround them with convex hulls. However, it remains hard to distinguish specific and common nodes in dense areas due to the numerous overlapping hulls. In their study, Ghoniem et al. [START_REF] Ghoniem | On the Readability of Graphs Using Node-Link and Matrix-Based Representations: A Controlled Experiment and Statistical Analysis[END_REF] have shown that matrix visualization is more appropriate than node-link diagrams to analyze dense relationships in a network. NodeTrix [START_REF] Henry | NodeTrix: a hybrid visualization of social networks[END_REF] adopts a hybrid approach nesting matrix visualization in a node-link diagram to support analysis of intra-and inter-relationships between communities in social networks. In the case of biclusters, Furby [START_REF] Streit | Furby: fuzzy force-directed bicluster visualization[END_REF] shows the two dimensions of biclusters in matrices. They are nested in a node-link diagram to depict their relationships. Nonetheless, with a plethora of overlapping biclusters, edge crossing generates visual clutter.

To explore topics extracted from a large collection of texts, Médoc et al. [START_REF] Médoc | Visual Exploration of Topic Variants Through a Hybrid Biclustering Approach (english translation)[END_REF] draw an overview of a thousand of overlapping term × document biclusters. They are conveyed as a term hierarchy enabling quick exploration of the term sets of biclusters by grouping the most common terms at the top levels and branching the more specific term variations deeper in the tree. Such an approach allows to depict commonalities and specificities of biclusters but in one dimension only. In order to address these issues, we propose a nested visualization [START_REF] Javed | Exploring the design space of composite visualization[END_REF], built upon a multilayer network model [START_REF] Mcgee | The state of the art in multilayer network visualization[END_REF] supporting the analysis of many overlapping biclusters in the two dimensions simultaneously.

THE HYBRID APPROACH

Data model. To build a visual text analytics solution, we first apply topic modeling algorithms on a term × document matrix, each document being a vector of distinct terms with TF-IDF weights. In this work, we define a topic as a sub-matrix containing the subsets of terms and documents associated to it by any topic modeling algorithm. To support thorough scrutiny of each topic, the various shades of topics are extracted from each sub-matrix by Bimax biclustering algorithm [START_REF] Prelić | A systematic comparison and evaluation of biclustering methods for gene expression data[END_REF]. Bimax biclusters capture the distinct combinations of terms shared by multiple documents.

Visual Design. The objective of the visualization is to help drill down into any topic of interest by conveying all extracted biclusters while fetching their commonalities and specificities in the two dimensions. From the term × document bicluster results, we build a multi-layer graph where nodes can be of three types: documents, terms and biclusters. Edges link terms and documents to the biclusters in which they appear. To reduce edge crossing and visual clutter, our approach consists in representing highly connected biclusters as a matrix visualization (see Figure . 1a). We therefore apply a community detection algorithm [START_REF] Clauset | Finding community structure in very large networks[END_REF] to a graph where a pair of biclusters are linked by an edge if they share documents or terms. As a result, each community of biclusters is displayed in a matrix where columns are biclusters and rows are terms. The analyst can thus quickly interpret the community at a glance by reading vertically the labels aligned on the right side (see Figure 1b). To facilitate the identification of different terms and their repetition in other communities, we assign a distinct color to each term. The documents are however drawn as circle shapes and connected to individual biclusters inside the communities. In orange and blue, we distinguish documents shared between different communities from those specific to one only.

Layout. The nodes are placed strategically with a force-directed layout to reduce edge crossings. At each iteration we consider the centroids of groups to move similar communities closer to each other. Groups of communities are computed with a Jaccard-based distance taking into account the shared terms and documents. According to Figure 2, the positions of communities and their shared documents are set by the force-directed layout while the position of specific documents is delayed until the stabilisation of the layout.

DESIGN OF A CONTROLLED EXPERIMENT

We aim to compare our hybrid visualization with Furby's design [START_REF] Streit | Furby: fuzzy force-directed bicluster visualization[END_REF] where matrices represent individual biclusters and the hierarchical approach proposed in [START_REF] Médoc | Visual Exploration of Topic Variants Through a Hybrid Biclustering Approach (english translation)[END_REF]. The comparison is based on four tasks: 1) Identify common and distinct terms between two selected biclusters in the same community ; 2) Identify common and distinct documents between two selected biclusters in two different communities ; 3) Find all the biclusters containing one given document ; 4) Find all the biclusters containing one given term. We are considering a between-subject experimental design such that the subjects will use either multi-layer, Furby's or hirarchical design, to carry out the four different tasks on 9 topics of 3 different sizes. To minimize bias, we will randomize the topic order presented to each subject. One additional topic will be used to train the subjects on the visualization and the task execution. User performance is assessed by task completion time and accuracy. Accuracy is measured by a Jaccard distance between the target sets and user answers for each task.

CONCLUSION

The hybrid visualization proposed in this work conveys overlapping biclusters by displaying bicluster communities in a term × bicluster matrix nested in a node-link diagram. In future work, we aim to run a user study showing the performance of three alternate visual designs: our hybrid approach based on multi-layer network visualization, a node-link diagram nesting term × document matrices and a hierarchical approach proposed in [START_REF] Médoc | Visual Exploration of Topic Variants Through a Hybrid Biclustering Approach (english translation)[END_REF], with different topic sizes.

Figure 2 :

 2 Figure 2: Three steps are applied to position the nodes: the placement of (1) group centroids and (2) communities with their shared documents alternates during the force-directed layout iterations; (3) in the end, the placement of specific documents is relative to their communities.