Loïc Caroux
email: loic.caroux@univ-tlse2.fr

Presence in video games : A systematic review and meta-analysis of the effects of game design choices

Keywords: Game user research, player experience, visual display, multiplayer, questionnaire, scale

Presence in video games: A systematic review and meta-analysis of the effects of game design choices

Introduction

Presence is the feeling of being inside a virtual environment (see for reviews [START_REF] Felton | Presence: A review[END_REF][START_REF] Wilkinson | A mini review of presence and immersion in virtual reality[END_REF]. This psychological concept is a widely assessed dimension of a user's experience interacting with a computer-generated environment. The sense of presence is usually described as a three-dimensional construct [START_REF] Felton | Presence: A review[END_REF][START_REF] Terkildsen | Measuring presence in video games: An investigation of the potential use of physiological measures as indicators of presence[END_REF].

Spatial presence (or physical presence) is the dimension of presence that describes the user's feeling of being physically in the virtual environment and the ability to interact with that environment and the objects in it as if it were real. Social presence is the dimension that describes the feeling of coexistence with other beings in the virtual environment. Self-presence is the dimension that describes the feeling of extension and prolongation of the user's body in the avatar displayed in the virtual environment as if it were their own body.

Presence can be particularly experienced by players interacting with video games [START_REF] Terkildsen | Measuring presence in video games: An investigation of the potential use of physiological measures as indicators of presence[END_REF]. Video games are interactive systems in which users interact with a virtual environment for the primary purpose of fun and entertainment [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF]. This virtual environment is usually displayed via an audio-visual terminal. The player sends commands to the environment via an artifact (gamepad, motion controller, etc.) and the environment evolves according to these commands. The sense of presence is one of the dimensions that can be assessed when studying the player experience [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF][START_REF] Terkildsen | Measuring presence in video games: An investigation of the potential use of physiological measures as indicators of presence[END_REF]. Player experience is the individual, personal experience held by the player during and immediately after playing the game [START_REF] Wiemeyer | Player experience[END_REF]. Player experience includes dimensions related to enjoyment, such as flow and emotions, and engagement, such as immersion and presence. Note that presence and immersion are two concepts that are frequently studied to assess player experience and may be still used interchangeably in some studies. However, several authors have made efforts to differentiate these constructs (see for example [START_REF] Cairns | Who but not where: The effect of social play on immersion in digital games[END_REF] or [START_REF] Nilsson | Immersion Revisited: A review of existing definitions of immersion and their relation to different theories of presence[END_REF]). For example, immersion can be defined as "the sensation of being surrounded by the virtual environment that increases proportionally with the number of modalities provided with artificial stimuli" [START_REF] Mcmahan | Immersion, engagement, and presence: A method for analyzing 3-D video games[END_REF][START_REF] Nilsson | Immersion Revisited: A review of existing definitions of immersion and their relation to different theories of presence[END_REF]. Immersion is primarily relative to sensory perception, while presence, a feeling of being "in the game", is a more mental, psychological perception.

Although presence is a widely used construct for assessing player experience, it is not always clear how the sense of presence is influenced by game design choices. A previous systematic review of the literature by [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF] showed that player experience can be influenced by various hardware and software aspects of video games. Three main categories of game design were highlighted in this previous review: input/output information (i.e. display and control modes), ingame contents (i.e. gameplay mechanics, narrative and challenge), and multiplayer aspects (i.e., nature of the co-player, co-playing mode, and online gaming techniques). Presence was addressed in this previous review, but along with several other dimensions and without quantitative evidence on the specific impacts of game design factors. Therefore, the results may be of limited use to researchers and designers who wish to know the specific effect of each design choice on presence. In addition, the number of studies that have investigated presence in video games has increased significantly over the past decade. For example, the impact of recent technological innovations in video games, such as virtual reality (VR) systems (e.g., [START_REF] Marre | Video game interfaces and diegesis: The impact on experts and novices' performance and experience in virtual reality[END_REF], was investigated in a rather embryonic fashion in this previous review, due to the small number of published studies at that time. Some of these have subsequently been addressed in more empirical studies published in the interim.

Furthermore, the effects of game design choices on player experience, including the dimension of presence, have been investigated in numerous experimental studies over the past two decades [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF]. Several published articles have provided state of art, literature surveys or reviews of what we know about the influence of various design choices on a given dimension of player experience, such as enjoyment [START_REF] Mekler | A systematic review of quantitative studies on the enjoyment of digital entertainment games[END_REF][START_REF] Schaffer | Digital game enjoyment: A literature review[END_REF][START_REF] Segundo Díaz | Building blocks for creating enjoyable games-A systematic literature review[END_REF], flow [START_REF] Khoshnoud | Peripheral-physiological and neural correlates of the flow experience while playing video games: A comprehensive review[END_REF], engagement [START_REF] Boyle | Engagement in digital entertainment games: A systematic review[END_REF] or emotions [START_REF] De Byl | A conceptual affective design framework for the use of emotions in computer game design[END_REF], or on the influence of a given design characteristic on the overall player experience, such as affective-based game adaptation [START_REF] Bontchev | Adaptation in affective video games: A literature review[END_REF][START_REF] Ng | A review of affective user-centered design for video games[END_REF], personalization techniques [START_REF] Karpinskyj | Video game personalisation techniques: A comprehensive survey[END_REF], brain-computer interfaces [START_REF] Marshall | Games, gameplay, and BCI: The state of the art[END_REF], or game accessibility [START_REF] Spiel | The purpose of play: How HCI games research fails neurodivergent populations[END_REF]. Their contribution is of course of great interest to the game user research community, including researchers and practitioners, but remains limited due to their solely descriptive nature. Their usefulness decreases when it comes to knowing the precise effect of each design choice or helping a game designer choose an element that would have a specific desired impact on dimensions of player experience like presence. We believe that the size of the body of experimental game user research literature is now large enough to allow more comprehensive and in-depth analyses of the data from this research such as meta-analyses.

The purpose of the present study is to focus in depth on the concept of presence in the player experience in video games. The main objective is to provide a more comprehensive view of the elements of game design that have an effect on the sense of presence in video games. This study is based on a systematic review of literature that described the different categories of game design elements that have been studied, but also explored the different studied dimensions of presence and the different techniques to assess it. In addition, a meta-analysis was conducted to assess the impact of the specific game design elements that were evaluated in these studies. The goal is to quantify the effect of these game design choices on the sense of presence based on a combination of data from different experimental studies. The results of the meta-analysis can help to better understand the specific effect of each design choice on presence and thus optimize the player experience when designing video games.

Methods

Literature search

The literature search and selection were performed in major online databases (Web of Science Core Collection, PsycInfo, Medline) to identify relevant studies for inclusion in this review. The search covered published research articles written in English and published through 2021, with no minimum year. The first step of the literature search was to use keywords related to video games and player experience. The search topic was intentionally broader than just the concept of presence. The search also included other main dimensions of player experience in order to capture as many articles in this area as possible. Some articles might have studied presence as a secondary dimension and therefore did not mention it in the title or abstract. Similarly, this approach allowed for the inclusion of studies in which presence was assessed among other dimensions but did not show a significant effect.

Therefore, the search query was performed with the following keywords related to video games (video games, digital games, computer games, electronic games, or videogames), player (play, player, or playing), and experience (experience, enjoyment, engagement, immersion, presence, flow, or emotion). A few articles were added manually after checking the databases of previous published literature reviews on player experience. The database search yielded 5,360 published articles after duplicate removing.

The articles were included according to the following inclusion criteria. The articles had to investigate the influence of game design factors (software and/or hardware) on player experience. Because the main objective of the present study was to focus on the elements of game design that have an effect on the sense of presence in video games, this review was based on studies that compared one given modality of game design factor to another (e.g., a rather recent display mode such as head-mounted display vs. a more traditional mode such as a monitor display). Thus, the articles were required to include controlled experiments (i.e. randomized controlled trials or within-participant experiments) that assessed the efficacy of these design factors on player experience compared to a "control condition". In addition, the objective of this review was to study player experience in a situation where entertainment was not competing with another user goal such as learning, training, etc.

Games for which entertainment is not the primary goal (e.g., serious game, game-based learning, rehabilitation game, physical training exergame) are designed to help the user achieve a primary goal (e.g., academic learning, training, cognitive rehabilitation, skill enhancement, behavioral change, etc.) that is not inherently related to the player-video game interaction (e.g., [START_REF] Krath | Revealing the theoretical basis of gamification: A systematic review and analysis of theory in research on gamification, serious games and game-based learning[END_REF][START_REF] Shahmoradi | A systematic review on serious games in attention rehabilitation and their effects[END_REF]. The present review aimed to focus on player experience data (including presence) collected in situations uniquely designed to achieve optimal player experience. Mixing data from studies with different user goals could bias the results of the review and meta-analysis. Thus, an excluding criterion was articles that studied games for which entertainment is not the primary goal.

Each abstract in the search database was reviewed for compatibility between the article and the inclusion criteria. When the abstract was ambiguous, the whole text was screened. This review was done independently by two persons, including the author of this article. The two reviewers then compared their results. If there was a disagreement about an article, the two reviewers discussed their choice and together decided to include or exclude the article. A total of 215 articles, published between 2000 and 20211 , met the inclusion criteria. The next step was to limit the search to articles that investigated the influence of game design factors on presence. Articles reporting studies that assessed presence, alone or among other dimensions of player experience, were included in the final database of this review. A total of 55 articles, published between 2004 and 2021, met the final inclusion criteria and were included in this review.

Data extraction

The data extracted from the studies reported in the remaining 55 articles concerned the type of game design characteristic studied and the modalities manipulated, the different measures (presence and other dimensions of player experience), and the type of technique for measuring presence.

Meta-analysis

Next, another selection was made from the studies reported in the included articles to further quantitative analyses. The studies were selected based on the categories of game design factors and presence dimensions. A study was included if there were at least two studies in the database that investigated the effect of the same game design factor on the same presence dimension, and for which corresponding data were available. The data were generally directly available in the article.

Where data were not directly available, the authors of the article were contacted to provide the data in question. If data were not available from the authors or if there was no response from them (mainly for older articles), the study was not included in the meta-analysis. A total of 38 studies, from 31 articles, were included in the meta-analysis.

Statistical analysis

The means and standard deviations (SD) for dependent variables were used to calculate effect sizes (ES; Hedges' g) for each outcome in the experimental and control conditions. The experimental condition typically involved the primary modality of the game design factor studied, and the control condition involved the classic modality of the game design factor or its absence (see the results section for the details of the different groups compared). Data were standardized using postintervention SD values. A random-effect model was chosen to analyze the effect of game design factors because the studies included in this meta-analysis used similar but not identical methods of measurement of presence [START_REF] Deeks | & on behalf of the Cochrane Statistical Methods Group[END_REF]. The ES values are presented with 95% confidence intervals (CI). Calculated ES were interpreted using the following scale: < 0.2, trivial; 0.2-0.6, small; > 0.6-1.2, moderate; > 1.2-2.0, large; > 2.0-4.0, very large; > 4.0, extremely large [START_REF] Hopkins | Progressive statistics for studies in sports medicine and exercise science[END_REF].

Heterogeneity was assessed using the I² statistic [START_REF] Higgins | Quantifying heterogeneity in a meta-analysis[END_REF]. Statistical significance was set at p ≤ 0.05. The analysis was performed with Review Manager 5.4.1 software (The Cochrane Collaboration, 2020).

Results

Dimensions of presence assessed

The systematic review revealed that presence was assessed as a global concept (global presence) or according to one or more of its dimensions (spatial presence, social presence, self-presence). Table 1 shows the repartition of the articles included in the systematic review according to the dimension(s) of presence assessed. Most articles (58%) studied presence as a global concept. Spatial presence was studied by 33% of the articles. Social presence (16%) and self-presence (7%) were studied by fewer articles. It should be noted that an article may aim to assess global presence and a specific dimension of presence, but also more than of one dimension at a time. The complete list of articles included in the systematic review is detailed in Appendix. Note. An article that studied more than one dimension of presence, and/or more than one game design factor was counted in multiple cells. Therefore, the total or sub-total numbers are not the sum of the corresponding column or raw.

Techniques used to measure presence

All the articles included in this review studied presence using questionnaires and scales. Table 2 presents the details of the questionnaires and scales used according to the dimension of presence assessed. Most articles used standardized scales. A very small proportion used ad hoc questionnaires (i.e. designed by the article authors without prior validation). Some scales were designed entirely to study presence (e.g., Slater-Usoh-Steed Presence Questionnaire (SUS) [START_REF] Slater | Depth of presence in virtual environments[END_REF]), while others are only sub-dimensions of a more general scale, which is designed to assess the whole user experience for example (e.g., Presence-Immersion dimension of the Player Experience of Need Satisfaction Scale (PENS) [START_REF] Ryan | The motivational pull of video games: A selfdetermination theory approach[END_REF]). Another discriminating feature of these scales is that some are specific to video games (e.g., Social Presence in Gaming Questionnaire (SPGQ) [START_REF] De Kort | Digital games as social presence technology: Development of the Social Presence in Gaming Questionnaire (SPGQ)[END_REF]), while other are designed to assess presence (or user experience) in more diverse systems than video games (e.g., Independent Television Commission-Sense of Presence Inventory (ITC-SOPI) [START_REF] Lessiter | A cross-media presence questionnaire: The ITC-Sense of Presence Inventory[END_REF]).

It should be noted that presence is rarely the only one dimension of player experience that has been assessed in the included articles. Enjoyment, flow, or emotions are examples of other dimensions that may have been assessed in some articles. This review does not attempt to detail these elements since its purpose is to focus solely on presence. Note. An article that studied more than one dimension of presence, and/or used more than one questionnaire or scale was counted in multiple cells.

Game design factors studied

Table 1 shows that presence has been studied in articles that involve many different game design factors. These game design factors fall under the major aspects of video games studied in game user research, namely information input/output techniques (32 articles), multiplayer aspects (14 articles), and in-game contents (12 articles). The following paragraphs detail the different game design factors that were studied in the articles included in the review.

Information input/output techniques

Sixteen articles studied the effects of visual display type. These articles focused on innovative methods of visual information display, such as HMDs (7 articles) and 3D-stereoscopic displays (6 articles). Other less studied concepts (one article) included screen size, fidelity, and image display quality. Fourteen articles studied the effects of control mode. Most of these articles studied motionbased controllers (9 articles). These controllers include tangible controllers (e.g., Wiimote, PS Move, or steering wheel controllers) or full body controllers (e.g., Kinect sensor). Other means of control, through speech or brain activity, and concepts related to control, such as control fidelity or aiming type in a shooting game, were also studied, but in only one article. Four articles investigated the effects of systems that combine an HMD with motion-based controllers, such as in VR game systems.

Other input/output information game design factors were studied by only one article per concept.

Regarding visual interfaces, point of view (1 st or 3 rd person), diegesis (how contextual information is presented), and graphical fidelity were studied. Regarding auditory interfaces, the presence of music and the fidelity of sound were studied.

Multiplayer aspects

Nine articles investigated the effects of the nature of the co-players. The co-players are other players in a same game environment, which can be controlled by another human being or by the computer.

Five articles studied the effect of the co-playing mode. In multiplayer games, the main goal may be shared among players and they must achieve the goal together (collaboration and cooperation), or players may have to act against their rivals to achieve victory (competition). Three articles investigated the effect of online playing (i.e. interacting in the same game environment without being in the same physical location).

In-game contents

Four articles investigated the effect of avatar appearance (representation of the player in the game environment). Two of them focused on the possibility to customize the appearance of the avatar.

One of them was about the choice between pre-defined appearances. The last one was about the possibility of having an avatar that physically resembles the player. Three articles investigated the effects of narrative in video games, which concerns how a context is included in the game environment. Two articles studied the effect of contextual in-game story on presence. One article studied the effect of a virtual agent (computer-controlled social entity) in the game environment.

Several other aspects of in-game contents were studied, but in only one article each. The effects of the type and variety of rewards, which were given to players after completing an objective for example, the level of difficulty, and the game mechanics (e.g., rules, main goal, or social aspect of the game) were also studied.

Meta-analysis

The meta-analysis was conducted based on the influence of different game design factors on global presence, and on each sub-dimension of presence. A total of 16 analyses were conducted: 6 analyses for global presence, 7 analyses for spatial presence, 2 analyses for social presence, and 1 analysis for self-presence. Table 3 provides a summary of all effect sizes calculated in this article, which are detailed in the following paragraphs. Note that the values of all effect sizes reported in the following paragraphs and tables are positive, meaning that if there is a statistically significant effect, it means that the feeling of presence is greater in the experimental condition than in the control condition.

Effects of game design factors on global presence

Table 4 presents the results of the meta-analyses of the influence of game design factors on global presence. There was a significant small effect of HMDs vs classic monitor displays on global presence.

There was a significant large effect of combined HMD & motion-based control devices vs classic devices (monitor display and classic controller, i.e. gamepad or keyboard/mouse set) on global presence. There was a significant moderate effect of human co-players vs computer co-players on global presence. There was a significant small effect of avatar choice vs no avatar choice (i.e. default avatar) on global presence.

Effects of game design factors on spatial presence

Table 5 presents the results of the meta-analyses of the influence of game design factors on spatial presence. There was a significant moderate effect of HMDs vs monitor displays on spatial presence.

There was a significant moderate effect of the presence of a contextual story vs the absence of story on spatial presence.

Effects of game design factors on social presence

Table 6 presents the results of the meta-analyses of the influence of game design factors on social presence. There was a significant very large effect of human co-players vs computer co-players on social presence. There was a marginally significant large effect of multiplayer cooperation/collaboration mode vs competition mode on social presence.

Effects of game design factors on self-presence

Table 7 presents the single result of the meta-analysis of the influence of a game design factor on self-presence. There was a significant moderate effect of the presence of a story vs the absence of a story on spatial presence.

Discussion

Main findings and research implications

The first objective of the present research was to provide a more comprehensive view of the elements of game design that have an effect on the sense of presence in video games. The results showed that many major categories of game design aspects are well represented in the literature that was reviewed. The impact of input/output information techniques, in-game contents, and multiplayer aspects on player presence have been investigated in numerous articles. However, the proportion of articles is not homogeneous within these categories. The most studied factors are mainly related to input/output information techniques and multiplayer aspects. The factors related to the in-game contents have been studied in a more disparate way.

Regarding input/output information techniques, the way visual information is displayed was particularly studied. The meta-analysis showed that displaying visual information via an HMD (as opposed to a classic monitor display), had a small effect on global presence and a moderate effect on spatial presence. Interestingly, in combination with motion controllers in a VR game system, the effect size on global presence was larger, while the effect of motion controllers alone did not reach significance.

In terms of the multiplayer aspects, the nature of co-player and the mode of co-playing were extensively studied and had some of the largest effects on presence in the meta-analysis. Playing with a human co-player (vs a computer-controlled co-player) had a moderate effect on global presence, and even a very large effect on social presence. The cooperation/collaboration mode (as opposed to competition mode) in a multiplayer game had a large effect on social presence.

Factors related to in-game contents are much less represented in the included articles. Only the effects of in-game story presence and choice of avatar appearance were investigated in the metaanalysis (intermediate effect of story presence on spatial presence and self-presence; small effect of player avatar choice on global presence).

While the most essential game design factors have been addressed in this present research regarding input/output information and multiplayer aspects, this is not the case for in-game contents.

Important elements of in-game contents, such as challenge and difficulty (e.g., [START_REF] Denisova | Measuring perceived challenge in digital games: Development & validation of the challenge originating from recent gameplay interaction scale (CORGIS)[END_REF][START_REF] Vahlo | Challenge types in gaming validation of videogame Challenge Inventory (CHA)[END_REF], have not been well addressed in the included articles. Other elements related to gameplay, such as game rules, game mechanics and functionality (e.g., Segundo [START_REF] Segundo Díaz | Building blocks for creating enjoyable games-A systematic literature review[END_REF] were also not addressed. Future research needs to study game design factors more broadly in order to gain a more comprehensive view of their influence on the sense of presence in video games.

The second objective of this research was to explore the different dimensions of presence studied in player-computer interaction, as well as the different techniques used to assess it. The results showed that presence has mainly been studied as a global construct. Few articles have studied the separate dimensions of presence. Of the three dimensions of presence, spatial presence has been the most studied, primarily in relation to input/output information techniques. Social presence has been less studied, primarily in relation to aspects of multiplayer games. Self-presence has been studied even less.

The fact that global presence was the most studied concept in the reviewed literature may be related to the fact that many articles have not focused on presence solely for the purpose of studying player experience. Global presence may be considered a sufficient assessment item when studying the overall player experience. However, this could be a real problem if the goal is to get a clear picture of the impact of a game design choice on presence. It may be crucial to know which dimension of presence is specifically impacted by a given design choice. As an example, the meta-analysis showed that the choice of avatar appearance has a significant effect on global presence. But it would have been more informative to study each specific dimension of presence. In particular, we could hypothesize that self-presence would be particularly enhanced in such a case since an avatar is a prolongation of the self in the virtual environment. Studies on multiplayer aspects illustrate this point perfectly. They showed that global presence was positively impacted by the nature of the co-player with a moderate effect size. However, when social presence is specifically studied, which could be considered as a naturally relevant dimension for this game design factor, the studies showed a very large effect size. In the same line, the meta-analysis did not reveal a significant effect of control mode on global presence or spatial presence. Given the nature of the different dimensions of presence, it would have been interesting to study the impact of these design factors on self-presence. It can be hypothesized that the mode of control is a type of prolongation of the self in the virtual environment and therefore has an impact on the sense of self-presence. Specific dimensions of presence should be studied more in future research, instead of, or at least in addition to, global presence. Targeting a specific dimension in relation to the nature of the game design factor would be particularly relevant. More generally, studies on social presence and self-presence are lacking and should be further explore in future research.

In terms of techniques for measuring presence, this review found that numerous questionnaires and scales were used in the included articles. While it is good that validated scales were widely used, it is unfortunate that there are very diverse. Given their heterogeneity, this may limit the validity of the comparison of results to some extent. The results of meta-analyses may be more robust when the measurement instruments are homogeneous. Future research should attempt to use similar scales to assess presence in video games. For example, the use of some global, presence-specific scales, such as the MPS [START_REF] Makransky | Development and validation of the Multimodal Presence Scale for virtual reality environments: A confirmatory factor analysis and item response theory approach[END_REF], have the advantage of assessing global presence and each subdimension at the same time. Unfortunately, the review did not reveal any such scales constructed specifically for video games.

Finally, the results of the literature review showed that presence was exclusively assessed with subjective questionnaires and scales. This is not surprising given that presence is primarily a concept that the player can experience when playing video games. Declarative instruments seem to be the best for collecting the player's impressions of their experience during their interaction with a game.

However, other techniques based on the player's behavior or physiological state exist and have been validated, included for video game situations (e.g., [START_REF] Terkildsen | Measuring presence in video games: An investigation of the potential use of physiological measures as indicators of presence[END_REF]. It may be worthwhile for future studies to include these techniques, instead of or in addition to subjective techniques, in order to have more varied data to characterize presence in video games.

Design recommendations

One of the benefits of this meta-analysis is that the effects of game design choices on the sense of presence were quantified and their size calculated based on the combination of data from different experimental studies. Therefore, these results may help to better optimize player experience in video game design than the results from isolated studies. Some design recommendations with their expected impact on the players' sense of presence, based on these results, are provided to game designers in Table 8.

Table 8. Design recommendations based on significant effects from the meta-analysis.

Game design factor Design recommendation

Expected impact on presence dimensions (based on effect sizes of the meta-analysis)

Global presence

Spatial presence

Social presence

Selfpresence

Display mode

Display the virtual environment in an HMD device (vs. a monitor display)

Limitations

As with many systematic reviews and meta-analyses, the results of this study may be limited by the body of literature reviewed, while the total number of articles included in this review was quite large given the relative recency of the field of game user research.

The limitations of this review are reached when it comes to exploring some specific game design factors or presence dimensions in more detail. For example, several game design factors described in the review were not included in the meta-analysis because they were unique in their research question (e.g., visual interface or online playing). Similarly, the meta-analysis includes comparisons with a small number of studies (2 or 3 studies), while other included a large number (5 or 6 studies).

This may limit the validity of the findings.

In addition, as noted above, the variety of comparisons may be considered as a limitation of the meta-analysis. While many game design factors were investigated in the included studies, not all existing factors explored in the literature on player-computer interaction were investigated in relation to the sense of presence (e.g., challenge, game rules, etc.). Similarly, the influence of included game design factors has not been studied on all dimensions of presence (e.g., the effect of control mode on self-presence). This may also limit the validity of the results.

Perspectives

The present research focused solely on presence, which is only one dimension of player experience in video games. The goal was to isolate this concept to specifically study the effects of game design factors on it. Future research should then focus on other dimensions of player experience. Two directions can be explored by these future studies.

First, future meta-analyses should be conducted on the other main dimensions of player experience, namely immersion, flow, and enjoyment for the most important ones. Some literature reviews already exist, such as general reviews on player-computer interaction (e.g., [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF], but also more specific reviews on enjoyment (e.g., Segundo [START_REF] Segundo Díaz | Building blocks for creating enjoyable games-A systematic literature review[END_REF] or flow (e.g., [START_REF] Khoshnoud | Peripheral-physiological and neural correlates of the flow experience while playing video games: A comprehensive review[END_REF]. But none of them provide quantitative evidence of the effects of game design choices on these essential dimensions of player experience.

Second, the concept of presence should be studied in relation to these other dimensions of player experience. The connection between presence and other dimensions have sometimes been studied, but mostly in a theoretical manner, such as the connection between flow and immersion [START_REF] Michailidis | Flow and immersion in video games: The aftermath of a conceptual challenge[END_REF]. Most of the articles included in this review have studied each dimension in isolation.

Future studies should examine the links between presence and other dimensions when investigating the effects of game design choices.

Conclusion

The present article reports a systematic review and metanalysis of the effects of game design factors on the sense of presence. The systematic review showed that many major categories of game design aspects are well represented in the reviewed literature. The most studied factors are mainly related to input/output information techniques and multiplayer aspects. Factors related to in-game contents have been studied in a more disparate manner. The meta-analysis showed that the way visual information is displayed, as well as the nature of the co-player and the mode of co-playing in multiplayer games have the largest effects on presence. The systematic review also showed that presence was primarily assessed using subjective questionnaires and scales, but that the wide variety of these instruments limits comparison across studies. This study revealed that future research should focus on investigating game design factors in a more targeted manner, systematically assessing presence with its most relevant sub-dimensions, and using more similar scales to assess presence. Finally, one of the benefits of this meta-analysis is that the effects of game design choices on the sense of presence were quantified and their size calculated based on combining data from different experimental studies. Therefore, these results can help to better understand the specific effect of each game design choice and thus optimize player experience in video game design than the results of isolated studies.

Table 1 .

 1 Number of articles included in the systematic review database by game design factor and presence dimension assessed.

				Presence dimension assessed
	Game aspect	Game design factor	Global presence	Spatial presence	Social presence	Self-presence	Total number of articles
		Display mode	10	6	1	1	16
		Control mode	6	8	-	1	14
	Input/output	Combined display & control device	3	-	1	-	4
	information	Visual interface	3	1	-	1	3
		Auditory interface	1	1	-	-	2
		Sub-total	20	15	2	3	35
		Co-player nature	5	2	3	-	9
	Multiplayer	Co-playing mode Online playing	2 -	--	3 3	--	5 3
		Sub-total	6	2	7	-	14
		Avatar	4	-	-	-	4
		Narrative	2	2	1	1	3
	In-game	Gameplay	2	1	-	-	3
	contents	Reward	2	-	-	-	2
		Difficulty	1	-	-	-	1
		Sub-total	10	3	1	1	12
	Total number of articles	32	18	9	4	55

Table 2 .

 2 Questionnaires and scales used in the included articles to assess presence

				Sub-	Specific to
	Presence		Specific to	dimension	video	Number
	dimension	Instrument	presence	of a more	game	of
	assessed		assessment	general	player	articles
				instrument	experience
		Presence-Immersion dimension of the Player Experience of Need Satisfaction Scale (PENS) (Ryan et al., 2006)		X	X
		Slater-Usoh-Steed Presence Questionnaire (SUS) (Slater et al., 1994)	X
		Presence dimension of the Game Engagement Questionnaire (GEQ) (Brockmyer et al., 2009)		X	X
		Verbal presence measure (Schneider et al., 2004)	X		X
		Telepresence scale (Kim & Biocca, 1997)	X
	Global	Presence questionnaire (Witmer & Singer, 1998)	X
	presence	Igroup Presence Questionnaire (IPQ) (Schubert et al., 2001)	X
		Independent Television Commission-Sense of Presence Inventory (ITC-SOPI) (Lessiter et al., 2001)	X
		Pictorial Presence Self-Assessment-Manikins (SAM) questionnaire (Weibel et al., 2015)	X
		Multimodal Presence Scale (Makransky et al., 2017)	X
		Presence Scale (Persky & Blascovich, 2008)	X		X
		Ad-hoc questionnaire
		Spatial presence dimension of the Independent Television
		Commission-Sense of Presence Inventory (ITC-SOPI)	X	X
		(Lessiter et al., 2001)
		Measurement, Effects, Conditions -Spatial Presence Questionnaire (MEC-SPQ) (Vorderer et al., 2004)	X
		Spatial Presence dimension of the Temple Presence Inventory (TPI) (Lombard & Ditton, 2000)	X	X
	Spatial presence	Spatial presence dimension of the Igroup Presence Questionnaire (IPQ) (Schubert et al., 2001) Spatial Presence Experience Scale (SPES) (Hartmann et al., 2016)	X X	X
		Spatial presence dimension of the Multimodal Presence Scale (Makransky et al., 2017)	X	X
		Physical presence dimension of the Sense of presence questionnaire (Lee et al., 2005)	X	X
		Ad-hoc questionnaire
	Social presence	Social Presence in Gaming Questionnaire (SPGQ) (De Kort et al., 2007)	X		X
		Self-presence dimension of the Sense of presence questionnaire (Lee et al., 2005)	X	X
	Self-presence	Self-presence dimension of the Multimodal Presence Scale (Makransky et al., 2017)		X
		Ad-hoc questionnaire

Table 3 .

 3 Summary of effect sizes of the influence of different game design factors on each dimension of presence

	Game design	Experimental	Control		Effect size and 95% interval confidence
	factor	condition	condition	Global presence	Spatial presence	Social presence	Self-presence
		HMD	Monitor display	.42 [.04, .80] *	.80 [.60, 1.01] *	-	-
	Display mode	3D-
		stereoscopic	2D display	.10 [-.20, .40]	.21 [-.24, .65]	-	-
		display
		Motion-based control	Classic control	.22 [-.13, .58]	.12 [-.05, .30]	-	-
	Control mode	Motion-based tangible controller	Classic controller	-	.06 [-.12, .23]	-	-
		Body motion-based control	Motion-based controller tangible	-	.23 [-.22, .68]	-	-
	Combined	HMD &	Classic display
	display &	motion-based	and control	1.64 [.89, 2.40] *	-	-	-
	control device	control device	device
	Auditory interface	Music presence	Music absence	-	.20 [-.28, .68]	-	-
	Co-player nature	Human co-player	Computer co-player	1.04 [.38, 1.70] *	-	2.17 [1.62, 2.73] *	-
	Co-playing mode	Cooperation/ collaboration mode	Competition mode	-	-	1.59 [-.04, 3.22] *	-
	Avatar	Avatar choice	Default avatar	.33 [-.01, .66] *	-	-	-
	Narrative	Story presence	Story absence	-	.80 [.24, 1.36] *	-	1.05 [.47, 1.62] *
	Note. * denotes a statistically significant effect

Table 4 .

 4 There were non-significant effects of 3D-stereoscopic displays vs classic 2D displays, and motionbased control (i.e. with a tangible or full body controller) vs classic control (i.e. with a gamepad or a keyboard/mouse set) on global presence. Meta-analysis of effects of game design factors on global presence

	Game design factor	Experimental condition	Control condition	Number of studies	Effect size	95% Lower limit confidence interval	95% Upper limit confidence interval	z-value	p-value	I²
		HMD	Monitor display	3	.42	.04	.80	2.16	.03	36%
	Display
	mode	3D-
		stereoscopic	2D display	3	.10	-.20	.40	.66	.51	28%
		display
	Control mode	Motion-based control	Classic control	2	.22	-.13	.58	1.24	.21	17%
	Combined display & control device	HMD & motion-based control device	Classic device display and control	3	1.64	.89	2.40	4.25	<.001 61%
	Co-player nature	Human co-player	Computer co-player	5	1.04	.38	1.70	3.08	.002	90%
	Avatar	Avatar choice	Default avatar	2	.33	-.01	.66	1.92	.05	0%

Table 5 .

 5 There were non-significant effects of 3D-stereoscopic displays vs 2D displays, motion-based control vs classic control, motion-based tangible controllers vs classic controllers, body motion-based control vs motion-based tangible controllers, and in-game music presence vs music absence on spatial presence. Meta-analysis of effects of game design factors on spatial presence

	Game design factor	Experimental condition	Control condition	Number of studies	Effect size	95% Lower limit confidence interval	95% Upper limit confidence interval	z-value	p-value	I²
		HMD	Monitor display	3	.80	.60	1.01	7.77 <.001 0%
	Display
	mode	3D-
		stereoscopic	2D display	2	.21	-.24	.65	.91	.36	70%
		display
		Motion-based control	Classic control	6	.12	-.05	.30	1.36	.17	50%
	Control mode	Motion-based tangible controller	Classic controller	5	.06	-.12	.23	.63	.53	42%
		Body motion-based control	Motion-based controller tangible	5	.23	-.22	.68	1.00	.32	82%
	Auditory interface	Music presence	Music absence	2	.20	-.28	.68	.81	.42	49%
	Narrative Story presence	Story absence	2	.80	.24	1.36	2.82	.005	0%

Table 6 .

 6 Meta-analysis of game design factors on social presence

	Game design factor	Experimental condition	Control condition	Number of studies	Effect size	95% Lower limit confidence interval	95% Upper limit confidence interval	z-value	p-value	I²
	Co-player nature	Human co-player	Computer co-player	2	2.17	1.62	2.73	7.65	<.001	0%
	Co-playing mode	Cooperation/ collaboration mode	Competition mode	2	1.59	-.04	3.22	1.92	.06	93%

Table 7 .

 7 Meta-analysis of game design factors on self-presence

	Game design factor	Experimental condition	Control condition	Number of studies	Effect size	95% Lower limit confidence interval	95% Upper limit confidence interval	z-value	p-value	I²
	Narrative Story presence	Story absence	2	1.05	.47	1.62	3.56	<.001 0%

Note that while the first year of publication (e.g., online) of some of the included articles was

or earlier (hence their inclusion in the literature search databases and in the current review), their final year of publication may be

Acknowledgements

The author thanks Anastasia Passemar for her help in reviewing the articles in the search database.

The author also thanks Noé Monsaingeon and Jean-Christophe Sakdavong for their comments on earlier drafts of this paper.

Appendix: List of articles included in the systematic review according to the game design factor studied and the presence dimension assessed Note. * denotes an article for which one or more studies are included in the meta-analysis. An article may be cited more than once in the table based on the game design factor and the presence dimension assessed.


Avner Seror
email: avner.seror@univ-amu.fr

Daniel Chen

Garance Génicot

Jun Hyung Kim

Mathieu Lefebvre

Olivier L'haridon

The Priced Survey Methodology *

Keywords: JEL C9, D91, C44 Revealed Preference, Invisible Assets, Survey, Priced Survey

In this paper, I introduce a novel methodology to conduct surveys. The priced survey methodology. Like standard surveys, priced surveys are easy to implement, and measure invisible assets such as feelings, happiness, knowledge, views, and attitudes on numerical scales. Unlike standard surveys, priced surveys allow to leverage decades of research on revealed preference and consumer demand in the analysis of invisible assets.

Introduction

Many fundamental drivers of choice such as price or income are observable realities. Others, such as feelings, happiness, knowledge, views, attitudes, intentions or reasoning are not. In social sciences, we often rely on integer scales to measure these invisible assets but this creates several important issues. 1 First, as demonstrated by [START_REF] Bond | The Sad Truth about Happiness Scales[END_REF], cardinal comparisons of average levels measured through integer scales may not be robust to simple monotonic transformation of the scales. Second, it is not possible to disentangle the mechanisms explaining the expression of invisible assets, as measured through survey questions. For example, one might partially misreport her feelings, knowledge, attitudes or 1 beliefs for various reasons that have to do with the data collection process or the topics of the survey. Third, similar to observable assets, invisible assets are articulated in complex ways, something that cannot be easily captured through independent survey questions.

In this paper, I introduce a novel methodology to measure invisible assets -the priced survey methodology -and show that it gives one solution to all previous challenges. Like standard surveys, priced surveys are easy to implement and rely on numerical scales. Unlike standard surveys, priced surveys allow to leverage the vibrant economic research on revealed preference and consumer demand that developed since [START_REF] Samuelson | A Note on the Pure Theory of Consumer's Behaviour[END_REF][START_REF] Samuelson | Consumption Theory in Terms of Revealed Preference[END_REF] in the analysis of invisible assets.

The basic idea of a priced survey is to add a price structure to a survey. A price structure is characterized by a budget in tokens and associates a price to each question in a given survey. Subjects are given various opportunities to fill the same survey under different price structures. Hence, priced surveys mimic revealed preference experiments implemented in the economic literature, where subjects are offered repeated opportunities to allocate a budget between different consumption goods (e.g., [START_REF] Andreoni | Giving According to GARP: An Experimental Test of the Consistency of Preferences for Altruism[END_REF]). To the extent that subjects are rational, their preferences over invisible goods can be recovered [START_REF] Afriat | The Construction of Utility Functions from Expenditure Data[END_REF]). This creates many potential applications. In the rest of this paper, I formally introduce the priced survey methodology and discuss several broad applications.

The Methodology

Let I = {1, . . . I} denote a set of subjects, and S = {1, . . . S} a survey of S questions asking subjects to report a subset of their invisible assets on numerical scales. For example, question s might be "All things considered, are you happy with your life these days? Please answer on a scale from 0 to N (s)". A subject i's answer to the survey can be represented by a vector q i = {q i,s } s∈S ∈ s∈S {0, . . . N (s)}, where N (s) > 0 is the highest numerical level that can be reported on the scale associated to question s.

In the methodology, subjects first answer a survey without a price structure. A survey without a price structure is referred to as a standard survey in the rest of the paper. Subject i's answer to the standard survey is denoted q i,0 . Subjects are then offered repeated opportunities to fill the same survey under different price structures that are experimentally set. Let K = {1, . . . K} be the index set of observations. The price structure of observation k is denoted (R k , p k ), with R k ∈ N + a budget in tokens and p k = {p k,s } s∈S ∈ R S ++ a price vector. In observation k, subjects have R k tokens to allocate to the survey, and increasing the answer to question s by one numerical unit costs p k,s tokens. I denote q i,k,s ∈ {0, . . . N (s)} subject i's answer to statement s in observation k and q i,k = {q i,k,s } s∈S his vector of answers. Finally, D i = {q i,k , p i,k } k∈K gives the set of data observed for subject i. By assumption, in any observation k, subjects are constrained to saturate their budget constraint. 2This design mimics revealed preference experiments. Instead of spending resources by choosing quantities of consumption goods, subjects pay certain prices to express invisible assets, e.g., happiness level, trust in institutions or religiosity. In this context, preferences reflect a subjective organization of thoughts, feelings, views, opinions or attitudes. They correspond to the broad subjective rules governing the expression of invisible assets. In contrast, the prices of invisible assets reflect the aggregate influence of all the factors that weigh on the expression of invisible assets, as measured through survey questions. The following generalized definitions of revealed preferences enables to set axiomatic basis for subjects' rationality.

3 Definition 1 Let v ∈ [0, 1] K . For subject i ∈ I, an observed bundle q i,k ∈ s∈S {0, . . . , N (s)} is 1. v-directly revealed preferred to a bundle q, denoted q i,k R 0 v q, if v k p i,k q i,k ≥ p i,k q or q = q i,k .
2. v-strictly directly revealed preferred to a bundle q ∈ s∈S {0, . . . , N (s)}, denoted

q i,k P 0 v q, if v k p i,k q i,k > p i,k q.
3. v-revealed preferred to a bundle q, denoted q i,k R v q, if there exists a sequence of observed bundles (q j , q k , . . . , q m ) such that q i,k R 0 v q j , . . . q m R 0 v q.

4. v-strictly revealed preferred to a bundle q, denoted q i,k P v q, if there exists a sequence of observed bundles (q j , q k , . . . , q m ) such that q i,k R 0 v q j , . . . q m R 0 v q and at least one of them is strict.

If v k = 1 for any k ∈ K, the previous definition gives the standard direct revealed preference relations (e.g., [START_REF] Varian | The Nonparametric Approach to Demand Analysis[END_REF]); if v k = 0, the revealed preference relations are vacuous, as no observation can be revealed preferred to another. Parameter v k can be thought of as the minimum difference between the expenditure on bundle q i,k and the expenditure on bundle q before q can be considered worse than the observed choice. We can introduce the notion of consistency for data sets:

Definition 2 Let v ∈ [0, 1] K . A dataset D i
satisfies the general axiom of revealed preference given v (GARP v ) if for every pair of observed bundles, q i,k R v q implies not qP 0 v q i,k .

Other concepts of consistency could be introduced. 4 As the experimental methodology mimics revealed preference experiments on consumption goods, existing procedures can be used to recover preferences.

Applications

Recoverability of Preferences

The parametric methods to recover preferences are motivated by this result established by [START_REF] Halevy | Parametric Recoverability of Preferences[END_REF], which generalizes [START_REF] Afriat | The Construction of Utility Functions from Expenditure Data[END_REF] theorem:5

Theorem 1 The following conditions are equivalent:

1. There exists a nonsatiated utility function that v-rationalizes the data.

The data satisfy GARP v .

3. There exists a continuous, monotone, and concave utility function that v-rationalizes the data.

From Theorem 1, the data can be v-rationalized by a variety of utility functions. Denoting u i a utility function, it v-rationalizes subject i's choices when u i (q i,k ) ≥ u i (q) for any q i,k and any q ∈ {q i,l } l∈K such that v k p k .q k ≥ p k .q. In the case where v = 1, S = 2, and u i (.) is strictly concave, subject i's decision can be represented as in figure 1. Subject i's optimal choice lies in the indifference set with the highest utility level of any point on the budget constraint p k .q = R k .

= (q1, q2) ∈ R 2 + ; ui(q) = ui(qi,k)} qi,k
Estimating utility functions allows to assess how invisible assets are articulated in a subject's choices. As an example, consider CES utility functions of the form:

u i (q) = S s=1 a i,s q ρ i s 1/ρ i , (1)
with q = {q s } s∈S , ρ i ≤ 1, a i,s ∈ [0, 1] and S s=1 a i,s = 1. Together, the parameters a i = {a i,s } s∈S and ρ i describe how the different invisible assets in the survey S work together for subject i. They give a measure of subject i's social norms or worldviews. Parameter a i,s corresponds to the weight of asset s for subject i. Parameter ρ i captures the convexity of subject i's preferences through the elasticity of substitution, σ i = 1/(ρ i -1). In the limit case where σ i → ∞, the assets in S are perfect substitute for agent i. In the limit case where σ i → 0, the assets in S are complement for subject i and (1) converges to the Leontief utility function. In the case where σ i = 1, (1) converges to the Cobb-Douglass utility function. For example, if S is a set of questions that measure subjects' attitudes with respect to religion, politics and gender roles, then we might expect attitudes to show some degree of complementarity across these dimensions (i.e. σ i might be low). By contrast, if S measures opposite attitudes (e.g., liberal versus conservative attitudes), then one might expect these attitudes to be substitute.

Large surveys such as the World Value Surveys often include many disparate questions. Hence, before estimating utility functions such as (1), it is important to separate out invisible assets in different categories. In standard surveys, invisible assets can be separated using principal component analysis. The resulting categories gives useful indicator of social norms. For example, indicators of social liberal or conservative values can be built by grouping views about gender equality, immigration, same-sex marriage, divorce, death penalty, or abortion (e.g., [START_REF] Norris | Cultural Backlash and the Rise of Populism: Trump, Brexit, and Authoritarian Populism. Cultural Backlash: Trump, Brexit, and the Rise of Authoritarian Populism[END_REF]). However, these categories are built using correlation patterns, and say little about how subjects group invisible assets when making decisions. In priced surveys, the weak separability tests traditionally used in the analysis of consumer demand can be exploited to separate goods in different categories. These tests have been developed, for example, by [START_REF] Varian | The Nonparametric Approach to Demand Analysis[END_REF], [START_REF] Fleissig | A nonparametric test of weak separability and consumer preferences[END_REF], and [START_REF] Cherchye | Revealed preference tests for weak separability: An integer programming approach[END_REF] and can directly be applied to priced surveys. 6A group of assets is said to be weakly separable if the marginal rate of substitution between any two goods in the group is independent from the quantities consumed of any asset outside this group [START_REF] Leontief | Introduction to a Theory of the Internal Structure of Functional Relationships[END_REF], [START_REF] Sono | The Effect of Price Changes on the Demand and Supply of Separable Goods[END_REF]). Marginal rates of substitutions capture the utility tradeoffs that the subjects face. Some subjects might face a utility tradeoff between happiness and social status one the one hand, and face a separate utility tradeoff between religiosity and earnings. For these subjects, social status is achieved at the expense of happiness only, while religiosity is perceived as achieved at the expense of higher earnings. For other subjects, happiness, earnings and social status might not be separated. Happiness would be perceived as achieved at the expense of both earnings and social status.

Finally, the priced survey methodology allows to build non-parametric tests for utility models traditionally used to study invisible assets. In the context of social identity for example, [START_REF] Shayo | Social Identity and Economic Policy[END_REF] argues that there exists a fundamental utility tradeoff between gains from identifying to a social group and the distance from that social group. Accordingly, more identification from the working class to national identity is predicted to reduce the working class' support for redistribution [START_REF] Shayo | A Model of Social Identity with an Application to Political Economy: Nation, Class, and Redistribution[END_REF]). Using the priced survey methodology, this utility tradeoff can be estimated given that preferences over redistribution and national pride are recovered. 7 Other examples of utility models that could be tested using priced survey methodologies include general models of identity and their many applications stemming from [START_REF] Akerlof | Economics and Identity*[END_REF], models of parenting (e.g., [START_REF] Doepke | Parenting With Style: Altruism and Paternalism in Intergenerational Preference Transmission[END_REF], [START_REF] Attanasio | Subjective Parental Beliefs. Their Measurement and Role[END_REF], [START_REF] Seror | Child Development in Parent-Child Interactions[END_REF]), cultural transmission [START_REF] Bisin | The Economics of Cultural Transmission and the Dynamics of Preferences[END_REF]), political attitudes and various invisible assets such as ethical norms [START_REF] Fedderson | Moral Bias in Large Elections: Theory and Experimental Evidence[END_REF]), views about globalization [START_REF] Rodrik | Why Does Globalization Fuel Populism? Economics, Culture, and the Rise of Right-Wing Populism[END_REF]), or emotions [START_REF] Passarelli | Emotions and Political Unrest[END_REF]).

Recoverability of Imputed Prices

In social sciences, it is often challenging to assess experimenter-demand effect, self-censorship and other similar phenomena to interpret measures of invisible assets, e.g., political attitudes, identity, feelings, or happiness.8 In this section, I show that the priced survey methodology allows to elicit the constraint that weighs on subjects' answers at the time the data are collected. The basic idea is to estimate the price vector that makes any subject i's answers to the standard survey q i,0 consistent with his behavior in the data set D i . These prices give direct estimates of the constraint that weighs on subject i's answer to any question in the survey S at the time the data are collected, e.g., the price of expressing a feeling, a political opinion or a view about race.

The set of prices at which subject i's answers to a standard survey are consistent with his behavior in the data set D i is denoted S(q i,0 , v, D i , v 0 ) and is characterized as: Definition 3 Given any vector q i,0 and a dataset D i , we define the set of prices that v-support q i,0 by: S(q i,0 , v, D i , v 0 ) = {p 0 : (p k , q i,k ), k = 0, . . . K, satisfies (v 0 , v)-GARP and p 0 q i,0 = 1} The requirement p 0 q i,0 = 1 is a normalization. Subjects' rationality when they answer the standard survey is denoted v 0 ∈ [0, 1] and is not observed. If v 0 is set to 0, no observation can be revealed preferred to q i,0 , so any price vector p 0 such that p 0 q i,0 = 1 can support q i,0 . Experimenters may approximate v 0 , using measures of subjects' rationality in the priced surveys (see Section 3.4). The set S(q i,0 , v, D i , v 0 ) can be derived by solving a set of linear inequalities: Algorithm 1

• Input: D i , v 0 , and q i,0 .

• Output: S(q i,0 , v, D i , v 0 ).

1. Set S 0 (q i,0 , v, D i , v 0 ) = {p 0 : p 0 q i,0 = 1} and k = 1 2. If v k p k q i,k ≥ p k q i,0 , set S k (q i,0 , v, D i , v 0 ) = {p 0 ∈ S k-1 (q i,0 , v, D i , v 0 ) such that v 0 p 0 q i,0 ≤ p 0 q i,k } and go to 4. 3. If v k p k q i,k > p k q i,0 , set S k (q i,0 , v, D i , v 0 ) = {p 0 ∈ S k-1 (q i,0 , v, D i , v 0 ) such that v 0 p 0 q i,0 < p 0 q i,k } and go to 4. 4. If k < K, set k = k + 1 and go to 2. 5. If k = K, return S K (q i,0 , v, D i , v 0 ).
Algorithm 1 derives the set of price vectors p i,0 that make the answer q i,0 consistent with all the decisions made by subject i. In step 1 of the algorithm, the prices that support q i,0 are assumed included in the hyperplane defined by the budget constraint. This is the largest set that can possibly support q i,0 . After step 1, the algorithm proceeds by elimination. It goes through each data point (p k , q i,k ), k ∈ {1, . . . K}. As the algorithm goes through observation (p k , q i,k ), if q i,k is revealed directly preferred to q i,0 (i.e., v k p k q i,k ≥ p k q i,0 ), then q i,0 cannot be revealed directly strictly preferred to q i,k , so v 0 p 0 q i,0 ≤ p 0 q i,k . Hence, the set S k (q i,0 , v, D i , v 0 ) can be characterized as the subset of S k-1 (q i,0 , v, D i , v 0 ) that makes all the observations up to k and (p 0 , q i,0 ) satisfy GARP v 0 ,v . As the algorithm reaches step K, the data set (p 0 , q i,0 )∪D i satisfies GARP v 0 ,v for any vector p 0 ∈ S K (q i,0 , v, D i , v 0 ).

Three remarks are in order. First, the tightness of the set S(q i,0 , v, D i , v 0 ) depends on the rationality v. If subjects are not rational and v = 0, then any price vector can support q i,0 .9 Finally, if there are only few observations for which q i,k is revealed preferred to q i,0 , then not much can be learnt from the data on the prices that support q i,0 because the conditions 2 and 3 of Algorithm 1 are often not satisfied.

There are two ways to improve the precision of the non-parametric prediction of the prices that support q i,0 . First, experimenters can tailor the price structures to a known estimate of q i,0 , so that the inequality p k .q i,0 < v k p k q i,k is likely verified for as many observations as possible. 10 Second, similar to consumer demand analysis, knowledge of expansion paths can substantially increase the tightness of the set S(q i,0 , v, D i ) [START_REF] Blundell | Best Nonparametric Bounds on Demand Responses[END_REF]). 11 Finally, when the data D i are used to recover a parametric utility function u i (.), it is possible to get a parametric estimate of the prices that support q i,0 . Indeed, in such cases, q i,0 can be interpreted as the outcome of a utility-maximizing behavior:

q i,0 = arg max q∈ s∈S [0,N (s)] u i (q) subject to p i .q = 1, (2)
where u i (.) is the utility function estimated for subject i. The imputed price p i,s corresponds to the cost of marginally increasing the answer to question s for subject i. 12 The imputed budget is normalized to one without loss of generality.

Fact 1

• For any i ∈ I and any vector q i,0 , there exists a price vector p i,0 such that q i,0 = arg max q∈ s∈S [0,N (s)] u i (q) under the constraint p i,0 .q ≤ 1.

• For any s ∈ S, p i,0,s = 1 λ ∂u i (q i,0 ) ∂q i,0,s if 0 < q i,0,s < N (s), and p i,0,s ≥ 1 λ ∂u i (q i,0 ) ∂q i,0,s (resp.

p i,0,s ≤ 1 λ ∂u i (q i,0 )
∂q i,0,s ) if q i,0,s = 0 (resp. q i,0,s = N (s)), given λ ≥ 0 the Lagrange multiplier associated with the optimization problem (9).

To the extent that q i,0 is not on a corner (i.e., 0 < q i,0,s < N (s) for any s ∈ S), and given subject i's continuous, monotone, and concave utility function u i (.), there is a unique vector of imputed prices that i faces when answering the survey. 13 In the standard survey, if subject i's answers to a pair (s, z) of questions is not on a corner, then the ratio p i,0,s /p i,0,z is

The experimental design could tailor the price structures to the observation of q i,0 for each individual in order to make the set S(q i,0 , v, D i , v 0 ) as precise as possible. Improving the precision of the prediction of the imputed price might require assuming that the data D i satisfies a generalized version of the Strong Axiom of Revealed Preference (SARP v ) rather than GARP v [START_REF] Blundell | Sharp for SARP: Nonparametric Bounds on Counterfactual Demands[END_REF]). The support of u i (.) is assumed to be the continuous set s∈S [0, N (s)] so that imputed prices could even be recovered for non-integer values of q i,0 . Equivalently, we could normalize one price instead of the budget and Fact 1 will hold. I chose to normalize the budget because imputed prices are more easy to interpret than imputed budgets.

q 0 i,0 q 1 i,0 q2 q1 q 0 i,0 q 1 i,0 q2 q1 qi q 0 i,0 q 1 i,0
simply equal to the marginal rate of substitution between these two assets ∂u i (q i,0 ) ∂q i,0,s / ∂u i (q i,0 ) ∂q i,0,z . In the standard survey, if subject i's answer to question s is zero (resp. N (s)), then the possible values of p i,0,s are bounded from below (resp. from above), p i,0,s ≥ 1 λ ∂u i (q i,0 ) ∂q i,0,s (resp. p i,0,s ≤ 1 λ ∂u i (q i,0 ) ∂q i,0,s ).

Interpreting Treatment Effects

Many studies seek to document how interventions or experimental treatments impact happiness, attitudes, feelings or perceptions, among other invisible assets. However, measuring treatment effects through standard surveys conflates two fundamental mechanisms. One is related to changes in subjects' preferences, and one is related to changes in the prices that subjects face when answering the survey. In this section, I show that using the priced survey methodology, it is possible to additively decompose a treatment effect into two components. One related to changes in prices (e.g., the treatment increases the price of expressing xenophobic attitude), and one related to changes in preferences (e.g., the treatment cultivates a more tolerant worldview). The external validity of experimental results might crucially depend on whether treatments primarily change prices or preferences.

In this section, I assume that subjects are evenly and randomly assigned to one of two groups. Let T i be a dummy variable that is equal to 1 if subject i is randomly assigned to the treatment group and 0 otherwise. I define

q i,0 =    q 0 i,0 if T i = 0 q 1 i,0 if T i = 1 and p i,0 =    p 0 i,0 if T i = 0 p 1 i,0 if T i = 1 (3)
The basic argument behind the decomposition is represented in Figure 2. As represented in the left panel, vector q 0 i,0 (resp. q 1 i,0 ) measures subject i's answer to the standard survey when i is not treated (resp. is treated). Using data from the standard survey, researchers can estimate the treatment effect, defined as the average difference between q 1 i,0 and q 0 i,0 across subjects. Using priced surveys, preferences and imputed prices can be recovered for any subject. As represented in the middle panel, both q 1 i,0 and q 0 i,0 can be interpreted as resulting from the maximization of a utility function subject to a budget constraint. Moreover, using the recovered utility functions and imputed prices, researchers can estimate the counterfactual qi , defined as the answer that subject i would have given if her preferences were affected by the treatment but not the price constraint weighing on her answer. As represented in the right panel, using this counterfactual, the treatment effect can be decomposed into a treatment effect due to changes in preferences (the difference between qi and q 0 i,0 ), and one due to changes in prices (the difference between q 1 i,0 and qi ). The vector qi cannot be observed empirically, as we cannot simultaneously observe subject i's budget constraint when she is not treated and her preferences when she is. However, it is possible to use the data to estimate q, a counterfactual for any individual i. q can be estimated either through a parametric approach or through a non-parametric approach.

In the parametric approach, the counterfactual q can be found by solving the following problem: q = arg max

q∈ s∈S [0,N (s)]
u 1 (q) under the constraint q.p 0 = 1, (

with u 1 a representative utility function for the treatment group, and p 0 the average imputed price vector in the control group. 14 Deriving a unique counterfactual q requires restricting the sample to interior answers, so that Fact 1 characterizes a unique imputed price vector for each individual i ∈ I. In the Appendix, I give an algorithm to compute a non-parametric counterfactual set Q, defined as the set of answers to the standard survey that are (i) consistent with the aggregate data in the treatment group and (ii) generated by an (average) imputed price compatible with answers in the control group. q i,0 is decomposed into two components, x i (q) and w i (q):

x i (q) =    q 0 i,0 if T i = 0 q if T i = 1 and w i (q) =    q if T i = 0 q 1 i,0 if T i = 1. (5)
In this decomposition, x i (q) measures how the treatment affects preferences, and w i (q) measures how the treatment affects prices. q i,0 can be expressed as:

q i,0 = α + βT i + η i (6)
in the case of constant treatment effects, where η i = {η i,s } s∈S is a vector of random variables that I assume identical and independently distributed.

Fact 2 The treatment effect β can be additively decomposed into two vectors, β = β 1 (q) + β 2 (q) with β 1 (q) the average treatment effect associated with the estimation of

w i (q) = α 1 + β 1 T i + ϵ i , (7)
and β 2 (q) the average treatment effect associated with the estimation of

x i (q) = α 2 + β 2 T i + µ i , (8)
with µ i and ϵ i two vectors of i.i.d random variables.

The proof is detailed in the Appendix. The vector β 1 (q) corresponds to the average treatment effect on subjects' preferences keeping their budget constraint equal to its average in the control group. The vector β 2 (q) corresponds to the average treatment effect on subjects' budget constraint, keeping their preferences equal to their "average" in the treatment group. In a non-parametric approach, given that the counterfactual set Q is not a singleton, one can deduce a set of decompositions, applying Fact 2 for all the vectors q that belong to Q. 15To summarize, this section shows that using data from priced surveys and the corresponding standard surveys allows to decompose any treatment effect into two components. One measures the treatment effect due to changes in preferences, and one measures the treatment effect due to changes in prices. The procedure is based on the estimation of a counterfactual. I developed a parametric approach above, which provides a unique decomposition. A non-parametric approach can be a useful complement to the parametric approach, and is developed in the Appendix.

Besides the decomposition, priced surveys allow to assess how the structure of preferences affect the magnitude of a treatment effect. As a simple example, we might expect that interventions seeking to affect one attitude, e.g., anti-vaccine attitude, to be less effective if that attitude complements more entrenched attitudes not targeted by the intervention, e.g., low trust in science, government and institutions.

Finally, using the priced survey methodology, experimenters can assess the welfare implications of experimental treatments (e.g., cultivating altruistic preferences through a given treatment might decrease subjects' welfare). Typically, experimenters could estimate and compare the indirect utility before and after a treatment. One common cardinalization of the indirect utility in consumer behavior is Samuelson's money metric function, which allows to express changes in well-being in monetary units. 16 Depending on whether imputed prices before or after the treatment are considered, this approach leads to the compensating variation or the equivalent variation [START_REF] Hicks | A Reconsideration of the Theory of Value. Part I[END_REF]).

Rationality

Is there a rationality behind the expression of feelings, happiness, knowledge, and other invisible assets? Using priced surveys, indices of rationality traditionally used in consumer demand can be computed for invisible assets. [START_REF] Varian | Goodness-of-fit in optimizing models[END_REF] suggested the following measure of rationality in his analysis of consumer demand:

v i = min v={v k } k∈K ∈[0,1] K ,D i satisfies GARPv K k=1 (v k -1) 2 , (9)
which measures the extent of utility-maximizing behavior implied by the data set D i . [START_REF] Afriat | Efficiency Estimation of Production Function[END_REF] critical cost efficiency index is the lower bound of the vector {v i,k } k∈K that solves (9). 17 If v i is close to 1, then agent i behaves almost rationally in the priced surveys. Parameter v i,k can be interpreted as the fraction of tokens that a subject is "wasting" by making choices that are inconsistent with observation k.

Other measures of rationality can be built, based on violations of GARP (or similar measures of rationality). The money pump index developed by [START_REF] Echenique | The Money Pump as a Measure of Revealed Preference Violations[END_REF] measures the severity of violations of GARP, assessing the money that arbitragers can "pump" from irrational subjects. [START_REF] Houtman | Determining all Maximal Data Subsets Consistent with Revealed Preference[END_REF] inconsistency index computes the minimal subset of observations that should be removed from the data set D i in order to eliminate cycles in the revealed preference relation.

Assessing subjects' rationality can be useful for several reasons. First, parametric methods to recover subjects' preferences over invisible assets can be based on the mazimization of subjects' predicted rationality [START_REF] Halevy | Parametric Recoverability of Preferences[END_REF]). Second, rationality measures allow to perform robustness checks by restricting the sample to the most rational subjects. This can be especially important in experimental studies where treatments affect subjects' rationality, as it becomes unclear whether treatment effects are driven by changes in prices, preferences, or rationality. 18 Third, it is possible to evaluate the correlates of rationality in the analysis of invisible goods. 19 Finally, combined with weak separability tests, researchers can evaluate whether there are subcategories of invisible assets for which subjects are more rational than for other subcategories. 20

Discussion

In this paper, I introduced a novel methodology to measure invisible assets -the priced survey methodology. It consists in giving subjects various opportunities to fill the same survey under different price structures. Subjects reveal their preferences over invisible assets As observed by Halevy, Persitz and Zrill (2018), [START_REF] Afriat | Efficiency Estimation of Production Function[END_REF] critical cost efficiency index can be characterized as

v i = min v={v k } k∈K ∈I,Di satisfies GARPv K k=1 (v k -1) 2 , with I = {v ∈ [0, 1] K : v = v1∀v ∈ [0, 1]}.
In that respect, it seems important in experimental studies to design interventions in such a way that subjects' rationality remains constant across treatment groups. See [START_REF] Echenique | The Money Pump as a Measure of Revealed Preference Violations[END_REF] and [START_REF] Choi | Who Is (More) Rational?[END_REF] for studies on the demographic variables that correlate with rationality in consumer demand. Is there a meta-rationality behind the determination of the categories where subjects' behavior is irrational? This question, close to the issue of self-motivated thinking [START_REF] Bénabou | Self-Confidence and Personal Motivation*[END_REF]), could potentially be tested too, using priced surveys.

in priced surveys, so the vibrant economic research on revealed preference and consumer demand that developed since [START_REF] Samuelson | A Note on the Pure Theory of Consumer's Behaviour[END_REF][START_REF] Samuelson | Consumption Theory in Terms of Revealed Preference[END_REF] can be leveraged to study choices over invisible assets.21 I suggest three broad applications of priced surveys. First, I give several guidelines for the recoverability of preferences over invisible assets. Second, I show that priced surveys can be used to elicit the constraint that weighs on subjects' answers at the time the data are collected. This can be useful for experimental studies seeking to disentangle whether treatment effects on invisible assets are explained by changes in preferences (e.g., the treatment cultivates a more tolerant worldview) or changes in prices (e.g., the treatment increases the price of expressing xenophobic attitudes). Finally, I detail how subjects' rationality can be assessed, using standard measures established in the revealed preference literature [START_REF] Afriat | Efficiency Estimation of Production Function[END_REF], [START_REF] Varian | Goodness-of-fit in optimizing models[END_REF], [START_REF] Houtman | Determining all Maximal Data Subsets Consistent with Revealed Preference[END_REF]).

Other potentially useful applications are not discussed in this paper. First, the dynamics of revealed preferences over invisible assets could be assessed, following works in demand analysis by [START_REF] Crawford | Habits Revealed[END_REF], or [START_REF] Demuynck | I'll Never Forget My First Cigarette: A Revealed Preference Analysis of The "Habits As Durables" Model[END_REF]. Such dynamic analysis could enrich our understanding of the evolution of happiness, depressive feelings, or extremist values among many others. The intergenerational transmission of invisible assets could also be assessed, implementing priced surveys in the appropriate samples. Second, there are several important studies on revealed preferences for multi-person demand behavior (e.g., [START_REF] Chiappori | Rational Household Labor Supply[END_REF], [START_REF] Brown | Testable Restrictions on the Equilibrium Manifold[END_REF], [START_REF] Cherchye | The Collective Model of Household Consumption: A Nonparametric Characterization[END_REF][START_REF] Crawford | Habits Revealed[END_REF], 2011)). Accounting for multi-person demand for invisible assets could be a starting point to an empirical analysis of social interactions and the demand for invisible assets in a general equilibrium framework.

Figure

  Figure 1: Utility maximization

Figure

  Figure 2: Decomposition

This assumption is made so that subjects make tradeoffs when they answer the survey in any given observation.

These definitions are taken from[START_REF] Halevy | Parametric Recoverability of Preferences[END_REF].

The Strong Axiom of Revealed Preference (SARP) might turn out important if researchers rely on expansion paths to increase the precision of their predictions[START_REF] Blundell | Sharp for SARP: Nonparametric Bounds on Counterfactual Demands[END_REF]).

 5 Diewert (1973),[START_REF] Varian | The Nonparametric Approach to Demand Analysis[END_REF], and[START_REF] Polisson | Afriat's Theorem and Samuelson's 'Eternal Darkness[END_REF] provide various statements and proofs of Afriat's Theorem.

Other useful tests can also be applied to priced surveys, including homotheticity, additive separability, and homothetic separability. These tests can be found in[START_REF] Varian | Non-parametric Tests of Consumer Behaviour[END_REF].

While there are potentially many ways to do so, one simple alternative consists in asking the two following questions. Question 1 asks "How much would you agree with the following statement on a scale from 0 to 10: we need larger income differences as incentives for individual effort". This question captures views about redistribution. Question 2 asks "How much do you agree with the following statements [1: Agree strongly; 2. Agree; 3 Neither agree nor disagree; 4. Disagree, 5. Disagree strongly.] The world would be a better place if people from other countries were more like the people in [Subject's country]". This question gauges feelings of national pride. Both questions are taken from the World Value Survey, and enable researchers to recover preferences over redistribution and national pride.

Methods have been developed in the experimental literature to assess experimenter-demand effects (e.g.,[START_REF] De Quidt | Measuring and Bounding Experimenter Demand[END_REF]). When self-censorship biases survey answers, researchers use different data to identify invisible assets. For example, Atkin, Colson-Sihra and Shayo (2021) use food consumption to identify the salience of religious and ethnic identity.[START_REF] Seror | Legalized Same-Sex Marriage and Coming Out in America: Evidence from Catholic Seminaries[END_REF] use data on enrollment in priestly studies in Catholic seminaries to identify the effect of same-sex marriage laws on the expression of homosexual identity.

Formally, for any pair of vectors v, u ∈ [0, 1] K+1 , if v ≥ u (meaning that for any k ∈ {0, . . . , K} v k ≥ u k ), then S(q i,0 , u, D i , v 0 ) ⊆ S(q i,0 , v, D i , v 0 ).

The representative utility function for the treatment group can be estimated by relying on the average value of the utility parameters in that group.

It is not necessarily true that Q is convex, so the set of decompositions might not be convex either.

On the welfare analysis of consumer behavior relying on revealed preference, see[START_REF] Samuelson | A Note on the Pure Theory of Consumer's Behaviour[END_REF][START_REF] Samuelson | Consumption Theory in Terms of Revealed Preference[END_REF],[START_REF] Afriat | The Construction of Utility Functions from Expenditure Data[END_REF],[START_REF] Diewert | Afriat and Revealed Preference Theory[END_REF],[START_REF] Varian | The Nonparametric Approach to Demand Analysis[END_REF],Blundell, Browning and[START_REF] Blundell | Best Nonparametric Bounds on Demand Responses[END_REF][START_REF] Blundell | Sharp for SARP: Nonparametric Bounds on Counterfactual Demands[END_REF].

For a recent review of this literature, see[START_REF] Crawford | Empirical Revealed Preference[END_REF].

Thierry Verdier for their insights and encouragements. Mistakes are my own. I acknowledge funding from the french government under the "France 2030" investment plan managed by the French National Research Agency (reference :ANR-17-EURE-0020) and from Excellence Initiative of Aix-Marseille University -A*MIDEX.

Appendix

A Proof of Fact 2

Proof. Given the random assignment of the subjects across the treatment and control groups, β = E(q i,0 | T i = 1) -E(q i,0 | T i = 0), (A.1) which can be rewritten

By definition of x i and w i ,

(A.4)

■

B Non-Parametric Decomposition

The non-parametric decomposition is based on the estimation of the counterfactual set Q.

To build this counterfactual set, I first aggregate the data in a given group.

Definition 4

The aggregate data of treatment group T ∈ {0, 1} is:

I q i,k the average answer in treatment group T and observation k.

20

From a standard result in microeconomic theory, an aggregate data set does not necessarily satisfy GARP 1 when the individual-level data sets do. Sufficient conditions can however be derived so that the aggregate data satisfy GARP 1 . I abstract from these issues in this proof, and denote v T the vector such that

Definition 5 For any pair of data sets (D 0 , D 1 ), vector q 0 0 and rationality level v 0 , the counterfactual set Q is defined as:

Q is the non-parametric equivalent of q, as characterized in (4). It gives all the possible answers that are consistent with the average behavior in the treatment group but generated under an average price vector compatible with subjects' behavior in the control group. Hence, it is as if prices -and rationality level (v 0 ) -were on average kept equal to what they are supposed to be in the control group, but choices made consistent with the "agregate" preferences in the treatment group. The algorithm below gives a simple procedure to compute Q for any pair of aggregate data sets (D 1 , D 1 ), average answer to the standard survey in the control group q 0 0 , and rationality v 0 associated with the average answer q 0 0 .

Algorithm 2

• Input: (D 1 , D 1 ), v 0 , and q 0 0 .

• Output: Q.

={(q(z)} z∈{1,...,Z} (A.6)

) such that v 0 p 0 q(z) ≤ p 0 q 1 k } and go to 4.

3. If v k p k q 1 k > p k q(z), set P k (q(z)) = {p 0 ∈ P k-1 (q(z)) such that v 0 p 0 q(z) < p 0 q 1 k } and go to 4.

4. If P k (q(z)) = ϕ set Qz = Qz-1 \ {q(z)} and go to 5. Otherwise:

and go to 5.

• If k = K, go to 5.

5. If z < Z, set z = z + 1 and go to 2. Otherwise, return QZ .

As for Algorithm 1, Algorithm 2 proceeds by elimination. It starts with the set Q0 that contains all the vectors of answers that could be supported by a price that generates an average answer in the control group. The algorithm then eliminates from this set all the elements that will violate GARP (v 0 ,v 1 ) in the aggregate data set of the treatment group. The algorithm proceeds by elimination, exactly like Algorithm 1. Since Q0 is a hyperplane of s∈S {1, . . . , N (s)}, it can take substantial time to run Algorithm 2 for large surveys or long scales.