• enrichments from MSigDB check out as well

• On top of that it discovers small signatures not found by any of the methods above, and their enrichments are highly relevant to glioblastoma

• Our signatures are on average enriched for smaller pathways/genesets than the competitors' signatures

• embarrassingly parallel, ready-to-deploy on an HPC cluster • Performance-oriented code, as few overheads as possible

• Aggressive optimization -03 flag during compilation

• embarrassingly parallel, ready-to-deploy on an HPC cluster • Performance-oriented code, as few overheads as possible

• Aggressive optimization -03 flag during compilation

• Partial update of the Friedman Statistic (not a complete recalculation)

• embarrassingly parallel, ready-to-deploy on an HPC cluster • Performance-oriented code, as few overheads as possible

••••

  ...the glioblastoma paper by Neftel et al. (2019) where they searched for signatures using global hierarchical clustering of cells and genes in each sample separately Were able to show benefit of certain gene inhibitors in glioblastoma xenografts on mice 9 | E. Zakiev/ J. Dreo | Statistical Discovery of Transcriptomic Cancer Signatures using Multimodal Local Search | 29/11/2022 ...the glioblastoma paper by Neftel et al. (2019) where they searched for signatures using global hierarchical clustering of cells and genes in each sample separately Were able to show benefit of certain gene inhibitors in glioblastoma xenografts on mice • Hierarchical clustering of cells to form partitions of each sample into up-regulated and "the rest of the sample" parts ...the glioblastoma paper by Neftel et al. (2019) where they searched for signatures using global hierarchical clustering of cells and genes in each sample separately Were able to show benefit of certain gene inhibitors in glioblastoma xenografts on mice • Hierarchical clustering of cells to form partitions of each sample into up-regulated and "the rest of the sample" parts • Avoiding batch effects: integrated common genesets instead ...the glioblastoma paper by Neftel et al. (2019) where they searched for signatures using global hierarchical clustering of cells and genes in each sample separately Problems: Global clustering (dominated by global similarities) ...the glioblastoma paper by Neftel et al. (2019) where they searched for signatures using global hierarchical clustering of cells and genes in each sample separately Problems: Global clustering (dominated by global similarities) • Search in each sample separately, missing out on a synchronized unified search ...the glioblastoma paper by Neftel et al. (2019) where they searched for signatures using global hierarchical clustering of cells and genes in each sample separately Problems: Global clustering (dominated by global similarities) • Search in each sample separately, missing out on a synchronized unified search • Dichotomous clustering: cell can belong to only 1 cluster (like many methods) ...the glioblastoma paper by Neftel et al. (2019) where they searched for signatures using global hierarchical clustering of cells and genes in each sample separately Problems: Global clustering (dominated by global similarities) • Search in each sample separately, missing out on a synchronized unified search • Dichotomous clustering: cell can belong to only 1 cluster (like many methods) • Extremely ad-hoc, overabundance of dubious thresholds Our method • Is not a mere clustering of the cells • Looks for signatures in all samples at the same time • Each cell can support multiple programs 35 | E. Zakiev/ J. Dreo | Statistical Discovery of Transcriptomic Cancer Signatures using Multimodal Local Search | 29/11/2022 Optimal signature K • Our method discovers similar signatures as Neftel et al, Seurat, HARMONY, LIGER and cNMF Results Glioblastoma Smartseq2 data from Nefel et al. 27 samples, 8k cells x 23 k genes. Known ground truth, known signatures • Our method discovers similar signatures as Neftel et al, Seurat, HARMONY, LIGER and cNMF • enrichments from MSigDB check out as well 39 | E. Zakiev/ J. Dreo | Statistical Discovery of Transcriptomic Cancer Signatures using Multimodal Local Search | 29/11/2022 Results Glioblastoma Smartseq2 data from Nefel et al. 27 samples, 8k cells x 23 k genes. Known ground truth, known signatures • Our method discovers similar signatures as Neftel et al, Seurat, HARMONY, LIGER and cNMF • enrichments from MSigDB check out as well • On top of that it discovers small signatures not found by any of the methods above, and their enrichments are highly relevant to glioblastoma 40 | E. Zakiev/ J. Dreo | Statistical Discovery of Transcriptomic Cancer Signatures using Multimodal Local Search | 29/11/2022 Results Glioblastoma Smartseq2 data from Nefel et al. 27 samples, 8k cells x 23 k genes. Known ground truth, known signatures • Our method discovers similar signatures as Neftel et al, Seurat, HARMONY, LIGER and cNMF

•

  Aggressive optimization -03 flag during compilation • Partial update of the Friedman Statistic (not a complete recalculation) • I borrowed R's "under the hood" Cpp implementation of log p of χ 2 distribution • very fast -uses Chebyshev rational approximations for Erfc(x ) calc • embarrassingly parallel, ready-to-deploy on an HPC cluster • Performance-oriented code, as few overheads as possible • Aggressive optimization -03 flag during compilation • Partial update of the Friedman Statistic (not a complete recalculation) • I borrowed R's "under the hood" Cpp implementation of log p of χ 2 distribution • very fast -uses Chebyshev rational approximations for Erfc(x ) calc Resulting algo scales as O(n) where n is the number of cells • embarrassingly parallel, ready-to-deploy on an HPC cluster • Performance-oriented code, as few overheads as possible • Aggressive optimization -03 flag during compilation • Partial update of the Friedman Statistic (not a complete recalculation) • I borrowed R's "under the hood" Cpp implementation of log p of χ 2 distribution • very fast -uses Chebyshev rational approximations for Erfc(x ) calc Resulting algo scales as O(n) where n is the number of cells One full iteration on Pasteur's Maestro Cluster core: 49 microseconds for 27 samples of 8000 cells 169 microseconds for 63 samples of 25k cells

samples, 8k cells x 23 k genes. Known ground truth, known signatures

Our method

• Is not a mere clustering of the cells • Looks for signatures in all samples at the same time After obtaining all the signatures, test them for association with resistance.

Signature's abundance goes up in "after" treatment vs "before" treatment? Potential chemoresistance signature After obtaining all the signatures, test them for association with resistance.

Signature's abundance goes up in "after" treatment vs "before" treatment? Potential chemoresistance signature

where n is the number of samples

Addendum 3: Friedman Statistic Quick Update

Friedman statistic with n genes: Denote

so that When adding a gene: n → n + 1

(A n + 24 m j=1 R j,n r j + 12 m j=1 r 2 j ) -(B n + 3m(m + 1) 2 (2n + 1)) (C n + m(m + 1)) -(D n + [1/(m -1)]{( g n+1 j=1 t 3 n+1,j ) -m}) (9)


Loïc Caroux
email: loic.caroux@univ.tlse2.fr

Morgane Pujol
email: morgane.pujol@univ-tlse2.fr

Player enjoyment in video games: A systematic review and metaanalysis of the effects of game design choices

Keywords: game user research, player experience, standardized scale, music, difficulty, control mode

A systematic review and metaanalysis of the effects of game design choices

Enjoyment is a widely assessed dimension of the video game player's experience.

A systematic literature review and a meta-analysis were conducted to provide a more comprehensive view of the elements of game design that affect the feeling of enjoyment, as well as its assessment techniques. The review showed that the studies covered major aspects of game design, such as in-game contents, input/output information, and, to a lesser extent, multiplayer aspects. The metaanalysis showed that only the presence of music in the game had a significant effect on enjoyment. Other factors, including more studied factors such as game difficulty and control mode, did not show significant effects. Implications for future research are discussed, such as further investigation of the effects of both player and game characteristics diversity on enjoyment in isolation and in interaction with game design factors, systematic use of standardized scales to measure enjoyment, and use of multidimensional scales or physiological measures.

Introduction

Player enjoyment in video games is related to positive player reactions during a gameplay session [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF][START_REF] Crutzen | Enjoyment: A conceptual exploration and overview of experimental evidence in the context of games for health[END_REF][START_REF] Fang | Development of an instrument to measure enjoyment of computer game play[END_REF][START_REF] Segundo Díaz | Building blocks for creating enjoyable games-A systematic literature review[END_REF]. It corresponds to the extent to which players positively evaluate their experience, cognitively and affectively [START_REF] Schaffer | Development and preliminary validation of the enjoyment questionnaire and the sources of enjoyment questionnaire[END_REF], and is partly associated with need satisfaction and guiltlessness [START_REF] Matthews | A framework for the assessment of enjoyment in video games[END_REF][START_REF] Mekler | A systematic review of quantitative studies on the enjoyment of digital entertainment games[END_REF].

Enjoyment is one of the main dimensions that can be assessed when studying the experience of players interacting with video games. Player experience (PX) is the individual, personal experience the player has during and immediately after playing the game [START_REF] Wiemeyer | Player experience[END_REF]. PX can be assessed along several other dimensions such as flow, emotions, engagement, immersion and presence, for the most studied in the literature [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF]. Note that enjoyment is a specific dimension of PX that can be easily confused with other concepts related to a positive psychological state of the video game player, such as flow or happiness. However, these concepts have been clearly differentiated in the literature. Basically, their difference lies mainly in the temporality and intensity of the feelings. Flow is a feeling that occurs when a balance between the player's skills and the challenge of the game is achieved when playing (e.g., [START_REF] Chen | Flow in games (and everything else)[END_REF][START_REF] Csikszentmihalyi | Flow: The psychology of optimal experience[END_REF]. It is a more intense feeling ("being one with the activity", "losing track of time", etc.) but less frequent than enjoyment, which can be felt even when a game is less challenging. See for example [START_REF] Abuhamdeh | On the relationship between flow and enjoyment[END_REF], [START_REF] Abuhamdeh | The importance of challenge for the enjoyment of intrinsically motivated, goal-directed activities[END_REF], [START_REF] Crutzen | Enjoyment: A conceptual exploration and overview of experimental evidence in the context of games for health[END_REF], or [START_REF] Matthews | A framework for the assessment of enjoyment in video games[END_REF] for further discussion of the difference between enjoyment and flow. Happiness is a more enduring feeling concept. While it may depend on the immediate sensation of joy and pleasure associated with participating in certain activities, it primarily concerns the evaluation of feelings and involves an appraisal of whether, on balance, one is relatively happy or not. Happiness referred as quality of life or well-being [START_REF] Boyle | A review of theories of player enjoyment in playing computer games[END_REF][START_REF] Nettle | Happiness: The science behind your smile[END_REF].

Enjoyment is a widely studied concept in the literature on player-computer interaction. Several literature reviews have specifically addressed this topic, in systematic reviews [START_REF] Mekler | A systematic review of quantitative studies on the enjoyment of digital entertainment games[END_REF][START_REF] Schaffer | Digital game enjoyment: A literature review[END_REF][START_REF] Segundo Díaz | Building blocks for creating enjoyable games-A systematic literature review[END_REF] or narrative reviews [START_REF] Boyle | A review of theories of player enjoyment in playing computer games[END_REF]. Some other broader literature reviews on PX have also addressed player enjoyment (e.g., [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF]. Each of these reviews has made a specific contribution to understanding the place of enjoyment in PX. [START_REF] Boyle | A review of theories of player enjoyment in playing computer games[END_REF] reviewed the existing theories that are proposed to explain enjoyment felt by players. [START_REF] Mekler | A systematic review of quantitative studies on the enjoyment of digital entertainment games[END_REF] reviewed the literature to analyze the operationalization, measures and determinants of enjoyment as well as its relation with other dimensions of PX, based on quantitative studies. More recently, [START_REF] Schaffer | Digital game enjoyment: A literature review[END_REF] reviewed the theories and measures of enjoyment, as well as design elements that make games enjoyable. Segundo Diaz et al. (2022) reviewed the literature to inventory game design elements that produce an enjoyable experience in the case of games with a purpose. Finally, [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF] showed that enjoyment, as well as other dimensions of PX, can be influenced by various hardware and software aspects of video games.

All of these previous literature reviews have shown that enjoyment is one of the most important dimensions of PX, and one of the most studied concepts in game user research. However, the contribution of these past studies is of course of great interest to the game user research community, including both researchers and designers, but is limited by their nature. The usefulness and actionability of this previous literature are limited when it comes to whether and how player enjoyment is influenced specifically by game design elements (i.e., elements of games that can be manipulated by game designers and developers, whether they are software or hardware elements), and whether and how game designers should design specific elements of their game in order to enhance enjoyment of their players.

First, with respect to the researchers, although some of these previous literature reviews followed a rigorous systematic review methodology, the results were primarily presented in a descriptive manner. They presented a qualitative analysis of previous studies, which could be qualitative or quantitative in nature. They did not, for example, and in some cases could not, present precise and robust quantitative evidence of the specific impact of a given game design factor and the magnitude of its impact on player enjoyment. Advancing PX theory requires that we are able provide explicit, precise, and robust evidence of influence of game design on PX dimensions such as enjoyment, capitalizing on existing results from multiple experimental studies.

Second, with respect to the designers, such descriptive and qualitative literature reviews might be difficult to use. When designing a system, let alone a video game, it would be more useful to exploit precise and quantitative effects to optimize the experience of players. The results obtained by quantitatively combining the results of multiple experimental studies could help to better understand the specific effect of different design choices on enjoyment and help game designers to choose an element that would have a desired specific impact on player enjoyment. Third, [START_REF] Caroux | Presence in video games: A systematic review and meta-analysis of the effects of game design choices[END_REF] argued that the size of the body of literature on experimental game user research is now large enough to allow for more comprehensive and in-depth analyses of data from this research, such as meta-analyses. For example, in their study, a meta-analysis was conducted on data from controlled experiments (i.e. randomized controlled trials or within-participant experiments) that quantitatively assessed the efficacy of game design factors on another important dimension of PX: sense of presence. The largest effects revealed by this meta-analysis were that playing games with a head-mounted display and motion controller rather than a monitor display and non-motion controller had a large effect on global presence. Similarly, playing with human co-players rather than computer-controlled co-players and playing cooperatively rather than competitively have a very large and large effect on social presence, respectively.

The purpose of the present study is to focus in depth on the concept of player enjoyment in video games. The main objective is to provide a more comprehensive and quantitative view of the elements of game design that have an effect on the feeling of enjoyment in video games. In order to achieve this goal, we followed the two-step methods proposed by [START_REF] Caroux | Presence in video games: A systematic review and meta-analysis of the effects of game design choices[END_REF] in their study on presence in video games. The first step was a systematic review of literature that described the different categories of game design elements that have been studied regarding player enjoyment, and explored the different techniques to assess it. The second step was a meta-analysis that was conducted to quantitatively assess the impact of the specific game design elements that were evaluated in these studies. The goal is to quantify the effect of these game design choices on the feeling of enjoyment based on a combination of results from different experimental studies. The main contribution of the present study is to present the quantitative results of this meta-analysis to help understand more accurately the specific effect of several game design choices on enjoyment, which could help researchers advance PX theory, and designers optimize PX when designing video games.

Methods

The methods used in the present research followed the same review and meta-analysis process as [START_REF] Caroux | Presence in video games: A systematic review and meta-analysis of the effects of game design choices[END_REF] in terms of literature search, data extraction, and statistical analysis. The following paragraphs outline the main steps of this method. See Caroux (2023) for more details and justification of the choices made in the methods process.

Literature search

The search for relevant literature was carried out in the Web of Science Core Collection, PsycInfo, and Medline online databases and included research articles written in English and published through to 2021. In order to obtain the maximum amount of data relevant to the objective of the present study, there were no limitations on the length of articles (e.g., minimum length) for inclusion. Full-length or shorter papers, such as extended abstracts, were included as long as they contained all the information necessary for review (i.e. detailed methods and results in particular).

To capture as many articles as possible that studied enjoyment, even as a secondary concept, the literature search query was conducted with broad keywords that included multiple PX concepts, beyond enjoyment. This approach also included other significant dimensions of PX to ensure maximum coverage of related articles. As some studies may have explored enjoyment as a secondary dimension and not explicitly mentioned it in the title or abstract, this approach allowed for their inclusion. Moreover, this method allowed for the inclusion of studies that evaluated enjoyment among other dimensions but did not reveal a significant impact. The search query employed keywords related to video games (video games, digital games, computer games, electronic games, or videogames), player (play, player, or playing), and experience (experience, enjoyment, engagement, immersion, presence, flow, or emotion). A total of 5,366 articles were identified, after removing duplicates and manually adding a few articles based on previously published literature reviews on PX.

The inclusion criteria for articles in the present review required that they (1) investigate the influence of game design factors (i.e., elements of games that can be manipulated by game designers and developers, whether software or hardware elements) on PX and (2) include controlled experiments (i.e., randomized controlled trials or within-participant experiments) that evaluate the efficacy of these design factors on PX compared to a control condition. One exclusion criterion was articles that studied games for which entertainment is not the primary goal, i.e., for which entertainment competes with another user goal such as learning, training, etc.

Each abstract in the search database was reviewed for compatibility between the article and the inclusion criteria. When the abstract was ambiguous, the entire text was reviewed. This review was performed independently by two individuals, including the first author of this article. The two reviewers then compared their results. If there was disagreement about an article, the two reviewers discussed their choice and together decided to include or exclude the article. A total of 221 articles, published between 2000 and 2021, met the inclusion criteria.

Next, articles reporting studies that assessed enjoyment, alone or among other dimensions of PX, were included in the final database of this review. A total of 70 articles, published between 2007 and 2021, met the final inclusion criteria and were included in this review.

Meta-analysis

To conduct an additional quantitative analysis, a subset of studies was selected from those reported in the included articles. Studies were included if at least two studies in the database had examined the same game design factor on player enjoyment and the data were available. If data were not provided in the article, the authors were contacted to provide them. If the data could not be obtained, the study was not included in the meta-analysis. The meta-analysis ultimately included 37 studies from 30 articles.

The statistical analysis used in this study was identical to that used by [START_REF] Caroux | Presence in video games: A systematic review and meta-analysis of the effects of game design choices[END_REF]. Mean and standard deviation (SD) values were used to calculate effect sizes (ES; Hedges' g) for each outcome in the experimental and control conditions. The experimental condition generally involved the original modality of the game design factor, while the control condition involved the classic modality or its absence (see the results section for details of the different groups compared). Data were standardized using post-intervention SD values. A random-effect model was chosen to analyze the effect of game design factors due to the use of similar but not identical methods to measure enjoyment in the included studies [START_REF] Deeks | & on behalf of the Cochrane Statistical Methods Group[END_REF]. ES values are presented with 95% confidence intervals (CI). Calculated ES were interpreted using the following scale: < 0.2, trivial; 0.2-0.6, small; > 0.6-1.2, moderate; > 1.2-2.0, large; > 2.0-4.0, very large; > 4.0, extremely large [START_REF] Hopkins | Progressive statistics for studies in sports medicine and exercise science[END_REF]. This scale is a translation of the interpretation scale of effects suggested by [START_REF] Cohen | Statistical power analysis for the behavioral sciences[END_REF] for standardized differences in means, as in this meta-analysis. Heterogeneity was assessed using the I² statistic [START_REF] Higgins | Quantifying heterogeneity in a metaanalysis[END_REF] and statistical significance was set at p ≤ 0.05.

The analysis was conducted using Review Manager 5.4.1 software (The Cochrane Collaboration, 2020).

Results

Techniques used to measure enjoyment

All of the articles included in this review studied enjoyment using scales and questionnaires. Table 1 shows the details of the instruments that were used according to their specificity to assess player enjoyment. The most commonly used instruments were ad-hoc, i.e., self-developed, non-standardized questionnaires (42 of 70 included articles, 60%). Of the remaining articles, which used standardized scales, the most commonly used scale was the Interest/Enjoyment subscale of the Intrinsic Motivation Inventory (IMI) [START_REF] Mcauley | Psychometric properties of the intrinsic motivation inventory in a competitive sport setting: A confirmatory factor analysis[END_REF][START_REF] Ryan | Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory[END_REF] (19 articles, 27%). Other standardized scales, such as the Instrument to Measure Enjoyment of Computer Game Play [START_REF] Fang | Development of an instrument to measure enjoyment of computer game play[END_REF] or the Enjoyment subscale of the Game User Experience Satisfaction Scale (GUESS) [START_REF] Phan | The development and validation of the game user experience satisfaction scale (GUESS)[END_REF] were used by a small number of included articles (between 1 and 3 articles per scale). The full list of articles included in the systematic review is detailed in the Appendix. Total number of articles 70

Game design factors studied

Table 2 shows that enjoyment has been studied in articles that involve many different game design factors. These game design factors fall under most of the major aspects of video games studied in game user research. We categorized the included articles according to the categorization proposed by [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF], namely in-game contents (i.e. gameplay mechanics and rules, narrative, and challenge) (36 articles), information input/output techniques (i.e. display and control modes, and interfaces) (34 articles), and multiplayer aspects (i.e., nature of the co-player, co-playing mode, and online gaming techniques) (11 articles). These categories relate to the main video game aspects of player-video game interaction that have been investigated in the previous empirical literature on PX. We followed this categorization to improve the readability of the following results. The following paragraphs detail the different game design factors that were studied in the articles included in the review. Avatar 5

Narrative 4

Feedbacks 4

Rewards 3

Sub-total 37

Input/output information

Control mode 19

Display mode 6

Visual interface 5

Auditory interface 4

Combined display & control device 1

Sub-total 33

Multiplayer Co-player nature 5

Co-playing mode 4

Online playing 3

Sub-total 11

Total number of articles 70

Note. An article that studied more than one game design factor was counted in multiple cells. Therefore, the total and sub-total numbers are not the sum of the corresponding cells.

In-game contents

Seventeen articles investigated the effects of difficulty characteristics. They focused on the effects of the level of game difficulty (e.g., easier or harder) [START_REF] Bateman | Effects of view, input device, and track width on video game driving[END_REF][START_REF] Corcos | Being enjoyably challenged is the key to an enjoyable gaming experience: An experimental approach in a first-person shooter game[END_REF][START_REF] De Simone | Is cheating a human function? The roles of presence, state hostility, and enjoyment in an unfair video game[END_REF][START_REF] Hagelback | Measuring player experience on runtime dynamic difficulty scaling in an RTS game[END_REF][START_REF] Klimmt | Player performance, satisfaction, and video game enjoyment[END_REF][START_REF] Misra | Exploring the pace of an endless runner game in stationary and mobile settings[END_REF][START_REF] Piselli | Relating cognitive models of computer games to user evaluations of entertainment[END_REF][START_REF] Schmierbach | No one likes to lose: The effect of game difficulty on competency, flow, and enjoyment[END_REF][START_REF] Tozman | Understanding the psychophysiology of flow: A driving simulator experiment to investigate the relationship between flow and heart rate variability[END_REF], on the effects of dynamic adjustment of game difficulty, i.e. techniques that adapt the level of difficulty in the course of the game play, either based on current performance or affective/emotional state of the player when playing [START_REF] Alexander | An investigation of the effects of game difficulty on player enjoyment[END_REF][START_REF] Demediuk | Challenging AI: Evaluating the effect of MCTS-driven dynamic difficulty adjustment on player enjoyment[END_REF][START_REF] Hagelback | Measuring player experience on runtime dynamic difficulty scaling in an RTS game[END_REF][START_REF] Liu | Dynamic difficulty adjustment in computer games through real-time anxiety-based affective feedback[END_REF][START_REF] Nagle | Towards a system of customized video game mechanics based on player personality: Relating the Big Five personality traits with difficulty adaptation in a first-person shooter game[END_REF][START_REF] Negini | Using affective state to adapt characters, NPCs, and the environment in a first-person shooter game[END_REF][START_REF] Obaid | Feed the Fish": An affect-aware game[END_REF][START_REF] Smeddinck | How to present game difficulty choices?: Exploring the impact on player experience[END_REF], or on the choice of difficulty level by the player [START_REF] Cuthbert | The effects of customisation on player experiences and motivation in a virtual reality game[END_REF][START_REF] Smeddinck | How to present game difficulty choices?: Exploring the impact on player experience[END_REF].

Six articles investigated the effects of gameplay mechanics, namely the level of action complexity (e.g., the number of in-game player actions required to complete a task) [START_REF] Berglund | Effects of reactive and strategic game mechanics in motion-based games[END_REF][START_REF] Piselli | Relating cognitive models of computer games to user evaluations of entertainment[END_REF][START_REF] Tamborini | Media enjoyment as need satisfaction: The contribution of hedonic and nonhedonic needs[END_REF], the social nature of a task (e.g., rescuing human or non-human entities) [START_REF] Wauck | Analyzing the effect of avatar self-similarity on men and women in a search and rescue game[END_REF], the implementation of a time limit to complete a task (I. G. [START_REF] Yildirim | Time pressure as video game design element and basic need satisfaction[END_REF], or the level geolocation sensor noise in an augmented reality game (i.e. that alters the accuracy of the geolocation of objects or players in a given physical space of game in the real world) [START_REF] Eishita | The impact on player experience in augmented reality outdoor games of different noise models[END_REF]. Five articles investigated the effects of avatar appearance in the game (i.e., the representation of the player in the virtual game environment). They focused on the possibility for players to customize themselves the appearance of their avatar [START_REF] Birk | Fostering intrinsic motivation through avatar identification in digital games[END_REF][START_REF] Cuthbert | The effects of customisation on player experiences and motivation in a virtual reality game[END_REF][START_REF] Schmierbach | Feeling the need for (personalized) speed: How natural controls and customization contribute to enjoyment of a racing game through enhanced immersion[END_REF], on the physical nature of the avatar, in terms of aesthetics appearance and physical behavior (human-like or robot) [START_REF] Normoyle | The effect of animation controller and avatar on player perceptions[END_REF], or on the possibility of having an avatar that physically resembles the player or a friend [START_REF] Wauck | Analyzing the effect of avatar self-similarity on men and women in a search and rescue game[END_REF]. Four articles investigated the effects of narrative in video games, which is concerned with how a story context is included in the game environment. They studied the effects of narrative structure (linear or branching, i.e. without or with possible alternatives in the story given the choices made by the player) [START_REF] Moser | Narrative control and player experience in role playing games: Decision points and branching narrative feedback[END_REF], 2015), the level of suspense in the story [START_REF] Klimmt | Experimental evidence for suspense as determinant of video game enjoyment[END_REF], or the presence or absence of a task goal presented to the player in the game [START_REF] Cicchirillo | The impact of video game character viewpoints and task on perceptions of cognitive and similarity identification[END_REF]. Four articles investigated the effects of in-game feedback (e.g., score, progress, etc.), i.e., its presence or absence, its visual appearance, or its effectance (perceived influence of players' actions on the game world) [START_REF] Bowey | Manipulating leaderboards to induce player experience[END_REF][START_REF] Kao | The effects of juiciness in an action RPG[END_REF][START_REF] Klimmt | Effectance and control as determinants of video game enjoyment[END_REF][START_REF] Siemens | Level up! The role of progress feedback type for encouraging intrinsic motivation and positive brand attitudes in public versus private gaming contexts[END_REF]. Finally, three articles studied the effects of the rewards provided when players reach a goal in the game, i.e., their presence or absence, their type and variety, or their adaptability to the player's personality [START_REF] Johnson | Greater rewards in videogames lead to more presence, enjoyment and effort[END_REF][START_REF] Phillips | The impact of rewards and trait reward responsiveness on player motivation[END_REF][START_REF] Volkmar | Player types and achievements-Using adaptive game design to foster intrinsic motivation[END_REF].

Information input/output techniques

Nineteen articles studied the effects of control mode. Most of these articles studied motion-based controllers, including tangible controllers (e.g., Wiimote, PS Move, or steering wheel controllers) or whole-body controllers (e.g., Kinect sensor) [START_REF] Berglund | Effects of reactive and strategic game mechanics in motion-based games[END_REF][START_REF] Bozgeyikli | Introducing tangible objects into motion controlled gameplay using Microsoft® Kinect TM : Introducing tangible objects into motion controlled gameplay[END_REF][START_REF] Kniestedt | Including non-gamers: A case study comparing touch and motion input in a 3D game for research[END_REF][START_REF] Limperos | Gaming across different consoles: Exploring the influence of control scheme on gameplayer enjoyment[END_REF][START_REF] Mcgloin | The impact of controller naturalness on spatial presence, gamer enjoyment, and perceived realism in a tennis simulation video game[END_REF][START_REF] Peña | Playing with power: Power poses affect enjoyment, presence, controller responsiveness, and arousal when playing natural motioncontrolled video games[END_REF][START_REF] Schmierbach | Feeling the need for (personalized) speed: How natural controls and customization contribute to enjoyment of a racing game through enhanced immersion[END_REF][START_REF] Shafer | The effects of interaction fidelity on game experience in virtual reality[END_REF][START_REF] Shafer | Spatial presence and perceived reality as predictors of motion-based video game enjoyment[END_REF][START_REF] Shafer | Controller required? The impact of natural mapping on interactivity, realism, presence, and enjoyment in motionbased video games[END_REF][START_REF] Skalski | Mapping the road to fun: Natural video game controllers, presence, and game enjoyment[END_REF][START_REF] Smeddinck | How to present game difficulty choices?: Exploring the impact on player experience[END_REF][START_REF] Tamborini | Defining media enjoyment as the satisfaction of intrinsic needs[END_REF][START_REF] Williams | The effects of dissociation, game controllers, and 3D versus 2D on presence and enjoyment[END_REF]. The other articles studied the effects of the level of responsiveness of the controls [START_REF] Jörg | How responsiveness affects players' perception in digital games[END_REF][START_REF] Normoyle | The effect of animation controller and avatar on player perceptions[END_REF], controller physical realism [START_REF] Wechselberger | Music game enjoyment and natural mapping beyond intuitiveness[END_REF], left-handed controllers [START_REF] Maubert Crotte | Left-handed control configuration for side-scrolling games[END_REF], and use of a paper-based sketching interface [START_REF] Macret | Can a paper-based sketching interface improve the gamer experience in strategy computer games?[END_REF]. Six articles studied the effects of visual display type. These articles focused on innovative methods of displaying visual information, such as head-mounted displays [START_REF] Lemmens | Fear and loathing in VR: The emotional and physiological effects of immersive games[END_REF][START_REF] Sekhavat | Sense of immersion in computer games using single and stereoscopic augmented reality[END_REF][START_REF] Weber | Experiencing presence in a gaming activity improves mood after a negative mood induction[END_REF][START_REF] Yildirim | Video game user experience: To VR, or not to VR?[END_REF], and 3Dstereoscopic displays [START_REF] Klimmt | Experimental evidence for suspense as determinant of video game enjoyment[END_REF][START_REF] Williams | The effects of dissociation, game controllers, and 3D versus 2D on presence and enjoyment[END_REF]. Five articles investigated the effects of the visual interface, namely the visual point of view (first person or third person) [START_REF] Bateman | Effects of view, input device, and track width on video game driving[END_REF][START_REF] Cicchirillo | The impact of video game character viewpoints and task on perceptions of cognitive and similarity identification[END_REF], the level of graphical fidelity [START_REF] Gerling | The effects of graphical fidelity on player experience[END_REF], or the amount of contextual information displayed [START_REF] Teruel | Assessing the impact of the awareness level on a co-operative game[END_REF] and its visual integration into the game world (diegetic design) [START_REF] Marre | Video game interfaces and diegesis: The impact on experts and novices' performance and experience in virtual reality[END_REF]. Four articles investigated the effects of the auditory interface, namely the presence of music [START_REF] Cassidy | The effects of music on time perception and performance of a driving game: Effects of music on time perception[END_REF][START_REF] Klimmt | Effects of soundtrack music on the video game experience[END_REF], the presence of sound effects [START_REF] Ballard | Horror videogame sound effects, cardiovascular arousal, and ethnic bias in the decision to shoot[END_REF][START_REF] Robb | The impact of health-related user interface sounds on player experience[END_REF]. Finally, one article investigated the effects of systems that combine a head-mounted display with motion-based controllers, such as in VR gaming systems [START_REF] Sekhavat | Sense of immersion in computer games using single and stereoscopic augmented reality[END_REF].

Multiplayer aspects

Five articles investigated the effects of the nature of the co-players, i.e. who can be controlled by another human being or by the computer [START_REF] Merritt | Choosing human team-mates: Perceived identity as a moderator of player preference and enjoyment[END_REF][START_REF] Shafer | Causes of state hostility and enjoyment in player versus player and player versus environment video games[END_REF][START_REF] Tamborini | Defining media enjoyment as the satisfaction of intrinsic needs[END_REF][START_REF] Vella | The impact of agency and familiarity in cooperative multiplayer games[END_REF][START_REF] Weibel | Playing online games against computer-vs. Human-controlled opponents: Effects on presence, flow, and enjoyment[END_REF]. Four articles studied the effects of the presence of a multiplayer game mode, and the nature of that mode, i.e. with a collaborative or cooperative goal between players, or with a competitive goal [START_REF] Beznosyk | User enjoyment and performance in collaborative and cooperative games in shared 3d virtual environments[END_REF][START_REF] Arellano | Measuring affective, physiological and behavioural differences in solo, competitive and collaborative games[END_REF][START_REF] Peng | Playing in parallel: The effects of multiplayer modes in active video game on motivation and physical exertion[END_REF][START_REF] Schmierbach | Electronic friend or virtual foe: Exploring the role of competitive and cooperative multiplayer video game modes in fostering enjoyment[END_REF]. Three articles investigated the effects of online play, i.e. interaction in the same game environment without being in the same physical location [START_REF] Heeter | Engineering sociability: Friendship drive, visibility, and social connection in anonymous colocated local wi-fi multiplayer online gaming[END_REF][START_REF] Peng | Playing in parallel: The effects of multiplayer modes in active video game on motivation and physical exertion[END_REF][START_REF] Reer | A self-determination theory-based laboratory experiment on social aspects of playing multiplayer first-person shooter games[END_REF].

Meta-analysis

The meta-analysis was conducted on the basis of the influence of different game design factors on enjoyment. A total of 13 analyses were conducted. It should be noted that only two analyses showed low statistical heterogeneity (I² = 0%) (level of control responsiveness, 3D-stereoscopic or 2D display). Two analyses showed substantial statistical heterogeneity (I² > 50%) (avatar choice, music presence or absence). The other nine analyses showed considerable statistical heterogeneity (I² > 75%). These large heterogeneities, which are often inevitable in meta-analyses, were due to the different magnitude and direction of effects in the different studies that were aggregated (e.g., [START_REF] Higgins | Measuring inconsistency in meta-analyses[END_REF]. For example, regarding the motion-based control type, some studies showed higher immersion with body motion-based control while other showed higher immersion with tangible motion-based control.

Discussion

Main findings and implications

The main objective of the present research was to provide a more comprehensive view of the elements of game design that have an effect on the feeling of enjoyment in video games. The review shows that studies investigated most of the major aspects of video game design, namely in-game contents, input and output information, and, to a lesser extent, multiplayer aspects. The variety of game design factors studied in these articles was high, with fourteen game design factors studied. It is quite comparable to the review by [START_REF] Caroux | Presence in video games: A systematic review and meta-analysis of the effects of game design choices[END_REF] on presence (13 game design factors). Only "feedbacks" were not addressed in this previous review. The total number was higher in the present review than those on presence (70 vs. 55 included articles), which allowed for more analyses (13 vs 11 analyses). Two game design factors were particularly studied when it came to determining their effects on player enjoyment: difficulty, which accounted for 43% of the articles on in-game contents (16 of 37 articles), and control mode, which accounted for 58% of the articles on input/output information (19 of 33 articles). These two factors were also well represented in the meta-analysis. Two comparisons were made for difficulty, related to difficulty level and dynamic difficulty adjustment. Four comparisons were made for control mode, in relation to motion-based control (whether tangible or body control) and level of control responsiveness.

Interestingly, the meta-analysis showed only one significant effect across 13 comparisons. This effect was for the presence of music in the game, versus its absence, and was interpreted as a small effect. The remaining comparisons, including factors that have been studied in a larger number of articles such as those related to game difficulty and control mode, did not reveal significant effects. It is worth noting that these results

are not really in line with previous literature reviews on the topic of enjoyment in video games. Reviews such as [START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF], [START_REF] Mekler | A systematic review of quantitative studies on the enjoyment of digital entertainment games[END_REF], Segundo Díaz et al.

(2022), or [START_REF] Schaffer | Digital game enjoyment: A literature review[END_REF] have reported the results of large sets of published studies that showed enjoyment could be influenced by several game design factors of in-game contents (e.g., difficulty, rewards, narrative, etc.), input/output information (e.g., interface, sounds, etc.), and multiplayer aspects (e.g., cooperation/competition, coplayer nature, etc.). While these studies were able to show significant effects of game design factors on player enjoyment in isolation, combining results from several experimental studies did not reveal that these well-studied factors would have a significant effect on enjoyment.

Several studies included in the database for the present research did not study enjoyment as a primary variable. Some of these studies could have primarily investigated other dimensions of PX such as presence, immersion, or engagement. The results of these studies may have revealed significant effects for these other dimensions of PX, but not for enjoyment. The results of the present meta-analysis show that it was necessary to include articles that did not study enjoyment as a primary variable. This provided a larger set of studies than those that published significant positive effects of game design factors on enjoyment as a primary variable. Further research should enhance the study of PX in a broader way than simply studying enjoyment or another single dimension of PX, in order to get a more complete picture of a given game design factor on PX.

The fact that the present meta-analysis shows that only one game design factor out of the thirteen tested would have a real impact on player enjoyment may also be a call to better investigate the diversity of player and game characteristics in these studies (e.g., [START_REF] Schmierbach | Content analysis of video games: Challenges and potential solutions[END_REF]. Most of the analyses performed in the present meta-analysis showed substantial to considerable statistical heterogeneity. Some of the large differences between the results of studies investigating the same given game design factor may be explained by the diversity of player and/or game characteristics involved in the studies. Player characteristics such as expertise or skills, motivations, psychological outcomes of play (e.g., feeling of being in control, self-efficacy, etc.) could be other factors that could influence player enjoyment (e.g., Segundo [START_REF] Segundo Díaz | Building blocks for creating enjoyable games-A systematic literature review[END_REF]. In addition, the content of games can differ highly from level to level and game mode to game mode. A player can encounter a different experience at different times in the game or during different sessions of play. This is largely due to the interactive nature of the game that may produce large variations of PX, including enjoyment, over time. As Schmierbach (2009) also noted, results of studies on PX may also depend of the expertise of researchers in video games, and in particular the game they studied. In some studies, researchers may attempt to make sense of games they have never played and do not understand. Many complex activities take place in a video game, and those activities are often only decipherable to people who are "literate" in the symbols of the game. The quality of the measurement or analysis of player enjoyment can vary, for example, in terms of studying the "right" time of play with the "right" type of players' characteristics. Although some studies have found effects in this direction, more empirical research is needed to confirm precisely whether and how each of these player and game diversity variables might influence enjoyment in isolation and in interaction with game design factors.

A secondary objective of this research was to explore the different techniques used to assess player enjoyment. The review showed that measuring enjoyment in video games is a process that is not completely standardized. Most of the articles included in the present review (60% of the articles, 42 out of 70) used ad-hoc questionnaires, i.e.

non-standardized instruments, to assess enjoyment. One explanation could be that enjoyment in video games has long been a vague concept, which has been confused

with concepts such as flow, fun, engagement, happiness, etc. (e.g., [START_REF] Crutzen | Enjoyment: A conceptual exploration and overview of experimental evidence in the context of games for health[END_REF][START_REF] Matthews | A framework for the assessment of enjoyment in video games[END_REF]. Authors might then consider that enjoyment does not need to be rigorously assessed. An interesting example is the use in some studies of a one-item scale, as previously noted by [START_REF] Mekler | A systematic review of quantitative studies on the enjoyment of digital entertainment games[END_REF]. The present review showed that several standardized scales exist and can be used to properly assess enjoyment. Some of these scales were designed specifically (fully or partly) to assess enjoyment in video games, such as the Instrument to Measure Enjoyment of Computer Game Play [START_REF] Fang | Development of an instrument to measure enjoyment of computer game play[END_REF], or the Enjoyment subscale of the Game User Experience Satisfaction Scale (GUESS) [START_REF] Phan | The development and validation of the game user experience satisfaction scale (GUESS)[END_REF]. Further research should assess enjoyment carefully, avoiding non-standardized questionnaires, and using one of the existing validated and relevant instruments.

In addition, enjoyment has been widely assessed using scales that consider this construct as global. However, some authors have proposed that enjoyment can be considered as a multi-dimensional construct.

Limitations

Like many systematic reviews and meta-analyses, this study's outcomes might be limited by the pool of literature analyzed. However, despite the relative newness of game user research, this review included a fairly large number of articles. Yet, the review's constraints emerged when examining certain game design factors in more detail. Specifically, some game design factors detailed in the review were not incorporated into the meta-analysis due to their distinct research questions. These were gameplay, narrative, feedbacks and rewards regarding the in-game contents factors; combined display and control device regarding the input/output information factors; and co-playing mode and online playing regarding the multiplayer factors. It should be noted that their absence in the meta-analysis does not mean that they do not have a significant effect on player enjoyment. This suggests that future research should further study these factors in order to gather more data on their potential effect on player enjoyment. In addition, while the meta-analysis drew comparisons on a fairly large number of studies (5 or 6 studies), other comparisons were based on a smaller number

(2 or 3 studies), potentially compromising the findings' validity. A small number of studies may reduce the likelihood of revealing whether or not there is an effect of a given factor on player enjoyment.

Finally, this literature review and meta-analysis focused on studies on games where entertainment is the primary goal. This criterion excluded games with other purposes (e.g., learning, training, etc.). This may have excluded elements that could still be of interest to the questions posed in the present research, such as more data for statistical comparisons or more existing instruments for assessing player enjoyment. For example, scales such as the GAMEX scale have been developed to assess "gameful experience" in gamified applications [START_REF] Eppmann | Gameful experience in gamification: Construction and validation of a gameful experience scale [GAMEX[END_REF]. Although it was not developed to measure player experience in games where the primary purpose is entertainment, dimensions of this scale may be relevant to assessing player enjoyment.

Conclusion

The present article reports a systematic review and meta-analysis of the effects of game design factors on the feeling of enjoyment. The systematic review showed that studies investigated most of the major aspects of video game design, namely in-game contents, input and output information, and, to a lesser extent, multiplayer aspects. Two game design factors were particularly studied when investigating their effects on player enjoyment: difficulty (e.g., difficulty level, dynamic difficulty adjustment) and control mode (especially motion-based control). The meta-analysis showed that only the presence of music in the game, as opposed to its absence, had a significant effect on player enjoyment. The other analyses did not reveal any other significant effects of game design factors on enjoyment. The systematic review also showed that measuring enjoyment in video games is a process that is not fully standardized, as only a minority of articles used standardized scales to assess enjoyment. This study revealed that future research on player enjoyment should more experimentally investigate whether and how factors related to the diversity of game and player characteristics might influence enjoyment in isolation and in interaction with game design factors. Also, the assessment of enjoyment should be conducted using existing, validated, and relevant instruments, such as standardized scales. In addition, the use of less commonly used instruments such as multidimensional subjective scales or physiological state measurement, should also be further explored.

Moving forward, as noted by [START_REF] Caroux | Presence in video games: A systematic review and meta-analysis of the effects of game design choices[END_REF], future meta-analyses should continue to investigate the effects of game design factors on the other main dimensions of PX, namely immersion and flow for the most important ones. There are already a few literature reviews on these topics (e.g., [START_REF] Cruz | Player-centered game AI from a flow perspective: Towards a better understanding of past trends and future directions[END_REF][START_REF] Cairns | Immersion in digital games: Review of gaming experience research[END_REF][START_REF] Caroux | Player-video game interaction: A systematic review of current concepts[END_REF][START_REF] Khoshnoud | Peripheral-physiological and neural correlates of the flow experience while playing video games: A comprehensive review[END_REF][START_REF] Wilkinson | A mini review of presence and immersion in virtual reality[END_REF], but none of them provide quantitative evidence regarding these key dimensions of PX.

  For example, Fang et al. (2010) proposed a scale that assessed enjoyment along three dimensions: affective, behavioral and cognitive. Surprisingly, only 3 articles included in the present review (out of 28 articles that used standardized scales) used this scale to assess enjoyment. It would be interesting to know what type of enjoyment a given game design factor would impact. Further research should consider assessing enjoyment in more detail by studying it with several sub-dimensions. Finally, all of the included articles in the present review used subjective techniques to assess player enjoyment. More objective techniques could also be used to assess PX. In particular, players' physiological state could be an indicator of their experience in a given play situation (see Calvo-Morata et al., 2022 for a review). None of the included articles used these techniques to explicitly assess enjoyment. Some of them assessed related concepts such as arousal or emotional valence by measuring physiological parameters, but not in relation with enjoyment. Further research should further explore the relationship between physiological parameters and player enjoyment, and their use, in conjunction with standardized scales, to investigate how enjoyment might be influenced by game design factors.

Table 1 .

 1 Questionnaires and scales used in the included articles to assess enjoyment

			Sub-	Specific to
		Specific to	dimension	video	Number
	Instrument	enjoyment	of a more	game	of
		assessment	general	player	articles
			instrument	experience
	Interest/Enjoyment subscale of the Intrinsic Motivation Inventory (IMI) (McAuley et al., 1989; Ryan, 1982)		X		19
	Instrument to Measure Enjoyment of Computer Game Play (Fang et al., 2010)	X		X	3
	Items from the Flow State Scale (Jackson & Marsh, 1996)		X		2
	Items from the Media enjoyment scale (Murry Jr. & Dacin, 1996)	X	X		1
	Enjoyment subscale of the Core Elements of the Gaming
	Experience Questionnaire (CEGEQ) (Calvillo Gámez et al.,		X	X	1
	2010)
	Emotional involvement and enjoyment items from the
	Immersive Experience Questionnaire (IEQ) (Jennett et al.,		X		1
	2008)
	Enjoyment subscale of the Game User Experience Satisfaction Scale (GUESS) (Phan et al., 2016)		X	X	1
	Ad-hoc questionnaire				42

Table 2 .

 2 Number of articles included in the systematic review database by game design factor

	Game aspect	Game design factor	Total number of articles
		Difficulty	17
		Gameplay	6
	In-game
	contents

Table 3 .

 3 Table3presents the results of these analyses of the influence of game design factors on enjoyment. There was only one significant small positive effect of the presence vs. absence of music on enjoyment. There were non-significant effects of dynamic difficulty adjustment vs. nonadaptive difficulty, high vs. low level of difficulty, avatar choice vs. default avatar, motion-based vs. classic non-motion-based control, tangible motion-based vs. classic controller, body motion-based control vs. tangible motion-based controller, high vs. low level of control responsiveness, head-mounted vs. monitor display, 3D-stereoscopic vs. Meta-analysis of effects of game design factors on enjoyment

	2D display, first-person vs. third-person point of view, presence vs. absence of sound
	effects, and human vs. computer co-player, on enjoyment.

Acknowledgements

The authors thank Anastasia Passemar for her help in reviewing the articles in the search database.

Declaration of interest statement

The authors report there are no competing interests to declare.

Authors' biographies

Loïc Caroux is an associate professor in cognitive ergonomics at the University of Toulouse Jean Jaurès, France. He received his PhD in cognitive ergonomics in 2012 from the University of Poitiers, France. His research interests deal with human factors/ergonomics in everyday computing systems.

Morgane Pujol is a PhD student in cognitive ergonomics at the University of Toulouse Jean Jaurès, France, since 2022. Her thesis focuses on the state of flow in video games.

More specifically, she is interested in how this state can be induced and maintained in players.

Appendix: List of articles included in the systematic review according to the game design factor studied

Game aspect Game design factor Articles included

In-game contents Difficulty [START_REF] Alexander | An investigation of the effects of game difficulty on player enjoyment[END_REF][START_REF] Bateman | Effects of view, input device, and track width on video game driving[END_REF][START_REF] Corcos | Being enjoyably challenged is the key to an enjoyable gaming experience: An experimental approach in a first-person shooter game[END_REF][START_REF] Cuthbert | The effects of customisation on player experiences and motivation in a virtual reality game[END_REF][START_REF] De Simone | Is cheating a human function? The roles of presence, state hostility, and enjoyment in an unfair video game[END_REF][START_REF] Demediuk | Challenging AI: Evaluating the effect of MCTS-driven dynamic difficulty adjustment on player enjoyment[END_REF][START_REF] Hagelback | Measuring player experience on runtime dynamic difficulty scaling in an RTS game[END_REF][START_REF] Klimmt | Player performance, satisfaction, and video game enjoyment[END_REF][START_REF] Liu | Dynamic difficulty adjustment in computer games through real-time anxiety-based affective feedback[END_REF][START_REF] Misra | Exploring the pace of an endless runner game in stationary and mobile settings[END_REF][START_REF] Nagle | Towards a system of customized video game mechanics based on player personality: Relating the Big Five personality traits with difficulty adaptation in a first-person shooter game[END_REF][START_REF] Negini | Using affective state to adapt characters, NPCs, and the environment in a first-person shooter game[END_REF][START_REF] Obaid | Feed the Fish": An affect-aware game[END_REF][START_REF] Piselli | Relating cognitive models of computer games to user evaluations of entertainment[END_REF]Schmierbach et al., [START_REF] Kniestedt | Including non-gamers: A case study comparing touch and motion input in a 3D game for research[END_REF][START_REF] Limperos | Gaming across different consoles: Exploring the influence of control scheme on gameplayer enjoyment[END_REF][START_REF] Macret | Can a paper-based sketching interface improve the gamer experience in strategy computer games?[END_REF][START_REF] Maubert Crotte | Left-handed control configuration for side-scrolling games[END_REF][START_REF] Mcgloin | The impact of controller naturalness on spatial presence, gamer enjoyment, and perceived realism in a tennis simulation video game[END_REF][START_REF] Normoyle | The effect of animation controller and avatar on player perceptions[END_REF][START_REF] Peña | Playing with power: Power poses affect enjoyment, presence, controller responsiveness, and arousal when playing natural motioncontrolled video games[END_REF][START_REF] Schmierbach | Feeling the need for (personalized) speed: How natural controls and customization contribute to enjoyment of a racing game through enhanced immersion[END_REF][START_REF] Shafer | Spatial presence and perceived reality as predictors of motion-based video game enjoyment[END_REF][START_REF] Shafer | Controller required? The impact of natural mapping on interactivity, realism, presence, and enjoyment in motionbased video games[END_REF][START_REF] Shafer | The effects of interaction fidelity on game experience in virtual reality[END_REF][START_REF] Skalski | Mapping the road to fun: Natural video game controllers, presence, and game enjoyment[END_REF][START_REF] Smeddinck | How to present game difficulty choices?: Exploring the impact on player experience[END_REF][START_REF] Tamborini | Defining media enjoyment as the satisfaction of intrinsic needs[END_REF][START_REF] Wechselberger | Music game enjoyment and natural mapping beyond intuitiveness[END_REF] Co-playing mode [START_REF] Beznosyk | User enjoyment and performance in collaborative and cooperative games in shared 3d virtual environments[END_REF][START_REF] Arellano | Measuring affective, physiological and behavioural differences in solo, competitive and collaborative games[END_REF][START_REF] Peng | Playing in parallel: The effects of multiplayer modes in active video game on motivation and physical exertion[END_REF][START_REF] Schmierbach | Electronic friend or virtual foe: Exploring the role of competitive and cooperative multiplayer video game modes in fostering enjoyment[END_REF] Online playing [START_REF] Heeter | Engineering sociability: Friendship drive, visibility, and social connection in anonymous colocated local wi-fi multiplayer online gaming[END_REF][START_REF] Peng | Playing in parallel: The effects of multiplayer modes in active video game on motivation and physical exertion[END_REF][START_REF] Reer | A self-determination theory-based laboratory experiment on social aspects of playing multiplayer first-person shooter games[END_REF] Note. * denotes an article for which one or more studies are included in the meta-

analysis. An article may be cited more than once in the table whether it includes one or several studies that manipulate different game design factors.