Jochen Staudacher

Stefano Moretti

Felix Fritz
email: felix.fritz@dauphine.eu

socialranking: A package for evaluating ordinal power relations in cooperative game theory

Keywords: power relation, social ranking solution, cooperative game theory, dominance, cp-majority, Copeland, Kramer-Simpson, ordinal Banzhaf, dominance, lexicographical excellence, relations

This document gives a brief introduction to power relations and social ranking solutions aimed at ranking elements based on their contributions within coalitions. This document accompanies version 1.0.0 of the package socialranking.

Introduction

In the literature of cooperative games, the notion of power index [START_REF] Shapley | A method for evaluating the distribution of power in a committee system[END_REF][START_REF] Banzhaf | Weighted voting doesn't work: A mathematical analysis[END_REF][START_REF] Holler | Power, voting, and voting power: 30 years after[END_REF] has been widely studied to analyze the "influence" of individuals taking into account their ability to force a decision within groups or coalitions. In practical situations, however, the information concerning the strength of coalitions is hardly quantifiable. So, any attempt to numerically represent the influence of groups and individuals clashes with the complex and multiattribute nature of the problem and it seems more realistic to represent collective decision-making mechanisms using an ordinal coalitional framework based on two main ingredients: a binary relation over groups or coalitions and a ranking over the individuals.

The main objective of the package socialranking is to provide answers for the general problem of how to compare the elements of a finite set N given a ranking over the elements of its power-set (the set of all possible subsets of N ). To do this, the package socialranking implements a portfolio of solutions from the recent literature on social rankings [START_REF] Moretti | Some axiomatic and algorithmic perspectives on the social ranking problem[END_REF][START_REF] Khani | An ordinal banzhaf index for social ranking[END_REF][START_REF] Haret | Ceteris paribus majority for social ranking[END_REF][START_REF] Fayard | Ordinal social ranking: Simulation for CP-majority rule[END_REF][START_REF] Bernardi | Ranking objects from a preference relation over their subsets[END_REF][START_REF] Algaba | Lexicographic solutions for coalitional rankings[END_REF][START_REF] Allouche | Social ranking manipulability for the CP-majority, banzhaf and lexicographic excellence solutions[END_REF].

Quick start

A power relation (i.e, a ranking over subsets of a finite set N ; see the Section 2 for a formal definition) can be constructed using the functions PowerRelation() or as.PowerRelation(). Functions used to analyze a given PowerRelation object can be grouped into three main categories:

• Comparison functions, only comparing two elements;

• Score functions, calculating the scores for each element;

• Ranking functions, creating SocialRanking objects. Comparison and score functions are often used to evaluate a social ranking solution (see section 2 for a formal definition). Listed below are some of the most prominent functions and solutions introduced in the aforementioned papers. These functions may be called as follows. Lastly, an incidence matrix for all given coalitions can be constructed using powerRelationMatrix(pr) or as.relation(pr) from the relations package [START_REF] Meyer | Relations: Data structures and algorithms for relations[END_REF].

The incidence matrix may be displayed using relations::relation_incidence().

## ab 1 1 1 1 1 1 1 1 ## abc 0 1 1 1 1 1 1 1 ## ac 0 1 1 1 1 1 1 1 ## bc 0 1 1 1 1 1 1 1 ## a 0 0 0 0 1 1 1 1 ## c 0 0 0 0 1 1 1 1 ## {} 0 0 0 0 0 0 1 1 ## b 0 0 0 0 0 0 0 1

PowerRelation objects

We first introduce some basic definitions on binary relations. Let X be a set. A set R ⊆ X × X is said a binary relation on X. For two elements x, y ∈ X, xRy refers to their relation, more formally it means that (x, y) ∈ R. A binary relation (x, y) ∈ R is said to be

• reflexive, if for each x ∈ X, xRx, • transitive, if for each x, y, z ∈ X, xRy and yRz ⇒ xRz, • total, if for each x, y ∈ X, x ̸ = y ⇒ xRy or yRx, • symmetric, if for each x, y ∈ X, xRy ⇔ yRx,
• asymmetric, if for each x, y ∈ X, (x, y) ∈ R ⇒ (y, x) / ∈ R, and

• antisymmetric, if for each x, y ∈ X, xRy ∩ yRx ⇒ x = y. A preorder is defined as a reflexive and transitive relation. If it is total, it is called a total preorder. Additionally if it is antisymmetric, it is called a linear order.

Let N = {1, 2, . . . , n} be a finite set of elements, sometimes also called players. For some p ∈ {1, . . . , 2 n }, let P = {S 1 , S 2 , . . . , S p } be a set of coalitions such that S i ⊆ N for all i ∈ {1, . . . , p}. Thus P ⊆ 2 N , where 2 N denotes the power set of N , the set of all subsets or coalitions of N ,

S ∈ 2 N ⇔ S ⊆ N.
T (N ) denotes the set of all total preorders on N , T (P) the set of all total preorders on P. A single total preorder ⪰∈ T (P) is said a power relation.

In a given power relation ⪰∈ T (P) on P ⊆ 2 N , its symmetric part is denoted by ∼ (i.e., S ∼ T if S ⪰ T and T ⪰ S), whereas its asymmetric part is denoted by ≻ (i.e., S ≻ T if S ⪰ T and not T ⪰ S). In other terms, for S ∼ T we say that S is indifferent to T , whereas for S ≻ T we say that S is strictly better than T . Lastly for a given power relation in the form of S 1 ⪰ S 2 ⪰ . . . ⪰ S m , coalitions that are indifferent to one another can be grouped into equivalence classes i such that we get the quotient order 1 ≻ 2 ≻ . . . ≻ m .

Example 1.

Let N = {1, 2} be two players with its corresponding power set 2 N = {{1, 2}, {1}, {2}, ∅}. The following power relation is given:

⪰ = {({1, 2}, {1, 2}), ({1, 2}, {2}), ({1, 2}, ∅), ({1, 2}, {1}), ({2}, {2}), ({2}, ∅), ({2}, {1}), (∅, ∅), (∅, {2}), (∅, {1}), ({1}, {1}) }
This power relation can be rewritten in a consecutive order as: {1, 2} ≻ {2} ∼ ∅ ≻ {1}. Its quotient order is formed by three equivalence classes 1 = {{1, 2}}, 2 = {{2}, ∅}, and 3 = {{1}}; so the quotient order of ⪰ is such that {{1, 2}} ≻ {{2}, ∅} ≻ {{1}}.

Note that the way the set ⪰ is presented in the example is somewhat deliberate to better visualize occurring symmetries and asymmetries. This also lets us neatly represent a power relation in the form of an incidence matrix in chapter 4.

Creating PowerRelation objects

A power relation in the socialranking package is defined to be reflexive, transitive and total. In designing the package it was deemed logical to have the coalitions specified in a consecutive order, as seen in Example 1. Each coalition in that order is split either by a ">" (left side strictly better) or a "~" (two coalitions indifferent to one another). The following code chunk shows the power relation from Example Some may have spotted a "SingleCharElements" class missing in class(prLong) that has been there in class(pr). "SingleCharElements" influences how coalitions are printed. If it is removed from class(pr), the output will include the same curly braces and commas displayed in prLong.

class(pr) <class(pr)[-which(class(pr) == "SingleCharElements")] pr ## {1, 2} > ({2} ~{}) > {1}

Internally a PowerRelation is a list with four attributes.

While coalitions are formally defined as sets, meaning the order doesn't matter and each element is unique, the package tries to stay flexible. As such, coalitions will only be sorted during initialization, but duplicate elements will not be removed.

#! -1, 2 in the coalition {1, 1, 2, 2, 2} prAtts ## 11222 > (12 ~{}) prAtts$elements ## [1] 1 2 prAtts$coalitionLookup(c(1,2)) ## [1] 2 prAtts$coalitionLookup(c(2,1)) ## [1] 2 prAtts$coalitionLookup(c(2,1,2,1,2)) ## [1] 1 prAtts$elementLookup(2) ## [[1]] ## [1] 1 1 ## ## [[2]] ## [1] 1 1 ## ## [[3]] ## [1] 1 1 ## ## [[4]] ## [1] 2 1

Manipulating PowerRelation objects

It is strongly discouraged to directly manipulate PowerRelation objects, as its attributes are so tightly coupled. This would require updates in multiple places. Instead, it is advisable to simply create new PowerRelation objects.

To permutate the order of equivalence classes, it is possible to take the equivalence classes in $eqs and use a vector of indexes to move them around.

PowerRelation(rev(pr$eqs)) ## 2 > (1 ~{}) > 12

For permutating individual coalitions, using as.PowerRelation.list() may be more convenient since it doesn't require nested list indexing.

coalitions <unlist(pr$eqs

, recursive = FALSE) compares <-c(">", "~", ">") as.PowerRelation(coalitions[c(2,1,3,4)], comparators = compares) ## 1 > (12 ~{}) > 2

# notice that the length of comparators does not need to match # length(coalitions)-1 as.PowerRelation(rev(coalitions), comparators = c("~", ">")) ## (2 ~{}) > (1 ~12)

# not setting the comparators parameter turns it into a linear order as.PowerRelation(coalitions) ## 12 > 1 > {} > 2

appendMissingCoalitions()

Let ⪰∈ T (P). We may have not included all possible coalitions, such that P ⊂ 2 N , P ̸ = 2 N . appendMissingCoalitions() appends all the missing coalitions 2 N -P as a single equivalence class to the end of the power relation.

makePowerRelationMonotonic()

A power relation ⪰∈ T (P) is monotonic if S ⪰ T ⇒ T ⊂ S
for all S, T ⊆ N . In other terms, given a monotonic power relation, for any coalition, all its subsets cannot be ranked higher. makePowerRelationMonotonic() turns a potentially non-monotonic power relation into a monotonic one by moving and (optionally) adding all missing coalitions in 2 N -P to the corresponding equivalence classes.

Creating power sets

As the number of elements n increases, the number of possible coalitions increases to

|2 N | = 2 n
. createPowerset is a convenient function that not only creates a power set 2 N which can be used to call PowerRelation or as.PowerRelation, but also formats the function call in such a way that makes it easy to rearrange the ordering of the coalitions.

RStudio offers shortcuts such as Alt+Up or Alt+Down (Option+Up or Option+Down on MacOS) to move one or multiple lines of code up or down (see fig. 1).

createPowerset(

c("a", "b", "c"), result = "print" ) ## as.PowerRelation("

## abc ## > ab ## > ac ## > bc ## > a ## > b ## > c ## > {} ## ")
<-createPowerset(1:2, includeEmptySet = FALSE) ps ## [[1]] ## [1] 1 2 ## ## [[2]] ## [1] 1 ## ## [[3]] ## [1] 2 as.PowerRelation(ps) ## 12 > 1 > 2 # equivalent PowerRelation(list(ps)) ## (12 ~1 ~2) as.PowerRelation(createPowerset(letters[1:4])) ## abcd > abc > abd > acd > bcd > ab > ac > ad > bc > bd > cd > a > b > c > d > {}

Generating PowerRelation objects

Given a list of coalitions, it is possible to loop through all possible permutations of power relations using powerRelationGenerator(). Calling gen() in the example below always produces a unique PowerRelation object. If all permutations have been exhausted, NULL is returned.

coalitions <-list(c(1,2), 1, 2) gen <-powerRelationGenerator(coalitions) while(!is.null(pr <-gen())) { print(pr) } ## (12 ~1 ~2) ## (12 ~1) > 2 ## (12 ~2) > 1 ## (1 ~2) > 12 ## 12 > (1 ~2) ## 1 > (12 ~2) ## 2 > (12 ~1) ## 12 > 1 > 2 ## 12 > 2 > 1 ## 1 > 12 > 2 ## 2 > 12 > 1 ## 1 > 2 > 12 ## 2 > 1 > 12
Permutations over power relations can be split into two parts:

1. generating partitions, or, generating differently sized equivalence classes, and 2. moving coalitions between these partitions.

In the code example above, we started with a single partition of size three, wherein all coalitions are considered equally preferable. By the end, we have reached the maximum number of partitions, where each coalition is put inside an equivalence class of size 1.

The partition generation can be reversed, such that we first receive linear power relations.

gen <-powerRelationGenerator(coalitions, startWithLinearOrder = TRUE) while(!is.null(pr <gen()

)) { print(pr) } ## 12 > 1 > 2
Notice that the "moving coalitions" part was not reversed, only the order the partitions come in.

Similarly, we are also able to skip the current partition. Note: the number of possible power relations grows tremendously fast as the number of coalitions rises. To get to that number, first consider how many ways n coalitions can be split into k partitions, also known as the Stirling number of second kind,

S(n, k) = 1 k! k j=0 (-1) j k j (k -j) n .
The number of all possible partitions given n coalitions is known as the Bell number (see also numbers::bell()),

B n = k j=0 S(n, k).
Given a set of coalitions P ∈ 2 N , the number of total preorders in T (P) is

|T (P)| = |P| k=0 k! * S(|P|, k)
# of coalitions # of partitions # of total preorders

SocialRanking Objects

The main goal of the socialranking package is to rank elements based on a given power ranking. More formally we try to map R : T (P) → T (N ), associating to each power relation ⪰∈ T (P) a total preorder R(⪰) (or R ⪰ ) over the elements of N .

In this context iR ⪰ j tells us that, given a power relation ⪰ and applying a social ranking solution R(⪰), i is ranked higher than or equal to j. From here on out, > and ~also denote the asymmetric and the symmetric part of a social ranking, respectively, i > j indicating that i is strictly better than j, whereas in i ~j, i is indifferent to j.

In literature, iI ⪰ j and iP ⪰ j are often used to denote the symmetric and asymmetric part, respectively. iI ⪰ j therefore means that iR ⪰ j and jR ⪰ i, whereas iP ⪰ j implies that iR ⪰ j but not jR ⪰ j.

In section 3.1 we show how a general SocialRanking object can be constructed using the doRanking function. In the following sections, we will introduce the notion of dominance [START_REF] Moretti | Some axiomatic and algorithmic perspectives on the social ranking problem[END_REF], cumulative dominance [START_REF] Moretti | An axiomatic approach to social ranking under coalitional power relations[END_REF] and CP-Majority comparison [START_REF] Haret | Ceteris paribus majority for social ranking[END_REF] that lets us compare two elements before diving into the social ranking solutions of the Ordinal Banzhaf Index [START_REF] Khani | An ordinal banzhaf index for social ranking[END_REF], Copeland-like and Kramer-Simpson-like methods [START_REF] Allouche | Social ranking manipulability for the CP-majority, banzhaf and lexicographic excellence solutions[END_REF], and lastly the Lexicographical Excellence Solution [START_REF] Algaba | Lexicographic solutions for coalitional rankings[END_REF] (Lexcel) and the Dual Lexicographical Excellence solution [START_REF] Serramia | On the dominant set selection problem and its application to value alignment[END_REF] (Dual Lexcel).

Example 2. Let {a, b} ≻ ({a, c} ∼ {b, c}) ≻ ({a} ∼ {c}) > ∅ ≻ {b} be a power ranking.

Using the following social ranking solutions, we get:

• a > b > c for lexcelRanking • a > c

Creating SocialRanking objects

A SocialRanking object represents a total preorder in T (N ) over the elements of N . Internally they are saved as a list of vectors, each containing players that are indifferent to one another. This is somewhat similar to the equivalenceClasses attribute in PowerRelation objects.

The function doRanking offers a generic way of creating SocialRanking objects. Given a sortable vector or list of scores it determines the power relation between all players, where the names of the elements are determined from the names() attribute of scores. Hence, a PowerRelation object is not necessary to create a SocialRanking object.

# we define some arbitrary score vector where "a" scores highest. # "b" and "c" both score 1, thus they are indifferent.

Comparison Functions

Comparison functions only compare two elements in a given power relation. They do not offer a social ranking solution. However in cases such as CP-Majority comparison, those comparison functions may be used to construct a social ranking solution in some particular cases.

Dominance

Definition 1. (Dominance [START_REF] Moretti | Some axiomatic and algorithmic perspectives on the social ranking problem[END_REF]) Given a power relation ⪰∈ T (P) and two elements i, j ∈ N , i dominates j in ⪰ if S ∪ {i} ⪰ S ∪ {j} for each S ∈ 2 N \{i,j} . i also strictly dominates j if there exists S ∈ 2 N \{i,j} such that S ∪ {i} ≻ S ∪ {j}.

The implication is that for every coalition i and j can join, i has at least the same positive impact as j.

The

Cumulative Dominance

When comparing two players i, j ∈ N , instead of looking at particular coalitions S ∈ 2 N \{i,j} they can join, we look at how many stronger coalitions they can form at each point. This property was originally introduced in [START_REF] Moretti | An axiomatic approach to social ranking under coalitional power relations[END_REF] as a regular dominance axiom.

For a given power relation ⪰∈ T (P) and its corresponding quotient order 1 ≻ • • • ≻ m , the power of a player i is given by a vector Score Cumul (i) ∈ N m where we cumulatively sum the amount of times i appears in k for each index k.

Definition 2. (Cumulative Dominance Score) Given a power relation ⪰∈ T (P) and its quotient order 1 ≻ • • • ≻ m , the cumulative score vector Score Cumul (i) ∈ N m of an element i ∈ N is given by:

Score Cumul (i) = k t=1 |{S ∈ t : i ∈ S}| k∈{1,...,m} (1)
Definition 3. (Cumulative Dominance) Given two elements i, j ∈ N , i cumulatively dominates j in ⪰, if Score Cumul (i) k ≥ Score Cumul (j) k for each k ∈ {1, . . . , m}. i also strictly cumulatively dominates j if there exists a k such that Score Cumul (i) k > Score Cumul (j) k .

For a given PowerRelation object pr and two elements e1 and e2, cumulativeScores(pr) returns the vectors described in definition 2 for each element, cumulativelyDominates(pr, e1, e2) returns TRUE or FALSE based on definition 3.

pr <as.PowerRelation("ab > (ac ~bc) > (a ~c) > {} > b") cumulativeScores(pr) [START_REF] Holler | Power, voting, and voting power: 30 years after[END_REF], therefore a also strictly dominates b cumulativelyDominates(pr, "a", "b", strictly = TRUE)

## $a ## [1] 1 2 3 3 3 ## ## $b ## [1] 1 2 2 2 3 ## ## $c ## [1] 0 2 3 3 3 ## ## attr(,"class") ## [1] "CumulativeScores" # for each index k, $a[k] >= $b[k] cumulativelyDominates(pr, "a", "b") ## [1] TRUE # $a[3] > $b
## [1] TRUE # $b[1] > $c[1], but $c[3] > $b[3] # therefore

CP-Majority comparison

The Ceteris Paribus Majority (CP-Majority) relation is a somewhat relaxed version of the dominance property. Instead of checking if S ∪ {i} ⪰ S ∪ {j} for all S ∈ 2 N \{i,j} , the CP-Majority relation iR ⪰ CP j holds if the number of times S ∪ {i} ⪰ S ∪ {j} is greater than or equal to the number of times S ∪ {j} ⪰ S ∪ {i}.

Definition 4. (CP-Majority [6]) Let ⪰∈ T (P). The Ceteris Paribus majority relation is the binary relation R ⪰

CP ⊆ N × N such that for all i, j ∈ N :

iR ⪰ CP j ⇔ d ij (⪰) ≥ d ji (⪰) (2)
where d ij (⪰) represents the cardinality of the set D ij (⪰), the set of all coalitions S ∈ 2 N \{i,j} for which S ∪ {i} ⪰ S ∪ {j}.

cpMajorityComparisonScore(pr, e1, e2) calculates the two scores d ij (⪰) and -d ji (⪰ ). Notice the minus sign -that way we can use the sum of both values to determine the relation between e1 and e2.

pr <as.PowerRelation("ab > (ac ~bc) > (a ~c) > {} > b") cpMajorityComparisonScore(pr, "a", "b")

## [1] 2 -1 cpMajorityComparisonScore(pr, "b", "a") ## [1] 1 -2 if(sum(cpMajorityComparisonScore(pr, "a", "b")) >= 0) { print("a >= b") } else { print("b > a") } ## [1] "a >= b"
As a slight variation the logical parameter strictly calculates d ij (≻) and -d ji (≻), the number of coalitions S ∈ 2 N \{i,j} where S ∪ {i} ≻ S ∪ {j}.

# Now (ac ~bc) is not counted cpMajorityComparisonScore(pr, "a", "b", strictly = TRUE) ## [1] 1 0 # Notice that the sum is still the same sum(cpMajorityComparisonScore(pr, "a", "b", strictly = FALSE)) == sum(cpMajorityComparisonScore(pr, "a", "b", strictly = TRUE)) ## [START_REF] Shapley | A method for evaluating the distribution of power in a committee system[END_REF] TRUE Coincidentally, cpMajorityComparisonScore with strictly = TRUE can be used to determine if e1 (strictly) dominates e2. cpMajorityComparisonScore should be used for simple and quick calculations. The more comprehensive function cpMajorityComparison(pr, e1, e2) does the same calculations, but in the process retains more information about all the comparisons that might be interesting to a user, i.e., the set D ij (⪰) and D ji (⪰) as well as the relation iR ⪰ CP j. See the documentation for a full list of available data. The CP-Majority relation can generate cycles, which is the reason that it is not offered as a social ranking solution. Instead we will introduce the Copeland-like method and Kramer-Simpson-like method in chapters 3.3.2 and 3.3.3 that make use of the CP-Majority functions to determine a power relation between elements. For further readings on CP-Majority, see [START_REF] Fayard | Ordinal social ranking: Simulation for CP-majority rule[END_REF] and [START_REF] Allouche | Social ranking manipulability for the CP-majority, banzhaf and lexicographic excellence solutions[END_REF].

# extract more information in cpMajorityComparison

Social Ranking Solutions

Ordinal Banzhaf

The Ordinal Banzhaf Score is a vector defined by the principle of marginal contributions.

Intuitively speaking, if a player joining a coalition causes it to move up in the ranking, it can be interpreted as a positive contribution. On the contrary a negative contribution means that participating causes the coalition to go down in the ranking.

Definition 5. (Ordinal marginal contribution [START_REF] Khani | An ordinal banzhaf index for social ranking[END_REF]) Let ⪰∈ T (P). For a given element i ∈ N , its ordinal marginal contribution m S i (⪰) with right to a coalition S ∈ P is defined as:

m S i (⪰) =      1 if S ∪ {i} ≻ S -1 if S ≻ S ∪ {i} 0 otherwise (3) Definition 6. (Ordinal Banzhaf relation) Let ⪰∈ T (P). The Ordinal Banzhaf relation is the binary relation R ⪰ Banz ⊆ N × N such that for all i, j ∈ N : iR ⪰ Banz j ⇔ Score Banz (i) ≥ Score Banz (j), (4)
where Score Banz (i) = S m S i (⪰) for all S ∈ N \ {i}. Note that if S / ∈ P or S ∪ {i} / ∈ P, m S i (⪰) = 0. The function ordinalBanzhafScores() returns three numbers for each element, 1. the number of coalitions S where a player's contribution has a positive impact, 2. the number of coalitions S where a player's contribution has a negative impact, and 3. the number of coalitions S for which no information can be gathered, because S / ∈ P or S ∪ {i} / ∈ P.

The sum of the first two numbers determines the score of a player. Players with higher scores rank higher. # both players 1 and 2 have an Ordinal Banzhaf Score of 1 # therefore they are indifferent to one another # note that the empty set is missing, as such we cannot compare {}u{i} with {} ordinalBanzhafScores(pr)

## $`1# # [1] 1 0 1 ## ## $`2# # [1] 1 0 1 ## ##

Copeland-like method

The Copeland-like method of ranking elements based on the CP-Majority rule is strongly inspired by the Copeland score from social choice theory [START_REF] Copeland | A reasonable social welfare function[END_REF]. The score of an element i ∈ N is determined by the amount of the pairwise CP-Majority winning comparisons iR ⪰ CP j, minus the number of all losing comparisons jR ⪰ CP i against all other elements j ∈ N \ {i}.

Definition 7. (Copeland-like relation [10]) Let ⪰∈ T (P). The Copeland-like relation is the binary relation R ⪰

Cop ⊆ N × N such that for all i, j ∈ N :

iR ⪰ Cop j ⇔ Score Cop (i) ≥ Score Cop (j), (5)
where Score

Cop (i) = |{j ∈ N \ {i} : d ij (⪰) ≥ d ji (⪰)}| -|{j ∈ N \ {i} : d ij (⪰) ≤ d ji (⪰)}|
copelandScores(pr) returns two numerical values for each element, a positive number for the winning comparisons (shown in Score Cop (i) on the left) and a negative number for the losing comparisons (in Score Cop (i) on the right). Strongly inspired by the Kramer-Simpson method of social choice theory [START_REF] Simpson | On defining areas of voter choice: Professor tullock on stable voting[END_REF][START_REF] Kramer | A dynamical model of political equilibrium[END_REF], elements are ranked inversely to their greatest pairwise defeat over all possible CP-Majority comparisons.

Definition 8. (Kramer-Simpson-like relation [START_REF] Allouche | Social ranking manipulability for the CP-majority, banzhaf and lexicographic excellence solutions[END_REF]) Let ⪰∈ T (P). The Kramer-Simpsonlike relation is the binary relation R ⪰ KS ⊆ N × N such that for all i, j ∈ N :

iR ⪰ KS j ⇔ Score KS (i) ≤ Score KS (j), (6)
where Score KS (i) = max j d ji (⪰) for all j ∈ N \ {i}.

kramerSimpsonScores(pr) returns a single numerical value for each element, which sorted lowest to highest gives us the ranking solution.

pr <as.PowerRelation("(abc ~ab ~c ~a) > (b ~bc) > ac") unlist(kramerSimpsonScores(pr)) ## a b c ## 0 0 1

kramerSimpsonRanking(pr) ## a ~b > c

There is a small caveat to Definition 8. By default this function does not compare d ii (⪰).

In other terms, the score of every element is the maximum CP-Majority comparison score against all other elements. This is slightly different from the definition found in [START_REF] Allouche | Social ranking manipulability for the CP-majority, banzhaf and lexicographic excellence solutions[END_REF], where the CP-Majority comparison d ii (⪰) is also considered. Since by definition d ii (⪰) = 0, the Kramer-Simpson scores in those cases will never be negative, possibly discarding valuable information.

To still account for the original definition in [START_REF] Allouche | Social ranking manipulability for the CP-majority, banzhaf and lexicographic excellence solutions[END_REF], the functions kramerSimpsonScores and kramerSimpsonRanking offer a compIvsI parameter that can be set to TRUE if one wishes for d ii (⪰) to be included in the comparisons.

Lexicographical Excellence Solution

The idea behind the lexicographical excellence solution (Lexcel) is to reward elements appearing more frequently in higher ranked equivalence classes.

For a given power relation ⪰ and its quotient order 1 ≻ • • • ≻ m , we denote by i k the number of coalitions in k containing i:

i k = |{S ∈ k : i ∈ S}| (7)
for k ∈ {1, . . . , m}. Now, let Score Lex (i) be the m-dimensional vector Score Lex (i) = (i 1 , . . . , i m ) associated to ⪰. Consider the lexicographic order ≥ Lex among vectors i and j: i ≥ Lex j if either i = j or there exists t : i r = j r , r ∈ {1, . . . , t -1}, and i t > j t .

Definition 9. (Lexicographic-Excellence relation [START_REF] Bernardi | Ranking objects from a preference relation over their subsets[END_REF]) Let ⪰∈ T (P) with its corresponding quotient order

1 ≻ • • • ≻ m . The Lexicographic-Excellence relation is the binary relation R ⪰ Lex ⊆ N × N such that for all i, j ∈ N : iR ⪰ Lex j ⇔ Score Lex (i) ≥ Lex Score Lex (j) (8)
pr <as.PowerRelation("12 > (123 ~23 ~3) > (1 ~2) > 13")

# show the number of times an element appears in each equivalence class # e.g. 3 appears 3 times in [ [START_REF] Banzhaf | Weighted voting doesn't work: A mathematical analysis[END_REF]] and 1 time in [[4]] lapply(pr$equivalenceClasses, unlist) ## list() lexScores <-lexcelScores(pr) for(i in names(lexScores)) paste0("Lexcel score of element ", i, ": ", lexScores[i])

# at index 1, element 2 ranks higher than 3 lexScores

['2'] > lexScores['3'] ## [1] TRUE # at index 2, element 2 ranks higher than 1 lexScores['2'] > lexScores['1'] ## [1] TRUE lexcelRanking(pr) ## 2 > 1 > 3
For some generalizations of the Lexcel solution see also [START_REF] Algaba | Lexicographic solutions for coalitional rankings[END_REF].

Lexcel score vectors are very similar to the cumulative score vectors (3.2.2) in that the number of times an element appears in a given equivalence class is of interest. In fact, applying the base function cumsum on an element's lexcel score gives us its cumulative score.

lexcelCumulated <lapply(lexScores, cumsum) cumulScores <-cumulativeScores(pr) paste0(names(lexcelCumulated), ": ", lexcelCumulated, collapse = ', ') ## [1] "1: 1:4, 2: c(1, 3, 4, 4), 3: c(0, 3, 3, 4)" paste0(names(cumulScores), ": ", cumulScores, collapse = ', ') ## [1] "1: 1:4, 2: c(1, 3, 4, 4), 3: c(0, 3, 3, 4)"

Dual Lexicographical Excellence Solution

Similar to the Lexcel ranking, the Dual Lexcel also uses the Lexcel score vectors from definition 9 to establish a ranking. However, instead of rewarding higher frequencies in high ranking coalitions, it punishes higher frequencies in lower ranking coalitions, or, it punishes mediocrity [START_REF] Serramia | On the dominant set selection problem and its application to value alignment[END_REF].

Take the values i k for k ∈ {1, . . . , m} and the Lexcel score vector Score Lex (i) from section 3.3.4. Consider the dual lexicographical order ≥ DualLex among vectors i and j: i ≥ DualLex j if either i = j or there exists t : i t < j t and i r = j r , r ∈ {t + 1, . . . , m}.

Definition 10. (Dual Lexicographical-Excellence relation [START_REF] Serramia | On the dominant set selection problem and its application to value alignment[END_REF]) Let ⪰∈ T (P). The Dual Lexicographic-Excellence relation is the binary relation R ⪰ DualLex ⊆ N × N such that for all i, j ∈ N :

iR ⪰ DualLex j ⇔ Score Lex (i) ≥ DualLex Score Lex (j) (9)
The S3 class LexcelScores does not account for Dual Lexcel comparisons. Instead -rev(x) is called on a Lexcel score vector x such that the resulting comparisons produces a Dual Lexcel ranking solution.

Incidence Matrix

In our vignette we focused more on the intuitive aspects of power relations and social ranking solutions. To reiterate, a power relation is a total preorder, or a reflexive and transitive relation ⪰∈ P × P, where ∼ denotes the symmetric part and ≻ its asymmetric part.

A power relation can be viewed as an incidence matrix

(b ij ) = B ∈ {0, 1} |P|×|P| . Given two coalitions i, j ∈ P, if iRj then b ij = 1, else 0.
With help of the relations package, the functions relations::as.relation(pr) and powerRelationMatrix(pr) turn a PowerRelation object into a relation object.

relations then offers ways to display the relation object as an incidence matrix with relation_incidence(rel) and to test basic properties such relation_is_linear_order(rel), relation_is_acyclic(rel) and relation_is_antisymmetric(rel) (see relations package for more [START_REF] Meyer | Relations: Data structures and algorithms for relations[END_REF]).

Cycles and Transitive Closure

A cycle in a power relation exists, if there is one coalition S ∈ 2 N that appears twice. For example, in {1, 2} ≻ ({1} ∼ ∅) ≻ {1, 2}, the coalition {1, 2} appears at the beginning and at the end of the power relation.

Properly handling power relations and calculating social ranking solutions with cycles is somewhat ill-defined, hence a warning message is shown as soon as one is created.

as.PowerRelation("12 > 2 > (1 ~2) > 12") #! Warning in createLookupTables(equivalenceClasses): Found 2 duplicate coalitions, listed below. This violates transitivity and can cause issues with certain ranking solutions. You may want to take a look at socialranking::transitiveClosure(). #! -{2} #! -{1, 2} ## 12 > 2 > (1 ~2) > 12

Recall that a power relation is transitive, meaning for three coalitions x, y, z ∈ 2 N , if xRy and yRz, then xRz. If we introduce cycles, we pretty much introduce symmetry. Assume we have the power relation x ≻ y ≻ x. Then, even though xRy and yRx are defined as the asymmetric part of the power relation ⪰, together they form the symmetric power relation x ∼ y. transitiveClosure(pr) is a function that turns a power relation with cycles into one without one. In the process of removing duplicate coalitions, it turns all asymmectric relations within a cycle into symmetric relations.

  ("ab > a ~{} > b") ## ab > (a ~{}) > b as.PowerRelation(list(c(1,2), 1, c(), 2)) ## 12 > 1 > {} > 2 as.PowerRelation(list(c(1,2), 1, c(), 2), comparators = c(">", "~", ">")) ## 12 > (1 ~{}) > 2

  as.PowerRelation("ab > abc ~ac ~bc > a ~c > {} > b") # a dominates b, but b does not dominate a c(dominates(pr, "a", "b"), dominates(pr, "b", "a")) ## [1] TRUE FALSE # calculate cumulative scores scores <-cumulativeScores(pr) # show score of element a scores$

  rel <-relations::as.relation(pr) rel ## A binary relation of size 8 x 8. relations::relation_incidence(rel) ## Incidences: ## ab abc ac bc a c {} b

  in createLookupTables(equivalenceClasses): Found 1 coalition that contain elements more than once.

  pr <as.PowerRelation("12 > (1 ~{}) > 2") PowerRelation(pr$eqs[c(2, 3, 1)]) ## (1 ~{}) > 2 > 12

  pr <-PowerRelation(list( list(c("AT", "DE"), "FR"), list("DE"), list(c("AT", "FR"), "AT") )) pr ## ({AT, DE} ~{FR}) > {DE} > ({AT, FR} ~{AT}) # since we have 3 elements, the super set 2ˆN should include 8 coalitions appendMissingCoalitions(pr) ## ({AT, DE} ~{FR}) > {DE} > ({AT, FR} ~{AT}) > ({AT, DE, FR} {DE, FR} ~{})

  pr <as.PowerRelation("a > b > c ~ac > abc") makePowerRelationMonotonic(pr) ## (abc ~ab ~ac ~a) > (bc ~b) > c makePowerRelationMonotonic(pr, addMissingCoalitions = FALSE) ## (abc ~ac ~a) > b > c # notice how an empty coalition in some equivalence class # causes all remaining coalitions to be moved there makePowerRelationMonotonic(as.PowerRelation("ab > c > {} > abc > a > b")) ## (abc ~ab) > (ac ~bc ~c) > (a ~b ~{})

Figure 1 :

 1 Figure 1: Using Alt+Up or Alt+Down to move one or more lines of code By default, createPowerset() returns the power set in the form of a list. This list can be passed directly to as.PowerRelation() to create a linear order.

  ps

  gen <-powerRelationGenerator(coalitions) # partition 3 gen <-generateNextPartition(gen) # partition 2+1 gen <-generateNextPartition(gen) # partition 1+2 gen() ## 12 > (1 ~2)

  > b for dualLexcelRanking • a > b ~c for copelandRanking and kramerSimpsonRanking • a ~c > b for ordinalBanzhafRanking

  scores <c(a = 100, b = 1, c = 1) doRanking(scores) ## a > b ~c # we can also tell doRanking to punish higher scores doRanking(scores, decreasing = FALSE) ## b ~c > aWhen working with types that cannot be sorted (i.e., lists), a function can be passed to the compare parameter that allows comparisons between arbitrary elements. This function must take two parameters (i.e., a and b) and return a numeric value based on the comparison:• compare(a,b) > 0: a scores higher than b,• compare(a,b) < 0: a scores lower than b,• compare(a,b) == 0: a and b are equivalent. scores <list(a = c(3, 3, 3), b = c(2, 3, 2), c = c(7, 0, 2)) doRanking(scores, compare = function(a, b) sum(a) -sum(b)) ## a ~c > b # a and c are considered to be indifferent, because their sums are the same doRanking(scores, compare = function(a,b) sum(a) -sum(b), decreasing = FALSE) ## b > a ~c

  neither b nor c dominate each other cumulativelyDominates(pr, "b", "c") ## [1] FALSE cumulativelyDominates(pr, "c", "b") ## [1] FALSE Similar to the dominance property from our previous section, two elements not dominating one or the other does not indicate that they are indifferent.

1 #

 1 cpMajorityComparison(pr, "a", "b") ## a > b ## D_ab = {c, {}} ## D_ba = {c} ## Score of a = 2 ## Score of b = with strictly set to TRUE, coalition c does # neither appear in D_ab nor in D_ba cpMajorityComparison(pr, "a", "b", strictly = TRUE) ## a > b ## D_ab = {{}} ## D_ba = {} ## Score of a = 1 ## Score of b = 0

  as.PowerRelation("ab > a > {} > b") # player b has a negative impact on the empty set # -> player b's score is 1 -1 = 0 # -> player a's score is 2 -0 = 2 sapply(ordinalBanzhafScores(pr), function(score) sum(score[c(1

  pr <as.PowerRelation("(abc ~ab ~c ~a) > (b ~bc) > ac") scores <-copelandScores(pr) # Based on CP-Majority, a>=b and a>=c (+2), but b>=a(-1)

  pr <as.PowerRelation("b > (a ~c) > ab > (ac ~bc) > {} > abc") kramerSimpsonRanking(pr) ## b > a > c # notice how b's score is negative unlist

  pr <as.PowerRelation("12 > (123 ~23 ~3) > (1 ~2) > 13") lexScores <-lexcelScores(pr) # in regular Lexcel, 1 scores higher than 3 lexScores['1'] > lexScores['3'] ## [1] TRUE # turn Lexcel score into Dual Lexcel score dualLexScores <structure( lapply(lexcelScores(pr), function(r) -rev(r)), class = 'LexcelScores' ) # now 1 scores lower than 3 dualLexScores['1'] > dualLexScores['3'] ## [1] FALSE # element 2 comes out at the top in both Lexcel and Dual Lexcel lexcelRanking

  pr <as.PowerRelation("ab > a > {} > b") rel <-relations::as.relation(pr) relations::relation_incidence(rel) #relation_is_acyclic(rel), relations::relation_is_antisymmetric(rel), relations::relation_is_linear_order(rel), relations::relation_is_complete(rel), relations::relation_is_reflexive(rel), relations::relation_is_transitive(rel) ) ## [1] TRUE TRUE TRUE TRUE TRUE TRUE Note that the columns and rows are sorted by their names in relation_domain(rel), hence why each name is preceded by the ordering number. # a power relation where coalitions {1} and {2} are indifferent pr <as.PowerRelation("12 > (1 ~2)") rel <-relations::as.relation(pr) # we have both binary relations {1}R{2} as well as {2}R{1} relations::relation_incidence(rel) #relation_is_acyclic(rel), relations::relation_is_antisymmetric(rel), relations::relation_is_linear_order(rel), relations::relation_is_complete(rel), relations::relation_is_reflexive(rel), relations::relation_is_transitive(rel) ) ## [1] FALSE FALSE FALSE TRUE TRUE TRUE

  pr <-suppressWarnings( as.PowerRelation("c > ac > b > ac > (a ~b) > abc") ) transitiveClosure(pr) ## c > (ac ~b ~a) > abc

library(socialranking) pr <-PowerRelation(list( list(c(1,2)), list(2, c()), list(1)

  1 and how a correlating PowerRelation object can be constructed.

	))
	pr
	## 12 > (2 ~{}) > 1
	class(pr)
	## [1] "PowerRelation"	"SingleCharElements"
	Notice how coalitions such as {1, 2} are written as 12 to improve readability. Similarly,
	passing a string to the function as.PowerRelation() saves some typing on the user's
	end by interpreting each character of a coalition as a separate element. Note that spaces
	in that function are ignored.
	as.PowerRelation("12 > 2~{} > 1")
	## 12 > (2 ~{}) > 1
	The compact notation is only done in PowerRelation objects where every element is one
	digit or one character long. If this is not the case, curly braces and commas are added
	where needed.
	prLong <-PowerRelation(list(
	list(c("Alice", "Bob")),
	list("

Bob", c()),

	Attribute	Description	Value in pr
	elements	Sorted vector of elements	c(1,2)
		List containing lists, each	list(list(c(1,2)),
	eqs	containing coalitions in the	list(c(2), c()),
		same equivalence class	list(c(1)))
	coalitionLookup	Function to determine a coalition's equivalence class index	function(coalition)
	elementLookup	Function to determine, which coalitions an element takes part in	function(element)
	list("Alice")
	))
	prLong
	## {Alice, Bob} > ({Bob} ~{}) > {Alice}
	class(prLong)
	## [1] "PowerRelation"

  For any S ∈ 2 N \{i,j} , we can only compare S ∪ {i} ⪰ S ∪ {j} if both S ∪ {i} and S ∪ {j} take part in the power relation.Additionally, for S = ∅, we also want to compare {i} ⪰ {j}. In some situations however a comparison between singletons is not desired. For this reason the parameter includeEmptySet can be set to FALSE such that ∅ ∪ {i} ⪰ ∅ ∪ {j} is not considered in the CP-Majority comparison.

	# an element i dominates itself, but it does not strictly dominate itself
	# because there is no Sui > Sui
	dominates(pr, 1, 1)
	## [1] TRUE
	dominates(pr, 1, 1, strictly = TRUE)
	## [1] FALSE
	# 1 clearly dominates 2
	dominates(pr, 1, 2)
	## [1] TRUE
	dominates(pr, 2, 1)
	## [1] FALSE
	# 3 does not dominate 1, nor does 1 dominate 3, because
	# {}u3 > {}u1, but 2u1 > 2u3
	dominates(pr, 1, 3)
	## [1] FALSE
	dominates(pr, 3, 1)
	## [1] FALSE

function dominates(pr, e1, e2) only returns a logical value TRUE if e1 dominates e2, else FALSE. Note that e1 not dominating e2 does not indicate that e2 dominates e1, nor does it imply that e1 is indifferent to e2. pr <as.PowerRelation("3 > 1 > 2 > 12 > 13 > 23") pr <as.PowerRelation("ac > bc ~b > a ~abc > ab") # FALSE because ac > bc, whereas b > a dominates(pr, "a", "b") ## [1] FALSE # TRUE because ac > bc, ignoring b > a comparison dominates(pr, "a", "b", includeEmptySet = FALSE) ## [1] TRUE


Aurélie Mahalatchimy

Auxane Delage

Luc-Sylvain Gilbert

Hsin-Yu Kuo

Anna Couturier

Jennifer Lorigan

Clare Blackburn

Research community hub, a report on dissemination and uptake of information produced in WP4 via the EuroGCT online research and regulatory community hub

List of acronyms ATMP: Advanced Therapy Medicinal Products

EuroGCT: The European Consortium for Communicating Gene and Cell Therapy Information IMHT: The EuroGCT Information Management Hub Team, comprised of project staff employed directly by the project, based at the University of Edinburgh (UEDIN) and Aix-Marseille University (AMU) WP: Work Package D2.2 Report on dissemination and uptake of information produced in WP4 via the EuroGCT online research and regulatory community hub EuroGCT WP4 develops an information hub for the research and regulatory communities that facilitates access of these communities to information on the practical steps needed for cell and gene-based therapy development and promotes information exchange within and between these communities. This tailored section of www.eurogct.org aims to create a 'one-stop shop' on where to seek further information and guidance relating to manufacturing guidelines, regulatory requirements, intellectual property rights, market acceptability and ethical matters related to advanced therapies development.

Report D2.2 summarises the activities performed regarding the dissemination and uptake of information produced in WP4 between month 12 (since the launch of the new website) and month 30.

Establishment and updating of the Research Pathways

Hub, an interactive database

The EuroGCT online research and regulatory community hub, named 'Research Pathways', contains a set of information resources on the practical steps needed for cell and gene therapy development. This resource set can be viewed online at www.eurogct.org/research-pathways, and is outlined in Deliverable 4.2 -Resources on practical steps, and detailed in D4.1 -Landscape mapping.

The Research Pathways hub contains information on practical steps needed for cell and gene therapy development, and includes information resources developed by the EuroGCT consortium plus signposting to external information resources. The current database is built on the initial information published by this project (see deliverable D4.9), new content developed by EuroGCT during the current reporting period, and signposting to additional collated resources. Content within Research Pathways will be continuously updated throughout the project duration.

In the last reporting period, the EuroGCT IMHT carried out a review of user experience of the Research Pathways Hub. Based on the outcome of the review, in months 18-30 we have updated the section's design from a hierarchical structure to a database model, which gives improved searchability and usability. The improvement is further enhanced by carefully designed taxonomies used for tagging, new search and filter functions, which also link to grouped topical sections outlined as follows: Users can freely interact with the database by choosing to narrow down their searches with keywords or filters. They can also utilise the Research Pathway as a gateway to the topic-based sections and to look for resources within. We anticipate the improved searchability and usability resulting from the above work will greatly enhance the user's experience in search and navigation within the section, and in turn will improve engagement.

Summary of Dissemination Activities

Besides establishing the EuroGCT website and the 'Research Pathways' section within it, the EuroGCT Consortium has actively disseminated outputs of the project via event attendance, presentations (online and in-person), posters, and networking activities.

Because the content in the Research Pathways section is tailored toward experts, we have not prioritised dissemination through social media channels in WP4. Instead, we have focused on reaching WP4 target audiences directly in meetings and networking events for dissemination and engagement activities. The activities performed between months 12 and 30 are summarised below:

Events

Uptake

The uptake of information produced in WP4 via the EuroGCT online research and regulatory community hub can be demonstrated by inquiries we received on collaborations, collaborations in discussion or formed, and website traffic stats.

3.1

Collaboration formed and/or currently in discussion:

1) GetRadi -The GetRadi

Collaboration inquiries received:

1) COST Action HaploiPS (Generation of Human Induced Pluripotent Stem Cells From Haplo-selected Cord Blood Samples) 2) Collaboration with the Human Gene Therapy Journal on providing lay abstract of key papers published in this journal

Website traffic

Complementary to direct engagement, the establishment of the EuroGCT online research and regulatory community hub, Research Pathways, broadens our dissemination actions and makes these outputs accessible to everyone, independent of their geographical location.

We note that the Research Pathways section produced by WP4 contains information that is mainly targeted toward experts and to a less extent, informed non-experts. The expert groups represent a small fraction of the total online population, and the likelihood of attracting online traffic that is at a comparable scale to our public-facing counterpart is low. Thus, reporting merely on the absolute visitor count is not suitable for evaluating the information uptake among these groups and the resulting impact. In this section, we discuss and report the use of additional metrics to better assess our digital outreach, as well as information uptake by our target audience.

EuroGCT's effort on tracking webpage traffic is described in detail in D2.4. Briefly, we have made a conscious effort to focus on strict user privacy considerations, which results in challenges for reporting. For reporting uptake for WP4 content, we track passive user engagement using Fathom Analytics and Google Search Console. We will report webpage traffic using data drawn from Fathom Analytics; we will report on search rankings and top query terms using data drawn from Google Search Console. Based on Fathom Analytics, the top accessed WP4 pages and their unique visitor counts, and total views between website launch and 15 July 2023 are summarised in the The entry highlighted in grey is the main EuroGCT landing page, included here for benchmarking purpose. The three entries highlighted in blue refer to the same suite of information, the ATMP Engage Public and Patient Involvement Directory. The two entries highlighted in green are EuroGCT original resource pages.

Below we report and discuss features and user characteristics captured for three major types of pages in the Research Pathways information hub: 1) directory search pages, 2) topical pages, and 3) individual resource page.

1) Directory search pages, including Research Pathways and ATMP Engage PPI Directory:

The pages containing directories received total views that are 2-6 times higher than their unique visitor number. This is distinct from the public-facing resource pages (WP3) where the views and unique visitor number are near 1:1 (see D2.4). This observation suggests that these directory pages have started to be used as a reference page for certain user groups as a standing resource for regular re-visit and to look up resources. This suggestion is further supported by Fathom Analytic's referrer record for these WP4 pages being largely Direct/Unknown and not redirected from search engines (see Figure 1 and2). A referrer is the site a visitor was previously on before they were redirected to our site. The EuroGCT site's average bounce rate, defined as the percentage of visitors who enter the site and then leave, is 74%. A significantly lower bounce rate (36% for 'Research Pathways' and 54% for 'ATMP Engage PPI Directory') is recorded for these directory search pages (see Figure 1 and 2). This translates to 64% and 46% of users respectively, who remain on our site and continue to browse content on our website after landing at this page. 2) Topical pages:

Topical pages such as the 'Key Pathways Resources' page contain handpicked key resources on ATMP development and categorised by themes. For this type of page, the referrers are about 1:1 for search engines and Direct/unknown. A slightly lower bounce rate (62%) is recorded in Fathom Analytics (see Figure 3) indicating 38% of users remain on the site and continue to browse content. 3) Individual resources, including in-house resources and external resources:

Resources including 'Packaging and labelling' and 'Manufacturing authorisation' developed by the EuroGCT consortium are mostly accessed through organic searches as shown in Fathom Analytics (see Figure 4 for the stats of the 'Manufacturing authorisation' page as an example).

These pages appear in top 20 search results in Google using relevant query terms, and more often, are ranked within top 5 using exact search terms. (see Figure 5 for the stats of the 'Manufacturing authorisation' page as an example). The click-through rate (CTR) for these top-ranked search results are also significantly higher. For the case of 'Manufacturing authorisation' page, the CTR can often reach 10% when similar query terms are used in searches (see Figure 5). The CTR for this page averages 4.3%, significantly higher compared to the 1% for the entire EuroGCT site.

We have also recorded a significantly longer retention time for these EuroGCT original resource pages. For the 'Packaging and Labelling' and 'Manufacturing authorisation' and page, the average retention time is 4:10 and 4:13 minutes (see Figure 4), respectively. This is 2.5 times longer than the averaged 1:41 minutes for the entire site, indicating users are really reading and learning from these resources we created and published. In D2.4, we have reported our observation on a distinct user profile for WP3 and WP4 content. Here we report that the indicators that reflect information uptake vary depending on the nature of the webpage: for directory search pages, the total views to visitor ratio tends to be higher and bounce rate is significantly lower; for individual resource pages the ratio is closer to 1:1 with a bounce rate representative of the site's bounce rate; topical pages sit in between the two types. We have also recorded the significantly longer user retention time for original resources produced by the EuroGCT consortium. We will continue to monitor website information uptake, and use these data to shape future reporting metrics and audience engagement strategies.

Research Pathways has a worldwide reach. Figure 6 shows that the Research Pathways pages have been accessed by users from countries beyond EU and UK. These countries include the US, Taiwan, India and China. This demonstrates that while the information regarding medicine legislation, regulation, and access is EU-specific, other Research Pathways resources have been utilised by communities outside of our target audience and can be useful irrespective of their geographical location.

Conclusion and future work

Activities on disseminating WP4 outputs include establishing and updating the EuroGCT website, event attendance, presentations (both online and in-person), posters, and networking activities. The Research Pathways section, the output of WP4, contains information mainly targeted toward experts. We have mainly focussed on in-person and online events that allows direct interaction with these WP4 target audiences. Our direct engagement at meetings and networking events is effective and has resulted in a number of inquiries, collaborations in discussion and formed.

Complementary to direct engagement, the establishment of the EuroGCT online research and regulatory community hub, Research Pathways, has broadened our dissemination actions and made this work accessible to audience in the EU, UK, and worldwide. It also serves audiences beyond our targeted expert audience groups, giving an impact that will be beyond traditional academic publishing. We anticipate the improved search-ability and usability in the Research Pathways will greatly enhance user's experience and their learning when reading the resources within the section. The new content type 'Case Studies' could attract interest and opportunities for collaboration, from both inside and outside of the consortium.

As for website traffic, reporting merely on the absolute visitor count is not suitable for evaluating the information uptake among the expert groups we are targeting. We need additional means and metrics including but not limited to, the user retention time, bounce rate, and the views to visitor ratio to reflect the information uptake. We will continue to monitor and evaluate these metrics, then use this information to feed into future reporting as well as to shape our audience engagement strategies. In an effort to help stakeholders to overcome the legal and scientific challenges raised during the development of a medicinal product, procedures, such as scientific advice, have been established in the European Union to enable early contact with regulators, especially in the field of gene and cell therapy. These procedures help to support innovation. They are various: some are available for all medicines and others target innovative medicines or Advanced Therapy Medicinal Products (ATMPs); some are legislative or regulatory schemes, others put into place services; some benefit to all types of medicines' developer, others target a specific type of developer. These procedures for early interactions between stakeholders and regulators are available to facilitate communication between institutions and medicine developers all along the medicine's lifecycle, but the main focus is on the pre-marketing authorisation phase. The exchanges occurring between regulators and the seeker of interaction can impact the chances an ATMP has to enter on the market. Thus, the aim is to foster their development by helping developers to navigate the regulatory pathways adequately for ATMPs to meet quality, safety, and efficacy requirements according to robust evidence to obtain marketing authorisation and access to patients. Knowing all the procedures, who can ask for them, which procedure to use, and with which regulator can be a challenge. Moreover, the extent of the possible support is often overlooked. This poster describes and discusses the multiple existing options available to ATMPs' developers to overcome challenges during the pre-marketing authorisation phase through early interactions with regulators, here focusing on the European Medicines Agency and National agencies. Communicating on the early procedures available at national and European levels will help to identify potential gaps and raise developers' awareness of the different procedures available to support the marketing authorisation of ATMPs.

List of Annexes

8 )

 8 Attracted one law assistant professor from the Netherlands to come for a 3months research stay at AMU to learn about ATMP regulations and to provide support on the organisation of a EuroGCT workshop at EMA. 9) Contacts and proposals for collaborations with the Alliance for Regenerative Medicine (ARM) and the International Society for Cell & Gene Therapy (ISCT) established after poster presentations and roundtable discussions at ISCT 2023. 10) Established an online collaboration: A. Mahalatchimy is working on an ATMP regulation entry for an online Encyclopaedia about "European Union Law" edited by Oxford University Press. 11) Received request to co-develop resources-an ATMP development Case Study on Libmeldy. EuroGCT WP4 is currently working together with Orchard Therapeutics on co-developing a case study an EMA-approved Gene therapy medicinal product (GTMP), Libmeldy, for the researcher and developer community to learn about its development process. 12) ISSCR Education Committee: EuroGCT has a long standing position on the ISSCR's Education Committee. This includes providing editorial oversight of the ISSCR Patient Handbook (A. Couturier, C. Blackburn). In November 2023, EuroGCT will run a communications seminar for all ISSCR members. Additionally, EuroGCT is planned to be involved in the public engagement and regulatory outreach for the ISSCR Annual Conference to be held in Hamburg, 2024. 13) ESGCT: EuroGCT has built collaborations with ESGCT to expand its support of European Gene and Cell Therapy researchers. This includes leading public engagement and regulatory support for the ESGCT Annual Congress -2023 (Brussels) and collaborating with ESGCT on a heatmap of gene and cell therapy centres.

Figure 1 .

 1 Figure 1. Webpage stats for the 'Research Pathways' directory search page. Main referrers are not search engines. A low bounce rate (36%) is recorded. Reported from Fathom Analytics between website launch and 15 July 2023.

Figure 2 .

 2 Figure 2. Webpage stats for the 'ATMP Engage PPI Directory' search page. Main referrers are not search engines. A low bounce rate (54%) is recorded. Reported from Fathom Analytics between website launch and 15 July 2023.

Figure 3 .

 3 Figure 3. Webpage stats for the 'Key Pathways Resources' topical page containing handpicked key resources on ATMP development. Referrers are about 1:1 for search engines and Direct/unknown. A lower bounce rate (62%) is recorded. Record in Fathom Analytics between website launch and 15 July 2023.

Figure 4 .

 4 Figure 4. Webpage stats for the 'Manufacturing Authorisation' page. Main referrers is Google. Average time on site is 4:13 minutes. Record in Fathom Analytics between website launch and 15 July 2023.

Figure 5 .

 5 Figure 5. Webpage stats (top) for 'Manufacturing Authorisation' page and the list of top query terms (bottom). Recorded by Google console between website launch and 15 July 2023.

Figure 6 .

 6 Figure 6. 'Research Pathways' page accesses listed by country recorded by Fathom analytics between website launch and 15 July

Annex 1 .

 1 Abstracts and posters list from February 2022 to July 2023 EuroGCT Annex 1. to Deliverable D.2.2 Abstracts and posters list from February 2022 to July 2023 Abstract published in Human Gene Therapy, Volume: 33, Issue 23-24: December 14, 2022, A1-A212. http://doi.org/10.1089/hum.2022.29225.abstracts Poster submitted to the International Society for Cell & Gene Therapy 2023, Annual meeting May 31-June 3, Paris, France Title: Mapping regulators' early interactions procedures to support innovation Auxane Delage 6 , Luc-Sylvain Gilbert, 7 Aurélie Mahalatchimy 8

   Key Research Pathways: this section contains the collection of resources developed by EuroGCT WP4.  Key Pathway Resources: this section contains collated key resources on ATMP development by medicines agencies, and existing complementary work produced by other organisations.  Case Studies: this new section contains case studies on ATMP development, and provides storylines for readers to walk through the resources in the Research Pathways database.

 Patient and Public Involvement: this links to the ATMP Engage Patient and Public Involvement (PPI) Directory, a searchable and filterable directory with PPI resources for developers of cell and gene therapies.

 Actors and Networks: this section contains our initial mapping of stakeholders in ATMP development in Europe and beyond.

Table 1 .

 1 List of EuroGCT attended events. Entries highlighted in blue are those at which we presented posters or gave an oral presentation. Entries highlighted in green are events hosted by EuroGCT/University of Edinburgh. Working together as the EuroGCT partnership European Society for Gene & Cell Therapy Congress 2022, 11-14 October 2022, Edinburgh, UK A. DELAGE, L.-S. GILBERT, H.-Y. KUO, J. LORIGAN, C. BLACKBURN, A. COUTURIER, A. WAITE, A. MAHALATCHIMY, Bringing gene and cell therapies from lab to patients: the EuroGCT research pathways 15th European Public Health Conference, 9-12 November 2022, Berlin, Germany Oral communication: A. MAHALATCHIMY, The pandemic Treaty from an EU law perspective, roundtable on "Access, equity and the relevance of the Pandemic Treaty: lessons from the COVID-19 response" organized by Elena PETELOS EU Health Governance Research Network seminar series, Panel discussion / webinar on "EU

	Date / Place	Host	Theme
	28-29 July 2022, 2022	American Institute of Chemical	Synthetic biology research,
	International Mammalian	Engineers (AIChE)	emphasizing both basic and applied
	Synthetic Biology		science.
	Workshop, Edinburgh, UK
	10 Oct 2022, Edinburgh, UK University of Edinburgh	EuroGCT Sharing best practice
			workshop
	11-14 October 2022,	European Society of Gene and	Gene and Cell Therapy -oral
	ESGCT 29 th Annual	Cell Therapy in collaboration with	presentation and poster presentations
	Congress, Edinburgh, UK	British Society for Gene and Cell
		Therapy	A EuroGCT kiosk: for in-person
			engagement with conference
			attendees, raising awareness about
			the project recruiting Network of
			Expertise members and translators.
	31 October 2022,	Centre for Genomics and	Recent advancements in gene therapy
	CGEM Away Day,	Experimental Medicine (CGEM),	research
	Edinburgh, UK	The University of Edinburgh
	07-08 Nov 2022, London,	University College London	Regulatory Science for Advanced
	UK		(Gene and Cell) Therapies short
			courses

Table 1 lists all events EuroGCT has attended during this reporting period. Entries highlighted in blue are those at which we presented posters or gave an oral presentation. Entries highlighted in green are events hosted by EuroGCT/University of Edinburgh. Please see D2.2 Annex for the posters and abstracts submitted within this reporting period. Event Oral communication / Poster 2022 International Mammalian Synthetic Biology Workshop (mSBW), 28-29 July 2022, Edinburgh, UK 1 poster: C. Blackburn, J. Lorigan, A. Waite, A. Couturier, H.-Y. Kuo, Communicating reliable and accessible information about gene and cell therapy 2022 EuroGCT Sharing best practice workshop, 10 Oct 2022, Edinburgh, UK 2 Oral presentations: -A. Waite and F. Willingham, Partnering with patients-Case study on ATMP Engage PPIE Working Group -A. Waite and A. Couturier,

Table 2 .

 2 List of EuroGCT oral presentations and posters. ATMP Engage Working Group (co-led by NHS England and NHS Improvement and Genetic Alliance UK) o ATMP Engage Equity of Access working group meetings o Genetic Alliance UK ATMPs and Equity roundtable discussion o ESGCT-EuroGCT meetings for collaborating on EuroGCT programmes during 2023 ESGCT Congress  EuroGCT partners meeting and Newsletter: o Online meetings held for Management group and General Committee according to grant agreement o Monthly newsletters for disseminating progresses made within the EuroGCT network

	2.3	Additional events
	 Recurring online meetings:
		o International Society for Stem Cell Research (ISSCR) Education Committee
		(Dr. Anna Couturier, standing member)
		o EuroGCT Industry Working Group Meeting (Inaugural Meeting June 26,
		2023)
		o

  collaboration is a Europe-wide project aimed at advancing gene therapy for rare diseases. With a focus on addressing technical challenges in the field, GetRadi trains 10 PhD students to become experts in gene therapy for rare diseases. EuroGCT, as a key partner in the collaboration, contributes by providing training in digital engagement to these students during the GetRadi Summer School. Additionally, EuroGCT serves as the platform for disseminating GetRadi digital engagement projects, fostering the development of impactful resources and also supporting young researchers.

	website.
	3) Established a collaboration with Canada and USA on legal aspects through
	the project "Law, Public Policy and Social License for Next-Generation
	Regenerative Medicine", funded by the Canadian Stem Cell Network (2022-
	2025) led by Prof Amy Zarzecny. A. Mahalatchimy is participating in this
	project as a EuroGCT partner and legal scholar.
	4) Interest and inquiries received from Genetically Modified Organisms national
	authorities. Contacts with GMO national authorities were established, and this
	has resulted in a regulatory workshop 'GMO-based ATMPs' planned for the
	2023 ESGCT Congress.
	5) CARTALLEU project: A team at the Hospital Clinic Barcelona running a
	CAR-T therapy clinical trial have expressly requested to collaborate with us
	on co-developing educational resources about CAR-T clinical trials. In
	addition, their CAR-T product has been selected as the first of the five EMA
	pilot programme participants. Through this collaboration, we aim to co-
	develop learning resources for the researcher and developer community to
	learn and leverage from the team's experience in navigating the regulatory
	approval process in EU, and how they interact with regulators in EMA's pilot
	programme.
	6) T2Evolve: this project working on accelerating development and increasing
	awareness and access to cancer immunotherapy has reached out to us for
	collaborations.

2) ATMP Engage is a collaboration in which EuroGCT is a central partner. ATMP Engage brings together UK-based stakeholders with an interest in ATMPs to discuss and collaborate on Patient and Public Involvement (PPI) activity. The output of this collaboration is the "ATMP Engage PPI Directory" (www.eurogct.org/ppi-directory), a searchable and filterable directory with PPI resources for developers of cell and gene therapies, hosted on the EuroGCT 7) Our relationship with The Committee for Advanced Therapies (CAT) at EMA has strengthened as a result of ESGCT 2022 and ISCT 2023 dissemination activities. EuroGCT is currently working on a proposal for a regulatory workshop at EMA in late 2024 with support from EMA/CAT.

Table 3 .

 3 Table 3 and ranked by total views:Based on Fathom Analytics, the unique visitor counts and total views for the top accessed WP4 pages between website launch and 15 July 2023 are summarised in Table3and ranked by total views. Top accessed WP4 pages and their unique visitor counts, and total views between 1 Feb 2022 (website launch) and 15 July 2023

	Webpage

Beyond legislative requirements, the complexity and scientific technicity of these medicines, as well as the necessary flexibility to take into account the evolutions and the scientific and regulatory experience gained has led to the development of an important amount of guidelines in the field. These guidelines have mainly been adopted by the European Medicines Agency (EMA), the European regulatory agency especially in charge of the scientific assessment of medicines to be commercialized in the EU. Some of them have also been adopted by the European Commission which grants the final marketing authorization for medicines to enter the EU market. The adoption of EU guidelines involves a public consultation procedure to gather comments from stakeholders and interested parties. Our poster aims at mapping these different actors, and their potential changes over time, to allow for a realistic and concrete portrait of all stakeholders and interested parties involved in the field of ATMPs regarding their European regulation through guidelines. The identification of EU scientific guidelines applicable to ATMPs, and of the actors involved in the consultation process has led to the establishment of a database with over 200 organizations corresponding to fourteen guidelines for which dates of production range from 2008 to 2022. The organizations have been classified by type (Business, Academia, National bodies, etc.), country, and theme of the drafts of guidelines they commented on. This poster especially highlights and discusses which organizations have influenced the adoption of EU scientific guidelines on ATMPs; some of them commenting almost all guidelines in the field while others considered only certain guidelines. Findings concern both the number and quality of stakeholders with the aim to understand over time, what types of organizations from which country have been the most active in answering to guidelines' consultations, who were the major or more prolific respondents, and what themes were more often commented. Ensuring the quality of medicines is a criterion to obtain ATMPs' marketing authorisation issued by the European Commission under the "centralised procedure" after a single application to the European Medicines Agency. All ATMPs marketed in the EU must be produced in accordance with EU quality standards to ensure the quality, safety and efficacy of medicines for the patients: Good Manufacturing Practice (GMP) principles and guidelines dedicated to ATMPs enforceable since 2017, and relevant parts of the European Pharmacopeia provide the main regulatory standards to comply with. The manufacture, meeting the GMP requirements from the raw materials to the final product, is particularly challenging for ATMPs regarding their level of complexity and their specific characteristics, especially the use of substances of human origin as starting materials, and the reproducibility when using live biological samples. The ATMPs' manufacturer has to maintain costly technical specifications capable of guaranteeing the reproducibility of the medicine's composition in accordance with GMP guidelines and the specific requirements of the ATMP marketing authorisation. Regulatory requirements are often seen as obstacles to the development of ATMPs. Nevertheless, they mainly provide a set of solutions to overpass the technical obstacles of ATMPs' manufacturing in order to ensure their quality for the safety of patients. It appears clearly within the various supporting regulatory tools currently available at the European level, which include both guidance documents and procedures for interactions with regulators. This poster will provide an overview of the challenges of ATMPs' manufacturing and link them to the relevant supporting regulatory tools highlighting both their strengths and their limits in the context of ATMPs development for patients. This work has been supported by the European Union's Horizon 2020 research and innovation programme under grant agreement No 965241 and by ANR-funded I-BioLex project (ANR-20-CE26-0007-01).


The European Consortium for Communicating Gene and Cell Therapy Information (EuroGCT) has two major objectives:

• To provide patients, people affected by conditions, healthcare professionals and citizens with accurate scientific, legal, ethical and societal information and with engagement opportunities, and thus to support better informed decision-making related to cell and gene-based therapies.

• To facilitate better decision-making at key points in development of new therapies and thus enable improved product development, by providing the research community and regulatory and healthcare authorities with an information source on the practical steps needed for cell and gene therapy development.

Work Package 4 (WP4) is developing an information hub for the research and regulatory communities that facilitates access of these communities to information on the practical steps needed for cell-and gene-based therapy development and promotes information exchange within and between these communities. Screenshots of these sections along with example pages will be shown in the following pages of this report. The 'Case Studies' section contains case studies on ATMP development. Currently there are two case studies on approved ATMPs (Holoclar and Strimvelis). Future work will involve codeveloping case studies with partners for them to share their experiences and learnings in ATMP development. More details can be found in D4.1-Landscape mapping.

Screenshot of 'Case Study: Holoclar': www.eurogct.org/case-study-holoclar 12:41, 27 June, 2023

Here is an example of case study pages. In this case study for Holoclar, we collated existing information about each stage of its development, from early-stage research to approval and beyond, to serve as a guide for the research community to learn about the development process, the keys to success, the challenges and how developers overcame them. These case studies provide researchers and developers with an opportunity to engage with and to create supporting resources for the researcher and developer community.

D 4. 2 -

 2 Resources on practical steps, a set of information resources on the practical steps needed for cell and gene therapy development

  Deliverable 4.2 is a set of information resources on the practical steps needed for cell and gene therapy development, which can be viewed online at www.eurogct.org/researchpathways.The Research Pathways page has been redeveloped and published online as a filterable and searchable database. (Details in D4.1). The indexed database contains information on practical steps needed for cell and gene therapy development developed by the EuroGCT consortium or signposting to external resources. These resources include guidance on manufacturing guidelines, regulatory and scientific guidelines, intellectual property rights, market acceptability and ethical matters. It builds on the initial information published (D4.9), and further enhanced by the new search/filter functions, links to grouped topical sections, as well as new content developed by the project and signposting to additional collated resources. Contents within Research Pathways will be continuously updated throughout the project duration.The Research Pathways page also offers a gateway towards topical sections outlined as follows:Key Research Pathways: this section contains the collection of resources developed by EuroGCT WP4.  Key Pathway Resources: this section contains collated key resources on ATMP development by medicines agencies, and existing complementary work produced by other organisations.  Case Studies: this section contains our collection of new 'case study' content type, presenting collated resources on the development of specific ATMPs.  Patient and Public Involvement: this links to the ATMP Engage Patient and Public Involvement (PPI) Directory, a searchable and filterable directory with PPI resources for developers of cell and gene therapies.  Actors and Network: this section contains our initial mapping of stakeholders in ATMP development, covering the breadth of advanced therapies activities in Europe and beyond.

  .org/researchpathways/commercialisation/market-access-atmps/expediting-marketing-authorisation-0 12:41, 27 June, 2023 This is an example page of EuroGCT's original resource. More published resources can be directly accessed through the Key Research Pathways Section: www.eurogct.org/researchpathways/key-research-pathways or be found through searching in the main Research Pathways database: https://www.eurogct.org/research-pathways Screenshot of 'Case study' section: www.eurogct.org/case-studies 12:41, 26 July, 2023

Screenshot of Research Pathways information hub: www.eurogct.org/research-pathways 12:26, 26 July 2023

The 'Patient and Public Involvement' (PPI) section is linked to the ATMP Engage PPI Directory. This directory contains more than 150 PPI resources for developers of cell and gene therapies, including toolkits, guidelines, advice, templates and case studies. The 'Actors and Network' section contains our initial information on the mapping of gene and cell therapy stakeholders in Europe and beyond. It will be further developed and reported in D4.3 due month 48.