Pierre Barrat-Charlaix

Anna Paola Muntoni

Kai Shimagaki

Martin Weigt

Francesco Zamponi

Over the last few decades, protein sequencing techniques have been developed and continuous experiments have been done. Thanks to all of these efforts, nowadays, we have obtained more than 2 • 10 8 protein sequence data. In order to deal with such a huge amount of biological data, now, we need theories and technologies to extract information that we can understand and interprete. The key idea to resolve this problem is statistical physics and the state of the art of machine learning (ML). Statistical physics is a field of physics that can successfully describe many complex systems by extracting or reducing variables to be interpretable variables based on simple principles. ML, on the other hand, can represent data (such as reconstruction and classification) without assuming how the data was generated, i.e. physical phenomenon behind of data.

In this dissertation, we report studies of protein sequence generative modeling and protein-residue contact predictions using statistical physicsinspired modeling and ML-oriented methods. In the first part, we review the general background of biology and genomics. Then we discuss statistical modelings for protein sequence. In particular, we review Direct Coupling Analysis (DCA), which is the core technology of our research. Part 2 introduces two generative models based on the DCA method. The Chapter 3 model specifically considers the parameters corresponding to spatial contact, and the Chapter 4 model considers the effect of correlation between data derived from phylogeneies. Chapter 5 proposes variable selection methods that make use of the knowledge of these methods. Part 3 discusses the effects of higher-order statistics contained in protein sequences and introduces deep learning-based generative models as a model that can go beyond pairwise interaction.

Résumé

Au cours des dernières décennies, des techniques de séquençage de protéines ont été développées et des expériences continues ont été menées. Grâce à tous ces efforts, de nos jours, nous avons obtenu plus de 2 • 10 8 données relative à des séquences de protéines. Afin de traiter une telle quantité de données biologiques, nous avons maintenant besoin de théories et de technologies pour extraire des informations de ces données que nous pouvons comprendre et pour apporter des idées. L'idée clé pour résoudre ce problème est la physique statistique et l'état de l'art de le machine learning (ML). La physique statistique est un domaine de la physique qui peut décrire avec succès de nombreux systèmes complexes en extrayant ou en réduisant les variables pour en faire des variables interprétables basées sur des principes simples. ML, d'autre part, peut représenter des données (par example en les reconstruisant ou en les classifiant) sans comprendre comment les données ont été générées, c'est-à-dire le phénomène physique à l'origine de la création de ces données.

Dans cette thèse, nous rapportons des études de modélisation générative de séquences protéiques et de prédictions de contacts protéines-résidus à l'aide de la modélisation statistique inspirée de la physique et de méthodes orientées ML. Dans la première partie, nous passons en revue le contexte général de la biologie et de la génomique. Ensuite, nous discutons des modélisations statistiques pour la séquence des protéines. En particulier, nous passons en revue l'analyse de couplage direct (DCA), qui est la technologie de base de notre recherche. La partie 2 présente deux modèles génératifs basés sur la méthode DCA. Le modèle du chapitre 3 considère spécifiquement les paramètres correspondant au contact spatial, et le modèle du chapitre 4 considère l'effet de la corrélation entre les données dérivées des phylogénéies. Le chapitre 5 propose des méthodes de sélection de variables qui utilisent la connaissance de ces méthodes. La troisième partie traite des effets des statistiques d'ordre supérieur contenues dans les séquences de protéines et présente des modèles génératifs basés sur l'apprentissage profond en tant que modèle pouvant aller au-delà de l'interaction par paires.

First and foremost, I would like to express my special gratitude to my supervisor Martin Weigt for his thoughtful and invaluable advice and his generosity and patience during my Ph.D. There were many memorable fortunate circumstances during my doctoral course, but having had him as my supervisor is one of the luckiest events in my life.

I would like to express my deepest gratitude to Olivier Rivoire and Federico Ricci-Tersenghi for their acceptance as thesis reviewers. I am deeply grateful for their accepting to review of such a long thesis while having other important works. I would also like to express my sincere gratitude to Flora Jay, Silvio Franz, and Nataliya Sokolovska for accepting to be a member of the jury.

My deep gratitude goes to my collaborator Francesco Zamponi for his many excellent ideas rooted in deep physics insights. In addition, I would also like to thank Rémi Monasson and Elodie Laine for being my thesis monitoring committees.

My heartfelt thanks go to my colleagues. First of all, I would like to thank my colleagues who already left the laboratory: thanks Pierre, Giancarlo, Edwin, Anna, Carlos, and Edoardo. Exciting scientific stories, cultural stories including French and Italian languages, having dinner, traveling to French villages, and staying together in Havana, all of which are invaluable experiences. I also deeply thank my colleagues who are still in the group: Thanks Nika, Maureen, Juan, Jeanne, Matteo, Barthélémy, Sabrina, Francesco. Thank you for all of your dedicated supports: presentation practices for postdoc interviews at UCR and reviews for my thesis. I learned a lot from you, especially during the process of writing my thesis. Besides these life events, I enjoyed many events with you, discussing various topics, playing chess, disassembling the complex coffee machine, etc. I would like to thank colleagues in LCQB for having talks, taking lunch/coffee, going out to bars, watching films, and watching many episodes of Attack on Titan. I would like to write all of your names, but I'll thank you directly when meeting you next time in Paris.

I would also like to thank Jérôme and Barbara for intriguing discussions about RBMs. They have helped deepen my understanding of the models. I would express my great appreciation to the people at Politecnico di Torino for letting me stay a couple of weeks. Especially Andrea Pagnani and Carlo Lucibello for your hospitality and scientific discussions. My gratitude also goes to people at the Havana University for letting me participate in the summer school of the complex system. Staying in Cuba for one month was a special experience. Thank you for your heartfelt hospitality.

As for financial support whole three years of my doctoral program, I sincerely appreciate the Honjo International Scholarship Foundation. I am deeply impressed with their dedication to connecting and supporting young scientists around the world. I would also like to thank people I met in Paris. Especially those I met in the Paris Young Physicists Association and the Japanese Researchers Association. Many interesting discussions and stories stimulated my scientific curiosity. I am also grateful for the experiences of having dinner and traveling together with wonderful people. I had various wonderful experiences at the Maison du Japon in the Cité Internationale Universitaire de Paris. I sincerely hope that such a wonderful facility for students/scientists/artists will be maintained eternally.

Finally, I would like to express my gratitude to the people who are close to me and care for me with special affection. I thank Émilie for being with me and doubling funs while halving sorrows. I sincerely thank my family for understanding and respecting my decisions and ideas. Thank you for your deep affection.

I Introduction

Chapter 1

Protein Structure and Sequence

In 1962, Sir John Cowdery Kendrew shared the Nobel Prize in Chemistry with Max Perutz because of their pioneering works to determine protein structures using X-ray crystallography. After ten years of technological advance, scientists were asking a question while examining structures of proteins and their mechanical and physiochemical properties: Why and how can an amino-acid sequence obtain its three-dimensional and quickly (in between 100 micro sec and 1 ms [1]), even if the number of the conformational possibilities is astronomically high? Nowadays, this question is known as Levinthal's paradox [2]. Later, an important hypothesis that helps to understand the question, was proposed by Christian B. Anfinsen [3]. Anfinsen's hypothesis or principle states that the structure of a protein is determined not by a biological process, but by the purely physicochemical properties of a particular amino-acid chain and its solvent environment. More precisely, the folding process can be interpreted as the process driven by a free energy landscape, which is uniquely characterized by the amino-acid sequence and the environment, and the natural state is the thermodynamically stable one. Anfinsen's hypothesis implies that it is possible to predict the threedimensional structure of proteins from these amino-acid sequences.

Predicting a three-dimensional structure as well as understanding the functionality of a protein from a set of amino-acid sequences has been a grand challenge in computational biology for over 50 years since Anfinsen's hypothesis was proposed [4,5].

Proteins

Proteins are biological macromolecular compounds that consist of at least one linear chain of amino acids linked by peptide bonds in between. They are the most essentials material in living systems: almost all life activities are directly associated with proteins. Most proteins fold into unique structures and have evolved to bind other proteins, DNA, RNA, or other molecules. Therefore they are required to form and maintain a precise spatial structure and biological functions. Well known function of proteins are:

• Structural proteins, which provide structural components.

• Contractile proteins, which associate with motion of muscles.

• Transport proteins, which carry other substances.

• Storage proteins, which store nutrients.

• Hormone, which regulate body metabolism and nervous system.

• Enzyme, which catalyze biochemical reactions.

• Protection, defend cells from foreign substances.

Each protein has a unique structure and functionality (except for intrinsically disordered proteins, which are not considered in this thesis). Hence if proteins are folded incorrectly or interact with other molecules inappropriately, it causes dysfunction of the system. In some diseases, protein misfolding is believed to be the primary cause, e.g., Alzheimer's disease, Parkinson's disease, Huntington's disease, and many other degenerative and neurodegenerative disorders [6].

Protein sequence

Proteins are made from some amino acids, ranges from about 30 to more than 10,000. The majority of them are 50 to 500 amino-acid residues in length. The amino-acid chains are tied due to covalent peptide bonds linking two consecutive amino acids. Therefore proteins are referred to as polypeptides, and protein chains are called polypeptide chains. The 20 naturally occurring amino acids have three-letter or one-letter abbreviations, and they are always written from N-terminus to C-terminus of a polypeptide chain. Figure 1.1: Left: An amino acid, a building block of proteins, contains both amine (base) and carboxylic acid. It is the R-group, a side chain that characterizes the 20 different natural amino acids. Right: The peptide bond, a joint between amino acids, emerges as a consequence of the dehydration synthesis. (Source [7])

Proteins are assembled from a set of 20 unique amino acids, which are organic compounds made of an α-carbon (C α ) with four substitutes, an amino (-NH2) group, a carboxyl (-COOH) group, a variable side chain (-R) and a hydrogen atom (-H) (Fig. 1.1 left).

Amino acids are commonly categorized into four groups, according to the charge and polarity of the side-chain [8]:

1. Nonpolar amino-acids group is hydrophobic, which means they are repelled from water since they have very small dipole moment due to the non-polarity. These amino acids have an aliphatic or aromatic side chain. These amino acids are generally located inside the protein core and participate in van der Waals interactions.

2. Uncharged polar amino acids are more water-soluble thus more hydrophilic than nonpolar amino acids. These types of amino acids are often found at the surface of proteins.

3. Aromatic amino acids. As the name suggested includes an aromatic ring and amphipathic, meaning that it has both hydrophilic and hydrophobic properties.

4. Charged amino acids, the side chains often include salt bridges that combine hydrogen bonding and ionic bonding. Amino acids classi-Figure 1.2: (1) The primary structure is given by an amino-acid chain, which is linked formed by peptide bonds. (2) The secondary structure can take a form of an alpha-helix or a beta-sheet. The hydrogen bonds, which are indicated as dotted lines, maintain the secondary structure. (3) The tertiary structure is a three-dimensional shape, which is formed by the further folding and binding of the secondary structure. (4) The quaternary structure occurs as a consequence of interactions between two or more tertiary structures.

As an example, we show here hemoglobin, a protein transporting oxygen to body tissues. (Figure is adapted from [7]).

Figure 1.3: Venn's diagram of the 20 naturally occurring amino acids, classified according to some of these physicochemical properties. (Source [9]) fied in this group contribute to the stabilization of three-dimensional protein structure.

As show in Fig. 1.3 these groups are partially overlapped. As an example, Histidine (H) is a polar, charged, and aromatic amino acid.

The structure of proteins involves four different levels, cf. Fig. 1.2 :

1. Primary structure, amino-acids sequence, which is defined as the linear polypeptide chains.

2. Secondary structure, locally folded structure that forms within a polypeptide chain. It is formed by the interactions among atoms in the backbone chain. The typical secondary structures: α-helices that is a single peptide chain that forms a helical staircase-like structure. βsheets that is a structure formed by segments of a polypeptide chain that line up next to each other and make a sheet-like structure.

3. Tertiary structure, a three-dimensional conformation that is formed by an entire polypeptide chain primarily due to interactions among the R-groups of amino acids.

4. Quaternary structure, a complex of multiple polypeptide chains.

These shapes of proteins can be divided into three distinct categories based on structure and functionality: globular proteins, fibrous proteins, and membrane proteins [10].

In this section, we explained some general properties of protein sequences.

In the next section, we will see the basic properties of protein structures and their role in nature.

Protein structure

Determining the structure of proteins is vital for understanding how thousands of molecules can work together and keep our bodies healthy. For instance, understanding the behavior of small-drug molecules and target proteins in solution is fundamentally important for drug discovery. Thus precise information of structure is essential. The branch of molecular biology that aims to understand molecular structures of proteins and RNA is called structural biology and is known as one of the representative areas of biology.

In general, most protein structures are determined by experimental methods such as X-ray crystallography1 . X-ray crystallography is the currently most favored technique for structure determination of protein in the sense of the accuracy of structure determination. Most of the structures included in PDB, a comprehensive protein structure database, are determined by this method [11] (there is a description of PDB in Appendix A.1). The structural information of the electron densities are encoded into the intensities of the diffraction patterns of crystallized proteins, which are known as structure factor [12]. Creating high-quality protein crystals is crucial for deducing the structural information of a protein. However, it is highly demanding to create relevant protein crystals in most cases, especially for relatively large proteins and certain types of protein. For example, most membrane proteins, which occupy 25% of all protein species and are essential as drug targets (as mentioned in the previous paragraphs), have their structures yet to be determined [13]. Therefore, determining protein structure using a computational approach is scientifically and medically important.

In the next section, we will see how to make protein sequences more amenable for applying statistical modeling. We will also discuss some statistical properties of protein sequences.

Multiple-sequence alignments and protein families

Beneficial proteins (or even if neutral) for adapting environments can spread among descendants and different species. Over the course of propagating proteins, i.e., evolution, associating genetic information can be changed while keeping its functions and a three-dimensional structure. Consequently, protein-sequence also involves mutation effects after the transcription (DNA → RNA) and translation (RNA → protein). A group that shares the same ancestor is called homologous, and characteristic similarity in such a group is called homology. A group of proteins in the relation of homologous is especially referred to as protein family (we will discuss the details in the next section). Homologous groups are important information resources that enable us to understand the genotype-phenotype relationship.

A codon, which is a combination of three consecutive nucleotides (A T G C) of DNA, can encode naturally occurring 20 different types of amino acids. There are three types of DNA point mutations that can affect protein sequences:

1. Substitution, replacing a nucleotide with another one 2 .

2. Deletion, remove one or more nucleotides. 3. Insertion, adding one or more nucleotide.

As a result of the evolution of DNA sequences, the corresponding protein sequences often have different lengths. Typically, consecutive aminoacid insertions and deletions can occur in regions such as loops that do not 2 In some cases, nucleotide substitutions do not cause any change in amino-acid encoding. For example, both AAA and AAG codons translated to the same amino-acid Lysine. It is called codon degeneracy, and this type of mutation is known as the silent mutation and does not influence the amino-acid sequence ensemble. On the other hand, when mutated to a codon that does not correspond to a naturally occurring amino acid (e.g., TAG, TAA, TGA), translation is stopped, and amino-acid sequence production is stopped at this codon position. This type of codon is called stop codon, and the mutation is called missense mutation, and there are some diseases due to the missense mutation. contribute to structural stabilization, i.e., it's not involved in structurepreserving constraints. Therefore we cannot naively compare the same residue sites of different sequences due to the insertions and deletions. Hence, what is needed is alignment, which makes the length of the sequence ensemble constant by inserting an additional symbol "-" corresponding to insertion so that the similarity between sequences is maximized.

The basis of sequence alignment is a pairwise alignment algorithm, a method to align two sequences based on dynamic programming.

Probably Smith-Waterman algorithm is the most standard pairwise alignment algorithm [14]. This algorithm requires O(L 1 L 2 ) computational cost (number of necessarily required operations) to align two sequences, where L 1 and L 2 are lengths of the two sequences to be aligned. It works well, but if we apply this method to M different sequences to align simultaneously, computational cost becomes min(L 1 , . . . , L M ) M ≤ L 1 • • • L M , therefore it will be impractical with increasing the number of sequences. However, as we will see in this thesis, in many cases, we need to align thousands of sequences simultaneously. Therefore, alignment methods should be extended to multiple sequence alignment for applying statistical argument. The most commonly used heuristics for multiple alignments are:

1. Progressive alignment aligns pairs of the most similar sequences, and then progressively aligns the partially aligned sequences until all sequences are aligned. One of the shortcomings of this method is that the alignment errors mainly due to the initial pairing of sequences propagate and accumulate to the final results. Methods of Clustal series [15] are the most well-known progressive algorithms.

2. Hidden Markov Model-based alignment is probabilistic alignment methods that construct probabilities or profile Hidden Markov models (HMMs) of sequences. For the initial construction of HMMs, it needs a small number of well-curated and aligned sequences (around 100 sequences), referred to as seed alignments. Then apply local optimization of HMM alignments scoring system. These HMMs provide us with the ratio of the likelihood that a given sequence originates from a statistically identical distribution to the HMM (or equivalently the ensemble of sequence used for the training of HMM) and the likelihood that the sequence appears by chance (randomly).

Hereafter, we assume HMM-based alignment (cf. the section about pro-file HMMs below for details). HMMs can also be used to search sequences using HMMs scoring systems. HMMer [16,17], HH-suite [18] and HMM-HMM [19], commonly used bioinformatics softwares can search sequences, particularly homologous sequences from sequence databases such as UniPro-tKB [20] and SwissProt [21]. Our study frequently uses Pfam, a database of aligned protein families that contains MSAs and profile HMMs to retrieve protein domain sequences for statistical modelings (there is Appendix A.1 about the databases and explained, UniProtKB, SwissProt, and Pfam database).

Protein families

A protein family comprises evolutionarily related proteins that share a common ancestor protein while being structurally and functionally similar. Hereafter, we assume that MSA refers to be the MSA of the protein family.

MSAs are assembled based on sequence similarities, typically have more than 20% of sequence identity among a protein family. Despite such a small similarity of sequences, corresponding 3D structures and functionalities for each sequence in a protein family are similar or almost identical. Fig. 1.4 shows a part of a multiple sequence alignment for one family.

As we mentioned earlier, each sequence is quite diverse from the others, even if around 20% of sequence identity can be considered the same family 3 . However, some sites are highly conserved (such as sites involving a Cysteine, C, in Fig. 1.4 ). Some pairs change in correlated ways, meaning that if one site changes, the other site will be adjusted accordingly (correlations could be both negative and positive values). From now on, we represent an MSA as a M × L rectangular matrix, where M and L are assumed to be the number of sequences and the length of aligned sequences. Typically the number M can be 10 3 ∼ 10 5 , and the number of amino acids L is 50 ∼ 500. We define a shorthand representation of an MSA matrix as (A m i ) m=1,...,M i=1,...,L , where A m i ∈ A is one of the 20 naturally occurring amino-acid letters or the insertion and deletion symbol "gap", A = {A, C, ..., Y, -}. Alternatively, we represent an MSA as (A 1:M ) t = (A 1 , . . . , A M ) t , where A ∈ A L is one sequence, and 1:M = {1, 2, . . . , M }. To describe the states of amino acids simply, we assume the amino-acids letters and the gap to be mapped to numbers 1:q = {1, 2, . . . , q}, where q is the total number of states, i.e., q = 21. Note that the following discussions do not depend on the way of mapping (this mapping causes no linear relation) 4 .

As we can see in Fig. 1.4, the site-specific frequencies seem to characterize MSAs well. That is, the independent single-site variables are statistically important variables to be regarded. Formally, the single-site frequencies are define as

f i (a) = 1 M M m=1 δ A m i ,a , (1.1)
where i being a site index that runs ∀i ∈ 1:L and a being as one of the alignment states that runs ∀a ∈ 1:q. The symbol, δ a,b is the Kronecker delta that takes value 1 if a = b, and 0 otherwise. Indeed, these single-site frequencies permit us to understand and construct MSAs. For example, the single-site entropy H i of {f i (a)}, allows us to estimate the amount of the relative information R i we can extract from knowing the id site i ∈ 1:L,

H i = - q a=1 f i (a) log 2 f i (a) R i = log 2 (q) -H i . (1.2)
By definition, the information content gives a large value for conservation sites (small entropy). On the contrary, it gives a small value for variable sites (large entropy). Fig. 1.4 shows an example of the sequence logo (denoted as "HMM Logo" in the figure), which represents how each site is conserved using the information content R i and the frequencies f i (a) for determining the symbol size.

Sequence motifs

Sequence motifs are site-specific nucleotide or amino-acid sequence patterns that are biologically significant and well characterize sequence ensemble. Typically these are closely associated with conserved and/or functionally relevant sites. Identifying functional regions or regularly appearing patterns in genetic data is a fundamental step to understand biological sequences. For example, to find DNA regions where transcription factors 5 can bind is known as a difficult and practically important problem [22,23]. Sequence motifs can also be used to find corresponding sequences (methodologically, this task can be done by combining strings search algorithms such as the Rabin-Karp algorithm and Boyer-Moore algorithm [24]). It should be noted here that the profile HMM can also be treated as sequence motifs.

Last, but not least, phylogenetic motifs, sequence motifs that can discriminate the differences between evolutionarily related sequences, are also important motifs [25,26].

Here, we show the basic methods to find sequence motifs, 1. Position frequency matrices (PFM). Count the number of occurrences of each state for each site in a given alignment. Formally, PFM at position i ∈ 1:L as a state a ∈ 1:q is M ia = M m=1 δ A m i ,a for an MSA. Another variant, which is essentially equivalent to PFM is the position probability matrix (PPM) which is normalized simply, thus equivalent to the {f i (a) = M ia M } single-site frequency defined in Eq. 1.1. 2. Position specific scoring matrices (PSSM). Provide an informationtheoretic weight that takes into account background frequencies. A PSSM is given as

M ia = f i (a) log f i (a)
f (a) 6 , where f (a) = 1 L L i=1 f i (a) .

These sequence motif methods are based on a preexisting MSA and explicitly depend on site-specific frequencies.

In the study of the Hopfield-Potts models (see in Sec. 4.3.2), we show that the patterns of Hopfield-Potts model are closely related to sequence motifs and can characterize the protein-subfamilies.

Profile Hidden Markov Models

Profile HMMs are probabilistic models that can represent ensembles of biological sequences. They are constructed from curated sequences, which is referred to as the seed alignments 7 . HMMs can be used to align sequences and search sequences based on a query sequence.

Definition -The following explanation of the profile HMMs is based on [14,27,28]. A HMM is defined as the two types of variables, visible variables and hidden variables that are denoted as v = (v 1 , v 2 , . . . , v L ) and h = (h 1 , h 2 , . . . , h L ), respectively. Here, v i is i-th observable taking one of the states in o = {o 1 , . . . , o q } that correspond to the set of amino acids for protein sequences (or nucleotide for DNA or RNA sequences), and h j is j th state of hidden variable that takes a state from s = {s 1 , . . . , s p }, corresponding to match, insertion, and deletion state in protein sequences. As the name implies, the model assumes the Markov property in the hidden space. That is, the probability of hidden variables can be written as

p(h i+1 = s |h i = s, h i-1 , . . . , h 1 ) = p(h i+1 = s |h i = s) = t(s |s) , (1.3)
where s, s ∈ s, and t(s |s) is called transition probability.

Similarly, the probability of the state v i depends only on the state of h i , formally it can be wwritten as p(v i = o|h i = s, v i-1 , h i-1 , . . . , h 1 , v 1 ) = p(v i = o|h i = s) = e(o|s) . (1.4) where o ∈ o, and e(o|s) is called the emission probability.

By summarizing these emission and transition probabilities, we can represent the joint distribution of the visible and hidden variables,

p(v, h) = p(v|h)p(h) p(v|h) = L i=1 e(v i |h i ) p(h) = p(h 1 ) L-1 i=1 t(h i+1 |h i )
.

(1.5)

Notably, the visible variables are conditionally independent when all hidden variables are given. Therefore, we can efficiently compute the observable sequences if hidden states are known.

Profile HMMs for protein sequences -For the basic HMMs, there are three states s = {M, I, D}, where M, I and D are referred as match, insertion and deletion states for each site i ∈ 1:L. The emission probabilities for these cases are (Fig. 1.5 ):

1. e(v i |h i = M ): If a hidden variable is in the match state, the visible state emits one of the amino-acid variables according to the sitespecific probability estimated by PPM f i (a) = M ia M . 2. e(v i |h i = I): If the hidden variable is in the insertion state, the visible state emits an amino-acid symbol according to the background An edge between the states is denoted as one of the transition probabilities t(s |s). (Source [14]) frequency f (a).

3. e(v i |h i = D): If the hidden variable is in the deletion state, the visible state emits the gap symbol with probability one.

. Fig. 1.5 shows a typical profile HMM architecture.

Application of profile HMMs -As we mentioned before, profile HMMs can provide a probability of an observed sequence. Particularly it can be used for finding an ensemble of sequences that are similar i.e., homologous to seed sequences of a given HMM, which corresponds to a high score. This task is known as the scoring problem. One way to compute the probability is marginalize the hidden variables h,

p(v) = h p(v, h) , (1.6)
however, the exact naive computation of Eq. 1.6 is intractable because the complexity increases exponentially as increase the number of hidden states. A more sophisticated approach enables us to compute the score function p(v) exactly by applying the forward-backward algorithm, a dynamic programming algorithm.

Second, another important application is to align a sequence to a given profile HMM. This problem is known as the optimal alignment problem. This problem needs to insert the gaps optimally. Therefore this problem is equivalent to find the optimal hidden sequence: h * = argmax h p(h|v) .

(1.7)

In order to find optimal hidden states, which equivalent to find an optimal path, typically the Viterbi algorithm [29], a dynamic programming algorithm, is employed.

Co-evolution

Residue-residue interactions are critically important for protein stabilization and acquisition of function. Suppose a mutation occurs at a site that alters the stability of a protein, residues that are influenced by the substituted residue are expected to be compensated over the evolutionary time scale. Therefore correlations between residues of MSAs contain coevolution information.

However, not all strong correlations are due to structural constraints since shared evolutionary history also induces background correlations or phylogenetic correlations.

The first attempts to predict the secondary and tertiary structure of the macromolecule from nucleic-acid sequences based on the insight of coevolution mechanism was proposed in 1991 [30]. In this study, a mutual information (MI) based method was applied. The first attempt to predict residue-residue contact predict regarding the co-evolutionary information was done by the most naive way i.e., analyzing covariance matrix of residues in 1994 [31]. The first method for predicting MI-based residue-residue contact, waiting another year, was done in 1995 [32].

MI is one of the simplest information-theoretic quantities that can quantify how much random variables X and Y influence each other. The definition of MI between two variables X and Y can be written as follows,

M I XY = x,y p XY (x, y) log p XY (x, y) p X (x)p Y (y) , (1.8)
where, p xy is the joint distribution of x and y, where as p X and p Y are marginalized distribution by one another, p X (x) = y p XY (x, y) and p Y (y) =

x p XY (x, y). The Eq. 1.8 is equivalent to the Kullback-Leiblear divergence (KLD) between the joint distribution p XY and the factorized independent distribution p X p Y 8 .

The MI function for the contact prediction is defined as following,

M I ij := q a,b=1 f ij (a, b) log f ij (a, b) f i (a)f j (b) , (1.10)
where i, j are site indices of protein-sequences running ∀i, j ∈ 1:L, and ∀a, b ∈ 1:q are naturally occurring amino acid or gap state. hereafter if there is no specific explanation of i, j, k, we assume these are indices of residues in 1:L, similarly a, b, c are amino acids/gap in A or 1:q. The definition of {f i (a)} is the same given in Eq. 1.1, and {f ij (a, b)} are pairwise-frequencies, which given as the natural generalization of Eq. 1.1 :

f ij (a, b) = 1 M M m=1 δ A m i ,a δ A m j ,b . (1.11)
If sites i and j are located in spatial contact, the occupation of amino acids at site i may receive influence from j (and vice versa), therefore M I ij should assume a large value in this case.

Here, "contact" means the minimal distance of heavy atoms of amino acids is less than 8 Å, and the correlations are used only for pairs that satisfy |i -j| > 4 in our study 9 .

The accuracy of the residue contact prediction based on the MI method was drastically improved using phylogenetic correction methods, which is called average product correction method (APC) [33](there is a section about 8 The Eq. 1.8 can be also represented as

M IXY = HX -H X|Y (1.9)
The last result Eq. 1.9 can be interpreted as the mutual information is the amount of information gained from the independent system X when adding the knowledge of the state of the variable Y . 9 We exclude neighboring 4 sites since there are around 4 amino acids in one pitch of alpha-helix, such short-range predicted contact are therefore considered to trivial result and not very useful in protein structure prediction.

APC in the next section).

The MI-based method can give better predictions than the naive covariancebased residue contact predictions. However, as we will see in the next section, the MI-based approach also leads substantial amount of false positive predictions.

Many such false positives are likely due to spurious correlation. Even though there is no direct relationship between the stochastic variables A-B, the presence of another variable C causes a significant correlation between A-C, similarly B-C, so that an apparent correlation can emerge between A-B as a result. From now on, we refer to such a spurious correlation as a non-direct correlation and a true correlation as a direct correlation.

The following sections discuss statistical protein sequence modeling that can generate artificial sequences and predict direct correlations that can fairly disentangle the direct and non-direct correlations. We also show statistical modeling-based methods such as Direct Coupling Analysis (DCA) outperform the MI-based contact prediction.

Chapter 2

Statistical Sequence Modeling

In this section, we review the maximum entropy (MaxEnt) principle, which enables us to predict structural information of proteins from amino-acid sequence alignment. Then, we formulate the Direct Coupling Alignment (DCA) using the MaxEnt principle. Next, we show the features DCA can learn from MSA. We show that the problems of DCA involved and approximation methods to overcome these problems.

Maximum-Entropy Modeling

The maximum entropy principle is a method to derive an analytical form of probability distributions, which are initially introduced by E.T. Jaynes [34]. The MaxEnt principle is a framework that explicitly gives us a functional form of a probability distribution by maximizing the entropy under the constraint of reproducing the empirical averaged values of a set of observables.

Suppose that we choose a set of observables {O µ (A)} µ∈1:P , where O µ : A L → R is a real-valued function of an amino-acid sequence. Then, we aim at constructing a statistical model p(A), which maximizes the entropy,

S[p] = - A∈A L p(A) log p(A) , (2.1)
under the following certain constraints, ∀µ ∈ 1:P ,

A∈A L p(A)O µ (A) = O µ A . (2.2)
Here we used a short hand representation

• A = A∈A L •f (A), where f (A) is an empirical distribution.
The advantage of MaxEnt modeling is that it guarantees the desired properties while keeping the minimum necessary conditions for the formulation by the maximization of the entropy. Therefore, it is an objective modeling method.

As a standard approach for solving this type of optimization problem with equality constraints, we apply the method of Lagrange multipliers. We introduce multipliers, λ µ , µ ∈ 0:P , which are the conjugate parameters for all constraints. Finally, a functional of the probability distribution is formulated as,

F[p] = - A∈A L p(A) log p(A) + P µ=1 λ µ   A∈A L p(A)O µ (A) -O µ A   + λ 0   A∈A L p(A) -1   .
Here, λ 0 imposes the normalization of p(A) as a probability distribution. By maximizing F[p] with respect to p, we get the general form of the MaxEnt distribution,

p((A)|{λ µ } µ ) = 1 Z({λ µ }) exp P µ=1 λ µ O µ (A) . (2.3)
We replace λ 0 explicitly by e -(λ 0 +1) = Z({λ µ }) = A∈A L exp P µ=1 λ µ O µ (A) the normalization factor (also called partition function in the statistical physics literature). Note that the probability distributions that can be found using the MaxEnt modeling form are always a linear-exponential family, and the energy function, which is the function defined within the exponential function, explicitly depends on this selected observables O µ (A) .

In the next sections we will see the derivations of profile model and DCA based on the MaxEnt principle.

Profile models

Profile models or single-site models can reproduce the position-specific frequencies of multiple sequence alignments. As we saw in the introduction section, site-specific frequencies {f i (a)} are significant features to characterize MSA and carry the information about conservation. Therefore profile models can be placed as the simplest model to describe MSAs. Actually, in sequence bioinformatics, they are the most important and applied statistical model.

In terms of MaxEnt modeling, the observables of the profile model are each state of residues O ia (A) = δ A i ,a for each site i ∈ 1:L and all amino-acid states a ∈ 1:q . Therefore, the constraints that have to be satisfied in our profile model are,

A∈A L p(A)O ia (A) = q A i =1 p(A i )δ A i ,a = f i (a) , (2.4)
where f i (a) are the single-site frequencies defined in Eq. 1.1. The Lagrange multipliers, which are the conjugate parameters of these observables are λ ia = h i (a), ∀i, a, and we can find the explicit form of profile models,

p((A)|{h i (a)} ia }) = 1 Z({h i (a)} ia ) exp i,a h i (a)δ A i ,a = L i=1 exp h i (A i ) q b=1 exp h i (b)
.

(2.5)

An important consequence here is that the model parameters of the profile model can be easily obtained by using the relation Eq. 2.4, h i (a) = log f i (a) + const. Note that the defined model factorizes over sites, therefore it is also called the independent site model.

Pairwise Potts model

We will see that the derivation of the pairwise Potts model (PPM) is a natural generalization of profile models within the MaxEnt modeling. PPM consider also pairwise interaction between amino acids, therefore the observables in MaxEnt are two-site (pairwise) quantities O iajb (A) = δ A i ,a δ A j ,b , for all of locies ∀ i, j ∈ 1:L and for all of the possible amono-acid letters ∀a, b ∈ 1:L. We also keep the single-site observables δ i (a). The additional constraints for the MaxEnt modeling for the PPM are thus

A∈A L p(A)O iajb (A) = q A i ,A j =1 p(A i , A j )δ A i ,a δ A j ,b = f ij (a, b) .
(2.6)

Here, we used the empirical two point frequencies extracted from data, f ij (a, b). After a little calculation we can get the following probability distribution,

p((A)|h, J) = 1 Z(h, J) exp i<j,a,b J ij (a, b)δ A i ,a δ A j ,b + i,a h i (a)δ A i ,a , (2.7
) with the J ij (a, b) being new Lagrange multipliers. Here Z(h, J) is a normalization factor,

Z(h, J) = A∈A L exp (-H(A)) -H(A) = i<j J ij (A i , A j ) + i h i (A i ) , (2.8)
where A(A) is known as the fully connected pairwise Potts energy function or Hamiltonian in physics. Thus, predicting the parameters of PPM means inferring the Hamiltonian 1 . In the case of the PPM, the normalization factor does not factorize due to the interactions between sites. Therefore analytically obtaining forms of partition functions is practically impossible. Moreover, the complexity for calculating it is q L . Thus, it will soon be impractical to obtain it directly as L increase (typically, L = 50 -500, hence too large to resolve directly). The log-likelihood function of the probability distribution for a given MSA is

l DCA (h, J ) = a,b f ij (a, b)J ij (a, b) + a f i (a)h i (a) -log(Z(h, J )) , (2.9)
and can be used in maximum-likelihood (ML) inference as an on objective function.

The Lagrange multipliers {J ij (a, b)} contain spatial information and are supposed to exclude the spurious correlations successfully, i.e., correlations that are not associated with structural information as discussed in Sec. 1.3. These pairwise parameters are called couplings in physics, and they are also referred to as direct correlations in the context of residue contact prediction.

A framework to predict residues contact based on the model in Eq. 2.9 is known as direct coupling analysis (DCA), which was initially introduced using message passing algorithm [37]. DCA has made significant contributions in statistical genetics and structural biology, through modifying and improving the model itself.

Due to the numerically intractable partition function in Eq. 2.8, some approximation methods were developed. In the following section, we present representative approximation methods for DCA.

Boltzmann DCA

One of the first important contributions of DCA was done using the message passing algorithm. It showed astonishing results but still, it needed to accelerate more since there are convergency issues2 , and the limit of treatable protein sizes were around 80 [37]. In this section, we introduce Boltzmann machine DCA (bmDCA), the algorithmically simplest and versatile model.

The Boltzmann machine (BM) is a generative model for an arbitrary distribution, which was introduced by Geoffrey E. Hinton in 1985 [38]. BM is characterized by an energy function, the mathematical form of the distribution is an exponential family. The probability distribution is learned by MCMC method in general. Historically speaking, BM has been investigated in the field of information theory or machine learning. However, the model is equivalent to the Sherrington-Kirkpatrick model in statistical physics, which was introduced in 1975. Probably, to adapt Boltzmann ma-chine learning to DCA is the most naive and intuitive idea [39].

The equation for updating the model parameters for bmDCA 3 follows exactly the derivative of the likelihood function l DCA introduced in Eq. 2.9 for the probability Eq. 2.8,

∂l DCA (h, J ) ∂h i (a) = δ A i ,a data -δ A i ,a H = f i (a) -p i (a) ∂l DCA (h, J ) ∂J ij (a, b) = δ A i ,a δ A j ,b data -δ A i ,a δ A j ,b H = f ij (a, b) -p ij (a, b)
.

(2.10)

Parameters are updated iteratively using gradient ascent until the fixed point equation Eq. 2.10 goes to zero ∀ i, j, a, b,

h i (a) ← h i (a) + η h ∂l DCA (h, J ) ∂h i (a) J ij (a) ← J ij (a) + η J ∂l DCA (h, J ) ∂J ij (a) . (2.11)
The average, • H over the statistical model is realized by using MC sampling from the distribution in Eq. 2.8. Note that MC sampling depends on the model parameters, we need to resample therefore at each time for computing the gradient of the log-likelihood with slightly updated model parameters {J ij (a, b), h i (a)}. Empirically, this learning algorithm suffers from the overlearning issue. We need to introduce regularization parameter for the optimization, the details of this regularization will be discussed in the following section.

One of the most significant features of bmDCA is that it constructs a generative model for protein sequence: It can reproduce one-point frequencies and two-point connected correlations for the empirical distribution. Moreover, bmDCA was reported to reproduce non-fitted quantities such as the three-point correlation, the principal component analysis (PCA) distribution, and the distribution of Hamming distances. Therefore bmDCA reaches incredibly high reproducibility of the empirical data. Furthermore, bmDCA can generate artificial functional protein sequences; details are explained in the following section 2.5 about the DCA applications.

bmDCA reproduces a wide range of statistics

MaxEnt modeling of bmDCA is constructed to reproduce single-site and pairwise frequencies, so also the connected correlations of the empirical distribution should be reproduced precisely. We show a scatter plot of twopoint connected correlations between a natural MSA (the ensemble of natural sequences used as training data in bmDCA) and an artificial MSA (an ensemble of generated sequences using MC sampling) in Fig. 2.1. The fitting accuracy is significantly high.

Historically, one of the most important contributions of the DCA method is the contact prediction of contacts between residue in the three-dimensional [41]. Both of the figures are adapted from [42].

structure of the protein. The key idea is that strong couplings {J ij (a, b)} are typically associated with spatial interactions due to the co-evolution under the constraint of keeping the structure. Fig. 2.2 shows the result of residue contact prediction for one exemplary protein family, we also compare the prediction with the MI method mentioned in the section 1.3.

As we mentioned several times, bmDCA reproduces non-fitted quantities, i.e., statistics that are not explicitly imposed to be reproduced. Most strikingly, none of the measurements can significantly distinguish natural sequences and sequences generated by bmDCA, which implies that the pairwise Potts distribution is effectively close to bee indistinguishable from the empirical distribution. In Fig. 6.1, we show the three-point connected correlations C ijk (a, b, c),

C ijk (a, b, c) = f ijk (a, b, c) + 2f i (a)f j (b)f k (c) -f ij (a, b)f k (c) -f jk (b, c)f i (a) -f ki (c, a)f j (b) , (2.12)
and PCA diagram as examples of the non-fitted quantities for PF00072. As we reviewed in this section, bmDCA can markedly well reproduce statistical properties of natural sequences. However, there are computational and statistical problems:

1. Computational complexity: The normalization factor of the probability distribution, the partition function Z, is defined as a sum over a q L sequence configuration space, hence it is intractable to calculate naively. Although bmDCA can realize the probability distribution without directly estimating the partition function Z, it takes substantial time to reach the equilibrium probability using MCMC, typically between five hours and three days. Therefore, it needs approximations to learn the model with a small computational cost.

2. Non-uniqueness of estimated parameters: Most of the estimated model parameters, especially those not involved in contacts, are close to zero but noisy. The gradient-based learning cannot achieve a global minimum with a finite number of MC samples for computing the gradient and with finite learning time. This issue becomes a bottleneck for accurately investigating, and interpreting coupling parameters.

Over-learning:

The bmDCA model depends on too many model parameters. The effective number of available sequences is 10 3 -10 5 typically. However, the number of model parameters to be optimized is 10 5 -10 7 . Therefore, DCA-based learning typically suffers from an over-learning problem.

Selection of variables:

The choice of the observables in MaxEnt might be subjective. Three-body interactions and/or collective variables might be inherently significant variables for protein sequences, while many of the one-and two-site frequencies are not important.

In the next section, we show alternative models to resolve some of these problems. In particular, sections 2.3.1 and 2.3.2 are computationally fast and frequently used as residue contact prediction methods, section 2.3.3 is a computationally fast sequence generative method, and section 2.3.4 is contact prediction methods with a small number of model parameters and is closely associate with our works.

Approximations of DCA

In this section, we review some well-established approximation methods that we use typically in our study are selected.

First, we review the mean-field DCA, the fastest DCA-based algorithm, to predict the residue contacts. Then we discuss the pseudo-likelihood DCA, which is also the first algorithm having the asymptotic consistency. Third, we review the very recent autoregressive DCA, which is the currently fastest algorithm to generate artificial sequences as well as to predict contact prediction. Finally, we discuss Hopfield-Potts DCA, which can be used for the contact prediction problem and can reduce the number of model parameters.

Mean-field DCA

Mean-field DCA (mfDCA) was introduced in 2011 [42]. The computational cost of this method is proportional to O(q 3 L 3 ), which is relatively small, hence requiring computational time is substantially faster than bmDCA. The downside is that this model cannot be used as a generative model. Parameters of mfDCA are typically associated with a low-temperature regime (1 < T ) due to overestimated coupling parameters.

The algorithm of mfDCA is based on a small-coupling expansion which is equivalent to a high-temperature expansion in physics. The following derivation is based on the article [42].

First, we introduce a perturbation parameter into the Hamiltonian,

-H(A|h, J ; α) = α i<j J ij (A i , A j ) + q i=1 h i (A i ) .
(2.13)

If we put α = 0, then the model will be an independent-site model or profile model. If α = 1, the standard pairwise model appears. Furthermore, we introduce the Gibbs potential,

-G(p|J ; α) = max h    log   A∈A L exp(-H(A|h, J ; α))   - L i=1 q-1 a=1 h i (a)p i (a)    ,
(2.14) , which is the Legendre transformation of the Helmholtz free energy F = log Z. Therefore short-hand representations of the Legendre transformation and its inverse are:

G(p|J ; α) = F(h|J ; α) -h t p F(h|J ; α) = G(p|J ; α) + h t p
Here, p i (a) is a single-site frequency of H(α = 1), and p ∈ R qL is a vector representation. Note that the sum over the amino acid takes only 1 to q -1 due to the lattice-gas gauge choice in order remove redundant degrees of freedom (we will discuss the gauge invariance in section 2.4.2). Since h i (a) and p i (a) are connected by the Legendre transformation,

h i (a) = ∂G(p) ∂p i (a) , p i (a) = F(h) ∂h i (a) , (2.15)
we also have,

(C -1 ) ij (a, b) = ∂h i (a) ∂p i (b) = ∂ 2 G(p) ∂p i (a)p i (b) . (2.16)
Here, C is the connected correlation matrix,

C ij (a, b) = f ij (a, b)-f i (a)f j (b),
which has L(q -1) dimensions and indices i, j ∈ 1:L, a, b ∈ 1:(q -1), this restriction is needed to make C be as invertible matrix (practically it needs pseudo-count, see the following section). For simplicity, we neglect the explicit dependence on J , α in Eq. 2.15.

When we expanding the Gibbs potential up to the first order with respect to α, we get,

G(p|J ; α) = G(p|J ; 0) + ∂G(p|J ; α) ∂α α=0 α + O(α 2 ) = G(p|J ; 0) + i<j J ij (A i , A j ) α=0 α + O(α 2 ) .
(2.17)

If we expand the Eq. 2.17 around α = 0, i.e., the profile model, then put G(p|J ; α = 1), we get,

G(p|J ) = L i=1 q a=1 p i (a) log(p i (a)) - i<j q a,b=1 J ij (a, b)p i (a)p j (b) .
(2.18)

Here, we put also p i (q) = 1 -q-1 a=1 p i (a). The aim is to find the selfconsistent equation for the parameters {h i (a)}. For this purpose, we perform the first and second partial derivatives of this first-order expansion of the Gibbs potential Eq. 2.18 with respect to the single point frequency p i (a). We find self-consistent equations for the model parameter h i (a):

p i (a) ∝ exp h i (a) + j∈∂i q b=1 J ij (a, b)p j (b) , (2.19)
which take the characteristic form of mean-field equations. We further determine the inverse of the connected correlation matrix as,

(C -1 ) ij (a, b) = -J i,j (a, b) for i = j δ a,b p i (a) + 1 p i (q)
for i = j .

(2.20)

Therefore, the inference problem for the couplings J ij (a, b) can be solved by just plugging the empirical connected correlation C into the Eq. 2.20, and by computing the inverse matrix. Surprisingly, the simple inverse of the covariance matrix gives accurate contact prediction. The inverse of the covariance matrix is probably the inherent quantity (there is a discussion why such a simple method can correctly deduce direct-interaction based on [44] in Chap. 5). Note that the covariance matrix is not a full rank matrix, in fact there are L zero eigenvalue modes 4 . The standard remedy for the rank deficient is a gauge transformation (see the technical section 2.4.2). However, even the reduced covariance matrix tends to be rank deficient, and it happens for sure if the number of the sequences is smaller than the (q -1)L, since

C = M -1 M m=1 (δ m -δ )(δ m -δ ) t , (2.21)
where δ is a vector representation of the single-site observables,

δ m = (δ A m 1 1 , . . . , δ A m L q ) t .
The pseudo-count method, which adds a small positive value to empirical counts is normally used for avoiding the rank deficient issue [42] (the detail is discussed in the technical section 2.4.1).

4 q a=1 Cij(a, b) = q b=1 Cij(a, b) = 0, ∀i, j ∈ {1, .
. . , L}, therefore we can easily construc L different zero modes, (0 . . . , 0, 1, 0, . . . , 0) t =L ⊗ (1, . . . , 1) t =q 2.3.2 Pseudo-likelihood maximization DCA plmDCA can accurately predict contacts with a small computational time, thereby out-performing mfDCA [45].

The first use of the pseudo-likelihood method for Markov Random fields was in 2010 [46], but it used an Ising model. Then the idea was adapted to protein sequence modeling in 2011 [47]. The objective function of plmDCA is not the likelihood for the pairwise distribution. Instead, it maximizes the log-likelihood of conditional single-site distributions, which we refer to as pseudo-likelihood function,

p(A m i = a|A m \i ) = exp(h i (a)) + j∂∈i J ij (a, A m j ) q b=1 exp(h i (b)) + j∂∈i J ij (b, A m j ) , (2.22)
with A \i = (A 1 , . . . , A i-1 , A i+1 , . . . , A L ). In consequence, the pseudo-likelihood function can be written as,

l plm (h, J ) := L i=1 l plm i (h i , J i ) , (2.23)
where the local pseudo-likelihood is given by

l plm i (h i , J i ) = M -1 M m=1 log p(A m i |A m \i ) = q a=1 f i (a)h i (a) + j∂∈i q a,b=1 f ij (a, b)J ij (a, b) -z i , (2.24)
where z i is a position-specific normalization constant, depending on A m \i .

z i := M -1 M m=1 log q b=1 exp(h i (b)) + j∂∈i J ij (b, A m j ) . (2.25)
It is known that as the number of sequences increases, the estimators of plmDCA for the Gibbs distribution become closer to the estimator of the full likelihood [48] 5 . To find the optimal solution for the pseudo-likelihood, there are two ways of strategies. The first approach is to maximize the l plm (h, J ) directly, this method is called symmetric PLM [49]. A solution of the pseudo-likelihood estimator is,

{h * , J * } = argmax h,J l plm (h, J ) .
(2.26)

An alternative approach is to maximize each local likelihood l plm i (h i , J i ), separately ∀i ∈ 1:L. Note that this method gives asymmetric couplings since the local likelihoods, given:

l plm i (h i , J i ) → J * i
ij and l plm j (h j , J j ) → J * j ij . These couplings results are generally different from each other. In practice, an arithmetic average of the two couplings is used for contact prediction. Therefore, the final solution should be written as

J * ij = 1 2 (J * i ij + J * j ij ) . (2.27)
This approach is called asymmetric PLM [45]. Its computational cost is smaller than that of symmetric PLM.

Recently, another related method called auto-regressive DCA (arDCA) was introduced. Although arDCA and plmDCA are closely associated with each other, the plmDCA is not a generative model, while arDCA is a generative model. In the next section we will briefly review arDCA.

Autoregressive DCA

Auto-regressive DCA (arDCA) was introduce in 2021 [START_REF] Trinquier | Efficient generative modeling of pro-tein sequences using simple autoregressive models[END_REF]. The key idea of the arDCA is to factorize the joint probability into conditional probabilities based on Bayes decomposition, which means

p(A|h, J ) = p(A 1 |h 1 ) L i=1 p(A i |A 1:(i-1) ; h i , J ) , (2.28)
where A 1:(i-1) = (A 1 , . . . , A i-1 ). The mathematical form of each factor is defined as

p(A i = a|A 1:(i-1) ; h i , J ) = exp( 1≤j<i J ij (a, A j ) + h i (a)) q b=1 exp( 1≤j<i J ij (b, A j ) + h i (b)) , (2.29)
and p(A 1 |h 1 ) is a standard profile distribution. The arDCA model gives astonishing results; the contact prediction is almost as good as plmDCA, which is the currently best contact prediction method. Furthermore, arDCA can be used as a generative model, and the reproduction of the statistics is as good as bmDCA. One of the significant differences from bmDCA is that the computational time is 100 to 1000 times faster due to the accessibility of the normalization rigorously and fastly. There were many situations in which the application of DCA was limited due to the time-consuming sampling using conventional generative models. Therefore, arDCA will be definitely useful to sample sequences in a short time and at a large scale.

In the next section, we introduce another mean-field approach, which has pairwise couplings but effectively reduces the number of model parameters by applying the low-rank representation of couplings.

Mean-field Hopfield-Potts DCA

The method of mean-field Hopfield-Potts DCA was introduced in 2013 [START_REF] Cocco | From principal component to direct coupling analysis of coevolution in proteins: Loweigenvalue modes are needed for structure prediction[END_REF]. This method can effectively reduce the number of model parameters by introducing a low-rank representation of the coupling matrix. Each non-zero mode of the coupling matrix is called patterns, and is associate with an eigenvector of Pearson correlation matrix in the mean-field approximation. These patterns are also closely related to position-specific scoring matrices, which are defined as characteristic sequence motifs as explained in Sec. 1.2.2.

While the dimensionality of the covariance matrix is L(q -1), the number of essential dimensions is much smaller. These inherently significant directions are called patterns as the following q × L matrix ξ = {ξ i (a)}, with i ∈ 1:L and a ∈ 1:q .

The log-score of a sequence

S(A|ξ) = L i=1 ξ i (A i ) 2 .
(2.30)

Within the Hopfield-Potts model, the statistical modeling of protein sequences A is characterized by the combination of P log-scores, given by P patterns:

p(A) = 1 Z exp   1 2L P + µ=1 S(A|ξ +,µ ) - 1 2L P - ν=1 S(A|ξ -,ν )   . (2.31)
The patterns denoted as ξ +,µ with pattern index µ ∈ 1:P + and ξ -,ν with pattern index ν ∈ 1:P -(= P -P + ) are called attractive-pattern and repulsivepattern, respectively. The log-likelihood function of the mean-field Hopfield-

Potts model leads 6 L {ξ +,µ , ξ -,ν } µ,ν |A 1:M = ia f i (a) log f i (a) + 1 2L P + µ=1 ia,jb f ij (a, b)ξ +,µ i (a)ξ +,µ j (b) - 1 2L P - ν=1 ia,jb f ij (a, b)ξ -,ν i (a)ξ -,ν j (b) + 1 2 P + µ=1 log 1 - 1 L i,a f i (a)ξ +,µ (a) 2 + 1 2 P - ν=1 log 1 - 1 L i,a f i (a)ξ -,ν (a) 2
.

(2.32)

The Eq. 2.31 corresponds to the specific case of PPM Eq. 2.8, with the couplings J ij (a, b) defined as

J ij (a, b) = 1 L P + µ=1 ξ +,µ i (a)ξ +,µ j (b) - 1 L P - ν=1 ξ -,ν i (a)ξ -,ν j (b) . (2.33)
As shown in the Eq. 2.33, the attractive pattern correspond to the P + largest eigenvalues (λ + 1 ≥ λ + 2 ≥ . . . ≥ λ + P + > 1) while the repulsive patterns correspond to the smallest P -eigenvalues (0 < λ - 1 ≤ . . . ≤ λ - P -).

The likelihood function in Eq. 2.32 can also be expressed as a function of eigenvalues {λ µ },

L[A 1:M ] = i,a f i (a) log f i (a) + 1 2 P ± µ=1 (λ ± µ -1 -log λ ± µ ) . (2.34)
Significant findings in [START_REF] Cocco | From principal component to direct coupling analysis of coevolution in proteins: Loweigenvalue modes are needed for structure prediction[END_REF] are the following: First, the attractive patterns are associate with the largest eigenvectors. On the other hand, the repulsive patterns associate with the smallest eigenvectors. Second, the dominant contributions on the likelihood are the patterns associated with the largest or smallest eigenvectors. Last but not least, the attractive patterns capture the sequence heterogeneity. On the other hand, the repulsive patterns tend to learn residue contact information.

Technical Points

In this section, we will show some practically necessary treatments for learning the DCA-based statistical models.

Regularization

Let us assume that there is no observation for certain frequencies f ij (a, b) = 0 with f i (a)f j (b) = 0. Suppose we impose the frequentist point of view in such a situation. In that case, we need to introduce a negatively infinite conjugate parameter, J ij (a, b) → -∞ into our statistical model, which causes unstable learning clearly. The standing point of Bayesian statistics, which is the opposite to the frequentist, tends to assume that the model parameter θ of a probability distribution p(x|θ) is also probabilistic and is governed by a prior distribution p(θ|α), where α is known as a so-called hyper-parameter.

Here we recall Bayes' theorem, p(θ|x) ∝ p(x|θ)p(θ).

(2.35)

In the context of the above mentioned problem, we can assume that the model parameters {J ij (a, b)} and {h i (a)} come from independent Gaussian distribution, J ij (a, b) ∼ N (0, λ -1 J ) and h i (a) ∼ N (0, λ -1 h ), hence we get another objective function reflecting the philosophy of Bayesian statistics,

l DCA (h, J ) ← l DCA (h, J ) + λ J J 2 + λ h h 2 , (2.36)
where l DCA is the log-likelihood function defined in Eq. 2.9, and A 2 is a L2 norm, and λ J , λ h are hyper-parameters 7 . The estimator using the Eq. 2.36 is called Maximum a posteriori estimation (MAP). There is a simple criterion for the hyper-parameter of DCA learning that states that λ J ∼ L/M and λ h ∼ 1/M , where M is the number of sequences in the training data [START_REF] Rizzato | Inference of compressed potts graphical models[END_REF]. In practice, we use several hyper-parameters for the couplings such as λ J ∈ {10 -4 , 2 • 10 -4 , . . . , 10 -3 }. Typically the value used 7 √ λJ and √ λ h are the standard deviation values of the Gaussian distributions N (0, λ -1 J ) and N (0, λ -1 h ) .

for the hyper-parameter depends strongly on the protein family, and careful tuning is needed. There exist different methods to regularize, such as L1 regularization, which corresponds to assuming the Laplace distribution as a prior distribution of the parameters. One of the advantage to use the L1 regularization is that it can sparsify the couplings [START_REF] David | Psicov: precise structural contact prediction using sparse inverse covariance estimation on large multiple sequence alignments[END_REF]. However, the L1 norm makes small parameters to zero even if they are statistically significant, as a consequence, it tends thus to lead to a poor generative model.

Another regularization method we use frequently is pseudo counts,

f i (a) ← (1 -α)f i (a) + α q , f ij (a, b) ← (1 -α)f ij (a, b) + α q 2 .
(2.37)

This method is typically used for mfDCA, and also we used it to investigate sparse Boltzmann machines (cf. Chap. 3). With a similar discussion of the relation between the L2 regularization and the Bayesian modeling, pseudo counts appear if we assume a Dirichlet distribution as a prior distribution of frequency counts. [14] .

Gauge Invariance and Gauge Transformation

In the case of DCA, the number of model parameters and the number of observables are exactly the same, qL + q 2 L(L -1)/2. However, the observables are not independent: The single site frequency are normalized q a=1 f i (a) = 1, and

q b=1 f ij (a, b) = f i (a)
, ∀i, j ∈ 1:L and ∀a, b ∈ 1:q. Therefore, there are L + qL redundant parameters. The number of independent conditions is (q -1)L + (q -1) 2 L(L -1)/2. In order to avoid an over-parameterization, we can reduce the number of parameters keeping the energy function invariant. This fact is known as a gauge invariance in physics, and to reduce redundant model parameters is known as a gauge choice [42,37]. Eq. 2.8 is invariant under the following transformations,

J ij (a, b) ← J ij (a, b) + J ij (a) + K ij (b) + c ij h i (a) ← h i (a) - j(>i) J ij (a) - j(<i) K ji (a) -H i , (2.38)
where J ij (a) and K ij (b) are arbitrary functions.

In DCA, we typically use lattice-gas gauge:

J ij (a, q) = J ij (q, b) = h i (q) = 0 , (2.39)
for all i, j ∈ 1:L and a, b ∈ 1:(q -1) [42].

Another frequently used gauge choice is zero-sum gauge or Ising gauge:

q a=1 J ij (a, b) = q b=1 J ij (a, b) = q a=1 h i (a) = 0, (2.40)
for all i, j, a, b. This gauge can be achieved using the transformations,

J ij ← J ij (a, b) -J ij (a, •) -J ij (•, b) + J ij (•, •) , h i (a) ← h i (a) -h i (•) + j( =i) (J ij (a, •) -J ij (•, •)) , (2.41)
where we denote g(•) as the average q -1 q a=1 g(a). It is easy to check that the zero-sum gauge minimizes the Frobenius norm of couplings, a fact exploited in DCA-based residue contact prediction (cf. Sec. 2.5.1). Here is a sketch of proof: Substitute the general form of the gauge transformation into the definition of the Frobenius norm, then take the functional derivative with respect to {J ij (a), K ji (a), H i }, note that the function of Frobenius is a convex function of these gauge variables. Therefore the set of variables that gives zero gradients can minimize the Frobenius. The zero-sum gauge satisfies this condition.

Phylogenetic Correction

Most inference methods assume that samples in the training data are independently and identically distributed (i.i.d.). However, this assumption is not true in general. In particular, ensembles of protein sequences or MSAs typically strongly depend on each other. Therefore they are not independent at least. This strong correlation among samples comes from the fact that sequences are evolutionarily related to each other, and this relation is known as the phylogenetic tree in evolutionary biology. Another reason is the selection bias of sequencing: some specific species such as E. coli are more frequently sequenced due to medical or academic interests.

Sequence reweighting -We use a simple reweighting method to reduce such biases [42]. First, we introduce a similarity threshold 0 < x < 1: If the number of identical residues between two sequences is greater than xL, these sequences are assumed to be carrying almost the same information. On the contrary, sequences that have smaller sequence similarities are considered to carry substantial information. Formally, the reweighting parameter for a sequence ensemble among the 1:M sequences can be written as

w m = M n=1 Θ(d mn < 1 -x) -1 , (2.42)
where d mn is the normalized Hamming distance between A m and A n , and Θ(c) is the Heaviside function that gives 1 if and only if the condition c is true, and 0 otherwise. Therefore, the more similar a sequence is to other sequences, the smaller its weight will be. Normally, we use x = 0.8, and the result of contact prediction is robust between 0.7 ≤ x ≤ 0.9 [42].

The single and pairwise frequencies should be modified accordingly,

f i (a) = 1 M ef f M m=1 w m δ A m i ,a f ij (a, b) = 1 M ef f M m=1 w m δ A m i ,a δ A m j ,b , (2.43)
where

M ef f = M m=1
w m , denotes the effective number of sequences.

Averaged Product Correction -The APC is a method to remedy the bias due to background noise and phylogenetic effect on the Frobenius norms of couplings. Initially, this method was applied to MI [33], then it was applied to DCA-based contact predictions.

Here, we show a derivation of APC based on Burger et al. [41]. The APC is given by

F AP C ij = F ij - F i * F * j F * * , (2.44)
where

F i * = 1 (L-1) L j( =i) F ij and F * * = 2 L(L-1)
L i<j F ij . The above Eq. 2.44 can be derived by assuming that the noise B ij that is in the observable pairwise interaction, factorizes into B i B j . Therefore the pairwise signal can be written in this way,

F ij = F true ij + B i B j ,
where F true ij is Frobenius norm removing the background noise. By summing over indices, we get following relations

F i * = F true i * + B i B * F * * = F true * * + B 2 * .
(2.45)

Assuming that the true pairwise signal is smaller than noise, we find that F i * ∼ B i B * and F * * ∼ B 2 * . We can thus remove the noise from the true pairwise signal and get,

F true ij = F ij - F i * F * j F * * . (2.46)
The above argument explains why APC works. There is another interesting method based on entropy correction [START_REF] Vorberg | Synthetic protein alignments by ccmgen quantify noise in residue-residue contact prediction[END_REF], which leads to very similar results.

Learning by Contrastive Divergence and Persistent Contrastive Divergence learning

Contrastive Divergence -Contrastive divergence (CD) is a point estimation method that can be applied to a broad class of inference problems, including exponential families (bmDCA is in this class), especially efficient for learning restricted Boltzmann machines (RBM). Initially, the CD method was proposed to reduce the vast computational time of learning statistical models. The initial attempt was based on the products of experts (PoE) model by Geoffrey E. Hinton [START_REF] Hinton | Training products of experts by minimizing contrastive divergence[END_REF], where the PoE is a latent variable model closely related to the mixture model.

The key idea of the CD method is to replace the sample average in the gradient of the log-likelihood with an average overs CD samples, which is computationally easily attainable. Here, the CD sample is an instance after k steps of MCMC initialized in the training data. Therefore, the size of the CD ensemble is exactly one of the training data. Typically, k is a small number, and even k = 1 is quite frequently chosen. Note that if k is sufficiently large, the CD method is converged to the conventional MCMC based learning.

An intuition of the CD learning success is as follows: The empirical distribution and the model distribution might be close. Hence, a few MC steps from the data points are assumed to be reasonable approximations of the equilibrium distribution of the model probability (in other words, start from one of the training data means starting from the almost equilibrated state in MCMC). Note that the principle of CD learning is different from ML learning. The policy of CD learning corresponds to the minimization of the transition probability flows escaping from the training data points (a similar idea can be found in Minimum Probability Flow learning [START_REF] Sohl-Dickstein | Minimum probability flow learning[END_REF][START_REF] Ahmed A Quadeer | Mpf-bml: a standalone gui-based package for maximum entropy model inference[END_REF]).

PCD has an algorithmically similar idea to CD learning. However, the objective function that becomes effectively more similar to ML learning is persistent contrastive divergence (PCD) learning. Empirically, PCD learning gives more stable and accurate results than CD learning in our study. The first epoch is precisely the same as the CD learning. Each sample is initialized by training data and update a few times. The difference will be more noticeable after the second epoch: The last states at the previous epoch are used as the initial states of the next epoch. Such as initialize a sample state at the 2nd (3rd) epoch by an updated state at the 1st (2nd) epoch.

More detailed descriptions about CD-based learning are discussed in the Appendix E.

Batch learning -Some MSAs contain many sequences (about 10 5 sequences for PF00072), which cause a very long learning time for CD-based learning. In this case, we use mini-batch learning, a powerful learning method to reduce computational time to estimate gradients of the objective function: we divide the MSA (M sequences) to sub-ensembles that contain B(< M ) sequences used for CD learning. Using smaller B will make the calculations faster, but increases the statistical error of the estimated gradients due to the reduction of sample size. Hence there is a tradeoff between accuracy and computational time. It is also closely related to the optimal learning rate. If B is large, the gradient can be calculated accurately, so a large learning rate may also be used. On the contrary, for small B, one needs to use small learning rates.

Other learning techniques

Hyper-parameter optimization -Even simple DCA techniques rely heavily on hyperparameters such as learning rates and the strength of regularization (typically, η J = 0.001 -0.1 and λ J = 0.001 -0.01, respectively). Empirically, generative models require particularly careful finetuning. What we normally use is grid search, a simple but exhaustive hyperparameter searching algorithm. For example, find the best possible com-bination among η J ∈ {0.001, . . . , 0.1}, λ J ∈ {0.001, . . . , 0.01} in the above example.

Assessment of learning -It is crucial to evaluate the learning process properly, to avoid excessively long learning times. For this aim, one can use the Pearson correlation of two-point connected correlations, or the maximum difference of correlations between the training data and data sampled from the model. The evaluation of CD-based learning is particularly important, and the convergence test must be made in consideration of the statistical fluctuations involved in the gradient (see also Appendix E.3).

Applications

Predicting residue-residue contacts and protein

Initially, DCA was proposed to predict residue contacts in proteins using only sequence information [42,[START_REF] Marks | Protein 3d structure computed from evolutionary sequence variation[END_REF]49,[START_REF] Kamisetty | Assessing the utility of coevolution-based residue-residue contact predictions in a sequence-and structure-rich era[END_REF][START_REF] Weinreb | 3d rna and functional interactions from evolutionary couplings[END_REF].

Here, we typically use 8 Å as a condition for two residues to be in "contact". If a relative distance between residues i and j is equal or less (greater) than this threshold, ≤ 8 Å(> 8 Å), we regard the pair as "contact" ("non-contact"). We represent it by a contact matrix, a binary matrix D ∈ {0, 1} L×L , results D ij = D ji = 1 ("contact"), otherwise 0 ("noncontact").

Accurately predicting such a contact maps is extremely helpful for 3Dprotein structure prediction, and it has been reported that the minimum number of necessary true contact predictions to reconstruct protein structure correctly is 8% -25% [START_REF] Vassura | Blurring contact maps of thousands of proteins: what we can learn by reconstructing 3d structure[END_REF][START_REF] Bittrich | Structuredistiller: Structural relevance scoring identifies the most informative entries of a contact map[END_REF] in a contact map. Therefore, structure prediction methods aim to minimize the difference between the native contact map and a computationally predicted contact map. Due to the availability of many experimentally determined protein structures, "true" contact maps are known for many proteins and can be used for testing prediction methods.

For DCA-based contact prediction, coupling matrices J ij ∈ R q×q are converted to a Frobenius norm F ij ∈ R, ∀i, j ∈ 1:L,

F ij = q a,b (J ij (a, b)) 2 .
(2.47)

As mentioned in Sec. 2.4.2, the zero-sum gauge is typically used for computing the Frobenius norm. It provides a real-scalar-valued score to a residue pair and can be used for contact prediction. In general, the APC score, the matrix after applying the APC correction in Eq. 2.44 to the Frobenius norm, is used.

To quantitative assess the accuracy of residue contact prediction, we commonly use the positive predictive value (PPV), which is defined as the number of true positives divided by the number of the total predictions8 . To consider only non-trivial contacts, we exclude the effects of backbone interactions due to peptide bindings such as |i-j| < 5. Therefore, the number of total prediction is n * = L(L -5)/2) .

In practice, we sort the (APC) score in descending order and represent the PPV as a function of the sorting rank, n. We represent the sorting of residue pairs as a function of the sorting rank as following,

σ(n) ∈ {(i, j) | ∀i ∈ 1:(L -5), ∀j ∈ 6:L, i < j} , with, S σ(1) ≥ S σ(2) , ≥, . . . , ≥, S σ(n * ) ,
where S σ(n) is a score ranked n-th. Note that the considering residue pairs are those satisfying |i -j| ≥ 5. Thereby, the PPV curves can be written as a function of the rank n ∈ 1:n * ,

P P V (n) = n m=1 D σ(m) /n , (2.48)
The rationale for this constraint is that the alpha helix turns back to the same position after four consecutive amino acids. The result of the contact prediction was shown in Fig. 2.2.

Sequence scoring

Sequence scoring -This problem is one of the most classical bioinformatics questions. As we saw in section 1.2.3, by marginalizing the hidden state of a profile HMM, we can estimate the likelihood function and evalu-ate the probability of any sequence. It allows us to estimate the probability that a sequence belongs to a particular sequence group (such as a homology group). Similarly, DCA-based statistical models provide a probability. Therefore it must be possible to use it as a sequence scoring method. In particular, the DCA-based method can consider the pairwise couplings. Hence the scoring system with DCA can regard coevolution effects or epistasis effects. Technically, a log probability of the pairwise distribution, which is essentially the negative of generalized Potts energy function, is employed as the sequence score.

Indeed, it was found that scoring systems considering pairwise interactions are inherently important for decoding amino-acid sequence information. In [START_REF] Socolich | Evolutionary information for specifying a protein fold[END_REF], an experiment was conducted to verify the role of pairwise interactions for functional artificial sequences of WW-domains (Pfam ID PF00397). They revealed that the pairwise interactions between amino acids are essential for proteins to be folded into the native structures. More specifically, they generated artificial amino-acid sequences from models considering different statistical features such as one-point and two-point frequencies of an MSA of natural sequences. They examined whether the artificial sequences are able to fold into the native structure.

The models to consider are:

• Natural (N): Natural WW sequences, selected randomly from the MSA (we hereby name the natural sequences as Nat sequences).

• Random (R): Random sequences, where each amino-acid at each site is randomly drawn from a random distribution that keeps only the mean frequency of the MSA, A i ∼ f (a).

• Independent conservation (IC): Site-independent conservation sequences, each amino acids at each site are drawn from the single-site distribution of the MSA, A i ∼ f i (a). Therefore, there is no statistical coupling between residues.

• Coupled conservation (CC): Sequences that keep both single-site and pairwise frequency, (A i , A j ) ∼ f ij (a, b). Sequences are generated by swapping an amino-acids in the same MSA while keeping pairwise frequencies. Specifically, this calculation was performed using MC sampling and a simulated annealing (SA) algorithm. The upper rights panel in Fig. 2.4 right shows energy distributions based on a PPM using the Adaptive Cluster Expansion (ACE) [START_REF] John P Barton | Ace: adaptive cluster expansion for maximum entropy graphical model inference[END_REF], a precise approximation of the pairwise-Potts model. It should be noted here that DCA relies on pairwise information, so it can provide an indicator that distinguishes between random, independent-site, and sequences considering 9 The MC sampling with the same construction takes a very long convergence time. As the number of sequences and the sequence length increase, it becomes more difficult to converge the equilibrium distribution due to the funnel-like landscape of free energy. Therefore, it is better to treat pairwise statistics using a statistical model for technical reasons as well.

the pairwise interactions.

Interestingly, whether the sequences can be folded or not is strongly correlated with the DCA energy. The bottom in Fig. 2.4 shows the energy values for each N, R, IC, and CC sequences used in the experiment [START_REF] William P Russ | Natural-like function in artificial ww domains[END_REF], the red and gray bands are assigned to the foldable and non-foldable sequences. The energy values for CC, IC, and R correspond to the energy histogram (the top panel) Potts energy, site-independent energy, and randomized energy, respectively (note, these sequences are generated differently as in [START_REF] William P Russ | Natural-like function in artificial ww domains[END_REF]).

The remarkable thing is that sequences with lower energies are more likely to hold the native structure, with a significant increase in the percentage of sequences that fold below a certain energy threshold. It suggests that the pairwise-Potts energies can reasonably distinguish between foldable-and non-foldable-sequences. This experiment also implies the possibility of the PPM to generate sequences with high-folding probability, meaning that the model can be used not only for scoring but also for generating functional protein sequences.

Protein sequence design

A direct experiment to learn artificial functional protein sequences based on DCA has been conducted recently [START_REF] William P Russ | An evolution-based model for designing chorismate mutase enzymes[END_REF]. This experiment successfully designs artificial protein sequences of chorismate mutases (CMs), a standard model protein, to investigate principles of catalysis and enzyme design using bmDCA model. CM acts as a catalyst and may accelerate the rate of a reaction required for cell growth by more than 1 million times. It is essential for bacterial growth in a minimal glucose of medium. In order to monitor CMs activity in an in vivo complementation assay, this experiment employed CM-deficient E. coli cells, and adapted selective conditions (lacking Phe and Tyr, which are products of the reactions requiring CM catalysis) in the minimal medium. It is known that E. coli strains lacking CM are auxotrophic to Tyr and Phe, and both the degree of supplementation of these amino acids and the expression level of the reintroduced CM gene quantitatively determines the growth rate. In other words, the more the CM genes are expressed, the easier it is to grow even in a medium with low Tyr and Phe concentration. Hereafter, we denote E. coli that express CM genes as EcCM.

Around 1130 protein sequences were included in the MSA that was used for learning a bmDCA model that can precisely reproduce one-point frequen-cies and two-point connected correlations in the MSA. Then more than 1900 sequences are generated under the several temperatures T ∈ {0.33, 0.66, 1.0}, where the temperature is a parameter rescaling model parameters globally and linearly. Therefore MC sampling is done using the following probability distribution,

p(A) ∼ exp 1/T i h i (A i ) + i<j J ij (A i , A j ) .
To understand, which natural and artificial sequences are functional the authors introduce the relative enrichment (r.e.) defined as the difference of log ratio of frequencies of enrichment in the population before and after selection. A schematic explanation of the pipeline of the experiments is also shown in Fig. 2.5.a.

r.e. = log

f sel (A) f inp (A) -log f sel (A ref ) f inp (A ref ) , (2.49)
where A ref is the sequence for EcCM, and the subscripts sel, inp are the set of selected and input samples, respectively. If the relative population size of the selected sequences is not changed, the r.e. becomes around zeros. On the contrary, the r.e. gives a positive value if the population of the selected sequence increases significantly. Fig. 2.5.c shows that the DCA energy (which is denoted as statistical energy in the original paper) is strongly correlated with the functionality of the r.e.. Sequences should have a low energy to realize functional sequences. There is a threshold in DCA energy above which only non-functional sequences exist.

As the results of the experiment, Fig. 2.6 shows that the sequences corresponding with low-energy drawn from T ∈ {0.33, 0.66} typically recapitulate the qualities of natural sequences. On the contrary, sequences from T = 1 poorly performed to recapitulate natural sequences. Overall, among the 1618 total artificial sequences, 481 sequences (∼ 30%, norm r.e. > 0.42) rescued growth in this experiment, although the top-hit sequences identities to any natural CM was between 42 and 92%. Even 46 sequences are functional with < 65% sequence identity to the natural sequences.

Fitness landscape

A fitness landscape can provide information about the genotype-phenotype evolutionary relation, including thermodynamic stability as well as the dy- [START_REF] Helen C Davison | What is antibiotic resistance and how can we measure it?[END_REF]. The five highest-scoring mutations shown in the figure (M182T, H153R, I247V, T265L/Q, and N276D) are reported as stabilizing mutants (red crosses). (Source [START_REF] Figliuzzi | Coevolutionary landscape inference and the context-dependence of mutations in beta-lactamase tem-1[END_REF]) namic process of protein folding [3]. Prediction of the fitness landscape has attracted great attention in engineering and biomedical applications due to the possibility to design proteins that have acquired properties different from those of natural species while maintaining the desired function (e.g., enzymatic function, increased stability, etc).

Particularly prediction of mutation (e.g., stability, types of potentially prospering mutant), which can be assessed by comparing the fitness values between a wild type and its variants (mutant), has been accelerated for the last five years thanks to the accurate high-throughput mutation data or deep mutational scanning [START_REF] Douglas | Deep mutational scanning: a new style of protein science[END_REF].

The DCA energy function takes into account the effects of conservation and co-evolution and can provide a protein sequence that folds into a naturally occurring structure with high probability, as we saw previously Sec. 2.5.3. From this consideration, the energy function of DCA is used as a sequence fitness landscape function, and the mutational landscape function can be predicted in terms of the DCA energy function,

∆E(A wt → A mut ) = H(A mut ) -H(A wt ) , (2.50)
where, A wt and A mut are sequences of wild type and mutant, respectively. Fig. 2.7 shows one of the remarkable results in [START_REF] Figliuzzi | Coevolutionary landscape inference and the context-dependence of mutations in beta-lactamase tem-1[END_REF] using the large-scale mutagenesis data of beta-lactamase (TEM1), a model enzyme that provides resistance against beta-lactam antibiotics. In this case, mutations considered were only single-site mutations. For the quantitative comparison to the computational prediction, the antibiotic's minimum inhibitory concentration (MIC) was used [START_REF] Helen C Davison | What is antibiotic resistance and how can we measure it?[END_REF].

This DCA-based fitness-landscape prediction has been applied, and its effectiveness has been recognized in many cases such as protein design and stability prediction [START_REF] Pengfei Tian | Co-evolutionary fitness landscapes for sequence design[END_REF], identification of drug resistance mutation in HIV [START_REF] Thomas | Identification of drug resistance mutations in hiv from constraints on natural evolution[END_REF], drug resistance-associated mutation in HIV-1 protease [START_REF] Zhang | Predominance of positive epistasis among drug resistance-associated mutations in hiv-1 protease[END_REF], virus escaping time and mutation-location prediction [START_REF] John P Barton | Relative rate and location of intra-host hiv evolution to evade cellular immunity are predictable[END_REF], antiviral drugs design [START_REF] Bravi | Direct coupling analysis of epistasis in allosteric materials[END_REF], and vaccine design [START_REF] Mann | The fitness landscape of hiv-1 gag: advanced modeling approaches and validation of model predictions by in vitro testing[END_REF][START_REF] Kaczorowski | Data-driven strategies for vaccine design[END_REF].

Other applications

Protein-protein interaction networks -Proteins rarely function alone in vivo and work with other proteins to realize functions such as metabolisms, transport, making components, etc. Such relationships between proteins are known as protein-protein interaction (PPI) networks, consisting of proteins as nodes and interactions between proteins as edges. Understanding PPI networks is a fundamental question in systems biology. PPI networks are also useful and give insight for understanding mechanisms of complex systems, such as identifying genes and proteins that are associated with disease using an assumption that neighbors of "disease genes" in PPI networks typically relate with similar diseases [START_REF] Sevimoglu | The role of protein interaction networks in systems biomedicine[END_REF][START_REF] Nguyen | Detecting disease genes based on semi-supervised learning and protein-protein interaction networks[END_REF][START_REF] Goh | The human disease network[END_REF][START_REF] Winter | Google goes cancer: improving outcome prediction for cancer patients by network-based ranking of marker genes[END_REF], to understand metabolism [START_REF] Bánky | Equal opportunity for low-degree network nodes: a pagerank-based method for protein target identification in metabolic graphs[END_REF], and to find proteins that can be influenced by certain chemical compounds [START_REF] Muhammed | Drug-target network[END_REF].

Ref. [START_REF] Feinauer | Inter-protein sequence co-evolution predicts known physical interactions in bacterial ribosomes and the trp operon[END_REF] shows that DCA can be also used for PPI predictions and successfully infers underlying PPI networks. Training data is provided as N MSAs for each protein family. An MSA contains M p sequences of length L p in each alignment D p for all of considering families p. The task of PPI network prediction can be decomposed into two steps:

1. Matching procedures (concatenate sequences): For applying DCAbased interaction networks prediction, it is necessary to concatenate sequences in two putative co-evolutionary related families (p, p ) in MSAs (D p , D p ). DCA learning is done on a new alignment D (p,p ) of sequence length L p + L p . Ref. [START_REF] Feinauer | Inter-protein sequence co-evolution predicts known physical interactions in bacterial ribosomes and the trp operon[END_REF] employed a matching algorithm based on the genomic distance between sequences.

Inference procedures (execute DCA learning):

The DCA learning using two concatenated sequences (A, A ) can be formulated as the following joint distribution,

p(A, A ) = 1 Z exp -H(A) -H (A ) -H int (A, A ) H int (A, A ) = - Lp i=1 L p j=1+Lp J ij (A i , A j ) , (2.51)
where H(A) and H (A ) are the standard DCA energy function in Eq. 2.8. The function H int (A, A ) is the function that regards coevolutionary couplings between the two families (p, p ). Ref. [START_REF] Feinauer | Inter-protein sequence co-evolution predicts known physical interactions in bacterial ribosomes and the trp operon[END_REF] used plmDCA method.

The DCA-based PPI prediction achieved 70% for small ribosomal subunit (SRU) and 90% for large ribosomal subunit (LRU) of the true positive rate among the top 10 predictions.

The authors discussed that these results can improve as the number of sequences available increases, which is likely to happen thanks to advances in next-generation sequencing technology.

Sequence pattern selection -Sequence motifs or position-specific scoring matrices (PSSM) are commonly used as representations of biological sequences to characterize ensembles of sequences such as functionality or phylogenetically related protein domains. Finding patterns of sequences to distinguish ensembles of data is a fundamental problem in bioinformatics.

It was found that the mean-field Hopfield-Potts model can construct patterns that are closely related to PCA vectors of the data covariance matrix in [START_REF] Cocco | From principal component to direct coupling analysis of coevolution in proteins: Loweigenvalue modes are needed for structure prediction[END_REF]. Although it is not a DCA-based method, there is another important method to find sequence motifs, statistical coupling analysis (SCA). SCA can The edge colors are assigned as follows: Green for true prediction. Red for false prediction (there is no such interaction in the experimental PPI networks). Gray for the interaction that is determined by experiment but not found in the prediction. (Source [START_REF] Feinauer | Inter-protein sequence co-evolution predicts known physical interactions in bacterial ribosomes and the trp operon[END_REF]) find other types of sequence motifs, which can distinguish groups of aminoacids that variate related to each other and can detect collectively evolving groups of positions, which are referred to as "protein sectors" [START_REF] Halabi | Protein sectors: evolutionary units of three-dimensional structure[END_REF][START_REF] Ranganathan | Note 109: A summary of sca calculations[END_REF][START_REF] Olivier Rivoire | Evolution-based functional decomposition of proteins[END_REF]. A similar method can be found in [START_REF] Grigolon | Identifying relevant positions in proteins by critical variable selection[END_REF]. It is based on a hierarchy of variables and aims to select statistically relevant residual variables.

Coupling parameters reveal biologically meaningful information -Indeed, today's best contact prediction methods rely heavily on deep neural networks, enabling more accurate three-dimensional structure prediction than other methods. The deep neural network (DNN) based methods may obtain impressive results, but a clear drawback of DNN is that it is hard to interpret millions of parameters and understand learning quantitatively. Concerning the interpretability problem of DNN, DCA-based algorithms provide clear relations among the statistical model, estimated parameters, and biophysical meaning while keeping almost the same performance with DNN.

Although the strong coupling parameter J ij (a, b) can be interpreted as biophysical interaction, strong couplings are extremely rare, and most of the remaining couplings are small, but some of them correspond with structural interaction [START_REF] Coucke | Direct coevolutionary couplings reflect biophysical residue interactions in proteins[END_REF]. It is desired to propose a method that allows the selection of statistically important but small couplings.

In the following section, we will report some statistical generative models of protein families. These models are assumed certain interaction architectures between residues to realize more biologically interpretable models, as shown in Fig. 2.9. Everything should be made as simple as possible, but no simpler.

-Albert Einstein (1879 -1955)

II

Variable selection for protein sequence modeling

Chapter 3

Specification of Sequences Statistics

Motivation

As shown in the previous chapter, DCA methods shed light on many biologically intriguing questions: realize probability distributions that can reproduce properties of biological sequence ensembles, predict residue-residue interactions, give an objective measure to score each sequence, reveal proteinprotein interaction networks, and generate functional artificial protein sequences.

DCA-based methods have succeeded in a wide range of studies. However, the model depends on a considerable number of parameters, typically 10 2 to 10 3 times greater than the number of sequences available for learning. Clearly, the model would be sensitive to statistical noise in the training data. Moreover, most of the estimated coupling parameters are very noisy. Especially, those with small coupling parameters are particularly dependent on their initial conditions (Ref. [START_REF] Barrat-Charlaix | Understanding and improving statistical models of protein sequences[END_REF] in section 6.1), which suggests that some of these small parameters are irrelevant. Though, other small couplings might come from properties of the protein family, such as physical contacts or the underlying phylogenies. Regarding the discrimination of relevant and irrelevant couplings parameters, we would like to address the following questions:

1. What is the minimum set of constraints required to reproduce the statistics of protein sequence ensembles within a generative model?

Can the minimally constrained models improve biological interpretability?

2. Is the minimally constrained model more robust than the standard fully connected pairwise Potts model (PPM)?

3. Can the minimally constrained model improve predictions of residues contacts and the effects of mutations that might be affected by statistical noise?

In order to address these questions, we propose a generative model that takes only into account the minimal set of constraints while minimizing the Kullback-Leibler divergence from the fully connected PPM. This method decimates redundant model parameters based on information-theoretic measures. This method can be applied to any problem that standard PPMs can handle.

Our work revealed that the remaining parameters after the decimations contain only statistically relevant parameters associate with 3D structures and/or phylogenies. As explained in the article (see the next section), the decimation protocol is different from both the color compression method in [START_REF] Rizzato | Inference of compressed potts graphical models[END_REF] and the pseudo-likelihood-based small coupling decimation in [START_REF] Decelle | Pseudolikelihood decimation algorithm improving the inference of the interaction network in a general class of ising models[END_REF].

Our contributions to this study are:

1. We found that we can decimate (remove) around 90% coupling parameters J ij (a, b) without decreasing the accuracy of residues contact predictions and reproducibilities of statistics (two-point and three-point connected correlations).

2. Our methods achieve lower densities of finite couplings than the other regularization methods (e.g., L1 regularization, empirical frequenciesbased decimation) without degrading the ability to reproduce statistics of the training data.

3. We found that by reducing the density of the coupling parameters, the model became more robust against small perturbations on the model parameters.

4. We also proposed a method for quantitatively estimating the difference between the contact and non-contact distribution. Using these measures, we found that the difference in contact and non-contact became more pronounced as the coupling densities decreased.

Article

Boltzmann machines (BM) are widely used as generative models. For example, pairwise Potts models (PM), which are instances of the BM class, provide accurate statistical models of families of evolutionarily related protein sequences. Their parameters are the local fields, which describe site-specific patterns of amino-acid conservation, and the two-site couplings, which mirror the coevolution between pairs of sites. This coevolution reflects structural and functional constraints acting on protein sequences during evolution. The most conservative choice to describe the coevolution signal is to include all possible two-site couplings into the PM. This choice, typical of what is known as Direct Coupling Analysis, has been successful for predicting residue contacts in the threedimensional structure, mutational effects, and in generating new functional sequences. However, the resulting PM suffers from important over-fitting effects: many couplings are small, noisy and hardly interpretable; the PM is close to a critical point, meaning that it is highly sensitive to small parameter perturbations. In this work, we introduce a general parameter-reduction procedure for BMs, via a controlled iterative decimation of the less statistically significant couplings, identified by an information-based criterion that selects either weak or statistically unsupported couplings. For several protein families, our procedure allows one to remove more than 90% of the PM couplings, while preserving the predictive and generative properties of the original dense PM, and the resulting model is far away from criticality, hence more robust to noise.

I. INTRODUCTION

Many applications of generative modeling, especially in biological systems, are confronted to a limited amount of available data, from which a large number of parameters have to be inferred [1]. A particularly interesting example is that of proteins, which belong to the most interesting complex systems in nature and are essential in almost all biological processes. Most of them robustly fold into well-defined three-dimensional structures, which in turn form the basis of their functionality. This triangular sequence-structure-function relationship has, over several decades now, attracted substantial attention in biological physics [2,3].

A fascinating approach to the generative modeling of biological sequences has emerged over the last years [4,5]. In the course of evolution, biological sequences accumulate mutations and become more diverse. We can now easily observe the sequence variability across large families of so-called homologous proteins, i.e. proteins of common evolutionary ancestry and of close to equivalent function but in different species or biological pathways [6]. Such homologous proteins may differ by 70-80% of their amino acids without substantial changes in structure and function. However, their sequence variability is not fully random: a vast majority of mutations is deleterious, reducing protein stability or functionality. They are thus suppressed by natural selection. Only protein variants of similar or even better functionality are maintained. In this way, the protein's structure and function constrain the viable sequence space that can be explored by evolution. Inverting this argument, the empirically observed variability of homologous sequences contains information about such evolutionary constraints, albeit frequently well hidden. This idea is at the basis of the concept of data-driven "sequence landscapes", i.e. classes of models that describe the statistical properties of protein families, assigning high probabilities to functional aminoacid sequences and low probabilities to non-functional ones [5,7]. The log-probability (or minus "energy") is thus interpreted as a measure of sequence fitness, hence the name of sequence (fitness) landscape [8]. Among the best known such models are Potts models (PM), parameterized by local fields and two-site interaction couplings (cf. below for details), and constructed via the Direct Couplings Analysis (DCA) method, which is now firmly established [5,7]. The DCA parameters can be obtained via inference or learning procedures [9][10][11][12], and they can be used to extract useful information on molecular structure [13][14][15][16] and function [17,18], on the effects of mutations [19,20], and to generate new artificially-designed molecules with specific properties [21,22].

A concrete implementation of DCA is the following [12]. Given training data in the form of a Multiple Sequence Alignment (MSA) of M homologous sequences of aligned length L, the PM parameters are learned by the so-called Boltzmann machine learning (BML) algorithm [23]. By performing gradient ascent on the log-likelihood of the model given the data, BML determines values of couplings and fields such that the one-and two-site model frequencies match the empirical ones derived from the MSA. A standard pairwise q-state PM is thus specified by q 2 L(L -1)/2 couplings and qL fields, where, for proteins, q = 21 corresponds to the 20 naturally occurring amino acids plus the gap symbol used for insertions or deletions.

Crucially, despite the fact that modern sequencing techniques are making available a huge amount of biological sequences, and in particular hundreds of millions of protein sequences [24], a serious over-fitting problem is present when PM are used as models of protein families. In fact, with typical sequence lengths L ∼ 50 -500, the parameters to be inferred are ∼ 10 6 -10 8 , which in most cases substantially exceeds the available information from the MSA. The resulting over-fitting is manifested in several ways: (i) many couplings turn out to be rather small and noisy, (ii) the PM is close to a critical point, i.e. it can be very susceptible to small changes in its parameters, and (iii) different training procedures, e.g. with different initial conditions, can lead to significant changes in the sets of parameters without affecting the fitting accuracy, which severely limits the interpretability of the model.

These observations call for a parameter reduction procedure, which aims at identifying a minimal set of couplings needed to accurately describe the training data without overfitting. Hopfield-Potts models [25] and the more general Restricted Boltzmann Machines (RBM) [26] lead to a dimensional reduction of parameter space by learning collective "patterns" from sequence data, which in turn can be interpreted as extended sequence motifs and are activated via a limited number of hidden variables. The resulting coupling matrix is lowrank but still dense. A complementary approach aims at sparsifying the network of couplings: 1 -norm regularization has been used in a number of approximate methods [27,28], but cannot be easily used for generative modeling, because the regularization penalizes also non-zero couplings, which in turn assume too small values. Alternatively, a "color-compression" scheme [29] has been proposed, which groups together sequence symbols with low frequency in specific sites. However, frequent symbols may also be involved into statistically non-supported couplings. Another example is that proposed in [30] where a candidate sparse graph topology is sought by pruning the MSA columns associated with low values of the mutual information. Although this method has to be preferred when L is so large to prevent the standard DCA implementations, it completely loses some information on the target statistical model. Overall, a statistically principled and efficient approach to construct sparse PM for protein sequence modeling is still lacking.

In this work, we introduce an information-theory based "decimation" procedure, which allows for an iterative and controlled removal of irrelevant couplings. As a result, parameters are removed either if they have no statistical support (as in color compression), or if they have statistical support for being very small. We show that up to about 90% of the coupling parameters can be removed without observing any substantial change in the fitting accuracy and in the generative properties of the resulting Sparse Potts Model (SPM). Although greedy, our pruning scheme does not require to add extra terms in the energy function of the model, at variance with any treatable regularization, like 1 or 2 , and therefore it preserves the generative properties of PM. Finally, we show that the resulting SPM are not close to criticality, at variance with the original PM learned using standard DCA. Our results thus demonstrate that the observed criticality of PMs inferred from protein sequence data is not an intrinsic feature of the biological systems themselves, cf. [31], but results from the over-fitting in the learning procedure.

II. AN INFORMATION-GUIDED DECIMATION PROCEDURE

With each sequence S = (s 1 , • • • , s L ) of length L, in which s i can take q possible values (q = 21 for proteins), a PM associates a statistical "energy" or Hamiltonian H(S), written as a sum over single-site fields h i (s i ) and two-site couplings J ij (s i , s j ):

H(S) = - 1≤i<j≤L J ij (s i , s j ) - 1≤i≤L h i (s i ) . (1)
The negative of the Hamiltonian can be interpreted as a "fitness score" for protein sequence S, with an associated Boltzmann probability P (S) = exp{-H(S)}/Z, where Z = S exp{-H(S)} is the partition function guaranteeing correct normalization of P . Hence, the surface defined by H(S) over the space of sequences can be interpreted as a "fitness landscape" or -using a more cautious term -"sequence landscape" for the protein family represented by the training MSA. We define the "model density" d as the number of non-zero couplings J ij (a, b) = 0 divided by the total number of possible couplings q 2 L(L -1)/2. Note that this definition is given element-wise, i.e. for each i, j, a, b, and not block-wise for entire q × q matrices J ij coupling two sites i, j. Fields are not decimated and do not contribute to the model density: we consider them an essential ingredient of the model because they encode amino-acid conservation.

A fully connected model, i.e. with d = 100%, can be trained to arbitrarily high accuracy using standard BML [12]. Let us define the empirical one-site frequency f i (a) of observing amino acid a in position i in the MSA, and two-site frequency f ij (a, b) of observing amino acid a in position i and b in position j in the same sequence of the MSA. BML performs a gradient ascent on the loglikelihood, which gives update equations for the couplings and fields at each learning epoch:

δh i (a) = η h [f i (a) -p i (a)] , δJ ij (a, b) = η J [f ij (a, b) -p ij (a, b)] , (2)
where the p i (a), p ij (a, b) are the one-and two-site marginal probabilities of the PM, which are estimated at each iteration of the learning by sampling P (S) via a standard Markov Chain Monte Carlo (MCMC) simulation, and η h , η J are the learning rates for fields and couplings. These equations are iterated until convergence to a fixed point, at which the model almost perfectly matches the empirical frequencies. Note that a PM trained in this way also corresponds to the maximum entropy or least constrained model that is compatible with the one-and two-site empirical frequencies [13,32].

Our decimation procedure consists in choosing pairs of sites i < j and amino acids a, b, and fixing the corresponding coupling permanently to J ij (a, b) = 0. The coupling is removed from the set of adjustable parameters, and the corresponding two-site frequency f ij (a, b) is no longer explicitly fitted in the subsequent BML epochs. However, an important property of PM is the so-called "gauge" or reparameterization invariance: the transformation

J ij (a, b) → J ij (a, b) + J ij (a) + K ij (b) , h i (a) → h i (a) -H i - j(>i) J ij (a) - j(<i) K ji (a) , (3)
leaves the Hamiltonian in Eq. ( 1) and the associated Boltzmann distribution P (S) invariant, for any choice of the J , K and H. Hence, a gauge transformation can transform a zero coupling into a non-zero one and vice versa. Because the decimation procedure fixes some couplings to zero, it breaks this invariance.

We thus begin our decimation procedure by a "gauge fixing" step, which sets to zero 2q -1 out of all q 2 entries of each coupling matrix J ij . To do so, we identify, independently for each pair of sites 1

≤ i < j ≤ L, the 2q -1 amino-acid pairs (a, b) of smallest connected correlation c ij (a, b) = f ij (a, b) -f i (a)f j (b)
, and fix the corresponding couplings J ij (a, b) to zero. Only the other q 2 -2q + 1 = (q -1) 2 couplings are updated using the BML, Eq. ( 2). This procedure chooses a model of minimal density d = [(q -1)/q] 2 = 90.7% out of all equivalent PM related by the gauge transformation in Eq. ( 3). The parameters are initialized using a "profile model" fitting only the one-site frequencies f i (a). This initial model has zero couplings and fields h

(0) i (a) = log f i (a)+H i , with the constant H i being fixed by a h (0)
i (a) = 0 (with a very small pseudo-count added to f i (a) to avoid infinite fields, see Appendix A 3). The fitting quality of the learned PM is tested by the Pearson correlation between the empirical c ij (a, b) and their counterparts in the model P (S), the latter being estimated from a large independently and identically distributed MCMC sample. For all protein families considered in this work, this Pearson correlation exceeds 0.95, see Fig. 1 and Appendix B 1. Note that the results of our decimation procedure depend on the initialization and gauge fixing described above. We tried a different initialization, either fixing both couplings and fields to zero, or initializing both using pseudo-likelihood maximization (PLM) [11]. We found qualitatively similar results, but with slightly worse performance (Appendix C 3).

Any further decimation of couplings changes the model. To measure the impact of removing a given coupling J ij (a, b) from a PM, we determine the symmetric Kullback-Leibler (KL) divergence between the Boltzmann distributions with and without that coupling. We thus consider a Potts Model with Hamiltonian H, and another with Hamiltonian H in which a given coupling is removed:

H (S) = H(S) + J ij (a, b)δ a,si δ b,sj . (4)
We observe that averages over P = e -H /Z can be expressed in terms of averages over P = e -H /Z as

O(S) P = S O(S)e -H (S) S e -H (S) = S O(S)e -Jij (a,b)δa,s i δ b,s j e -H(S) S e -Jij (a,b)δa,s i δ b,s j e -H(S) = O(S)e -Jij (a,b)δa,s i δ b,s j P e -Jij (a,b)δa,s i δ b,s j P . (5)
Hence, the symmetric Kullback-Leibler divergence of P and P is

D ab ij = D KL (P ||P ) + D KL (P ||P ) = - S [P (S) -P (S)][log P (S) -log P (S)] = H -H P -H -H P = J ij (a, b)δ a,si δ b,sj P -J ij (a, b)δ a,si δ b,sj P (6) = J ij (a, b)δ a,si δ b,sj P - J ij (a, b)δ a,si δ b,sj e -Jij (a,b)δa,s i δ b,s j P e -Jij (a,b)δa,s i δ b,s j P = J ij (a, b)p ij (a, b) - J ij (a, b)p ij (a, b)e -Jij (a,b) p ij (a, b)e -Jij (a,b) + 1 -p ij (a, b) ,
where p ij (a, b) = δ a,si δ b,sj P is the marginal two-site probability of P , which coincides, at convergence of Eq. ( 2), with the empirical frequency f ij (a, b). Note that we could also have equivalently used the nonsymmetrized KL divergence (Appendix A 1). At each decimation step, we now remove the least significant couplings, i.e. those with the lowest D ab ij . For computational efficiency, this is done for a fixed fraction (in this work we choose 1%) of all remaining couplings. Note that D ab ij = D(J, p ∼ f ) (dropping the indices for notational simplicity) goes to 0 either when f → 0 or f → 1 at fixed J, or when J → 0 at fixed f . More precisely, we have

D(J, f ) ∼ Jf (1 -e -J ) for f → 0, D(J, f ) ∼ J(f -1) for f → 1, and D(J, f ) ∼ J 2 f (1 -f ) for J → 0.
The first and second limits imply that finite couplings can be decimated if the corresponding frequency is close to zero or one, i.e. they have little statistical significance because the corresponding amino acids are almost never observed (as in color-compression [29]) or almost always observed. The third limit indicates that small couplings are decimated whatever f is (similar to the procedure proposed in [33] for the inverse Ising problem using PLM). Numerically, we observe that the percentage of pruned couplings corresponding to each category varies during decimation (Appendix C 2). After a decimation step, we perform additional BML iterations of Eq. ( 2) on all undecimated couplings and the fields, to reach convergence again. In this way, we progressively obtain PMs of reduced density, and we stop the decimation when d = 1%.

Note that in order to accurately estimate D ab ij , it is important that the PM learning is well converged before each decimation step. We attempted an "online" decimation in which couplings are pruned either after a fixed number of iterations of Eq. ( 2) or for having reached convergence, and found that this provides no advantage (Appendix C 4), neither in terms of generative performance (i.e. the Pearson correlations at d = 1% do not improve), nor in computational efficiency (i.e. the computational time required to reach d = 1% is almost unchanged). Other decimation strategies based on f ij (a, b) only (removing statistically unsupported couplings), or on J ij (a, b) only (removing small couplings), or on applying 1 -norm regularization to select relevant couplings, were found to perform substantially worse than the information-based procedure using Eq. ( 6) (Appendix C 1).

III. RESULTS AND DISCUSSION

We focus here on a representative protein family, the PF00076 family from the Pfam database [6], corresponding to a RNA recognition motif (RRM) of about 90 amino acids, known to bind single-stranded RNAs. The MSA provided by Pfam contains M = 137605 sequences of aligned length L = 70. Results obtained for other families (Appendix B) fully confirm the general conclusions drawn here for the RRM.

In Fig. 1 we show, for model densities down to 1%, the Pearson correlation coefficient between the empirical onesite frequencies f i (a) obtained from the original MSA, and the model one-site marginal probabilities p i (a), estimated by MCMC sampling. Similar curves are also shown for the two-site connected correlations c ij (a, b) and for a selected sub-set (specified in Appendix B 1) of threesite connected correlations, defined as The generative performance of the model is essentially unchanged down to a density of 10%, and slowly decays for even sparser couplings. However, even down to d = 1%, the Pearson coefficients remain at remarkably high values above 95% for the two-site correlations, and above 84% for the three-site correlations.

c ijk (a, b, c) = f ijk (a, b, c) -f ij (a, b)f k (c) -f jk (b, c)f i (a) -f ki (c, a)f j (b) + 2f i (a)f j (b)f k (c) , (7)
where i, j, k are the indices of the columns of the MSA (which take value from 1 to L), and a, b, c run over the amino-acids and the gap symbol (practically, from 1 to q). The one-site frequencies are perfectly reproduced by the model, i.e. f i (a) = p i (a), as a consequence of the fixed-point condition in Eq. ( 2), and the Pearson coefficient thus remains equal to one at all d (up to tiny deviations due to the finite MCMC samples used in BML and in estimating p i (a)). For the maximal density d max = [(q -1)/q] 2 obtained after gauge fixing, the two-site correlations should also be perfectly reproduced because of Eq. ( 2). In practice we only reach a Pearson coefficient of ∼ 0.975 due to sampling noise (Appendix A 4). On the contrary, for d < d max only a fraction of all two-site frequencies is explicitly fitted by the model via sparse BML. Nevertheless the two-site Pearson coefficient is essentially independent of d, up to a slight reduction when d < 10%. Finally, three-site correlations, that are never explicitly fitted by the model (the training process in Eq. ( 2) does not include three-site information), are nevertheless very accurately reproduced, with a Pearson coefficient around 0.94 for all d > 10%.

Note that the reproduction of unfitted observables is a highly non-trivial test for the generative properties of our models [12], i.e. of the capacity of the model to generate data being statistically close to indistinguishable from the natural sequence data used for model learning. Below density d = 10%, the generative quality of the model for three-site correlations is slightly reduced, remaining nevertheless very high (above 84% down to d = 1%). We discuss the generative property of the sparse models Even the most sparse model, with only 1.6% of couplings, shows an excellent performance at contact prediction. The curve for plmDCA, a standard DCA approach for contact prediction, is shown for reference and gives comparable results.

introducing additional metrics in Appendix D.

A second test of model quality is the prediction of structural contacts, which constituted the major application of DCA in the last years. The idea is that pairs of strongly interacting sites in the PM should correspond to close-by residues in the three-dimensional structure, which display strong coevolution to maintain the proper protein fold and functionality. Using the standard convention for coevolutionary contact prediction, we consider a pair of residues to be in contact if the distance between them is at most 8 Å, and we exclude easy-topredict short-range contacts by considering only pairs with |i -j| ≥ 4 in our analysis. The reference (groundtruth) distance was obtained by the package [34] that takes the shortest distance between heavy atoms in all protein structures registered in the Protein Data Bank (PDB) [35] for the given Pfam family. We follow the standard procedure for contact prediction, which consists in computing the average-product corrected (APC) Frobenius norms of the coupling matrices (note that the coupling matrices are transformed into the zero-sum gauge and that the gap states a, b = q are excluded from the sum [36]),

F ij = q-1 a,b=1 J ij (a, b) 2 ; F APC ij = F ij -k F ik k F kj kl F kl . (8)
In Fig. 2 we show the fraction of true contacts within the first k pairs of sites, ranked in decreasing order of F APC ij . We observe that the performance of the model at inferring the structural contacts is only slightly deteriorated even in the sparsest case d = 1.6%.

In Fig. 3 we show the probability distributions of couplings J ij (a, b), separately for pairs i < j corresponding to contacts and all the others. We observe that, both for contacts and non-contacts, the decimation affects the shape of the distribution around J ∼ 0 in a similar way, while the tails are essentially unaffected. Overall, these results explain why the performance of the PM for contact prediction using F APC ij is essentially independent of d (Fig. 2). Unfortunately, the large-J tail of the distributions of couplings on non-contacting sites does not change upon sparsifying the model, which suggest that our decimation procedure cannot help in devising better contact predictors.

In order to study the criticality of the models, we consider a simple perturbation of the statistical weight, i.e. we rescale the Hamiltonian H(S) by a formal inverse temperature β = 1/T and set P (S) ∼ e -βH(S) , in such a way that T = 1 corresponds to the original model trained on data, while measuring the variation of the model entropy S. In Fig. 4 we show the heat capacity C = T dS/dT of the PM for several sparsities (see Appendix B 4 for details on the computation of C). Note that a large C indicates a large variation of the model entropy with T , or equivalently that the model statistics changes strongly after a slight change of the parameters. This is indeed the best definition of criticality in statistical physics, keeping in mind that our models have finite size L and we thus cannot perform a finite-size scaling analysis to determine if the observed peak in C corresponds to a phase transition in the thermodynamic limit. In Fig. 4 we observe that the denser models display a FIG. 4. Criticality -Heat capacity as a function of temperature for models with different density. The densest models show a strong peak of specific heat close to the reference scale T = 1, which is a signature of criticality: the model is extremely sensitive to a small change of couplings, due to overfitting. On the contrary, sparse models display a much smaller peak, which is also shifted away from T = 1 towards lower temperatures, indicating a better robustness of the learning.

large peak in C close to T = 1, which indicates that the models are close to criticality. Upon sparsifying the model, the peak amplitude is strongly reduced and the peak is also shifted to lower temperatures, i.e. further away from the reference scale T = 1. These results suggest that the criticality of the dense models comes from over-fitting. Because the dense models have a huge number of parameters, they are able to fit all the details of the training data. As a consequence, the model becomes very sensitive to noise, and a little change of the parameters changes a lot the model statistics. On the contrary, sparse models have less parameters and are thus more robust to noise.

Ref. [37] provides Deep Mutational Scanning (DMS) data for a representative member of the PF00076 family, namely the RRM2 domain of the Poly(A)-Binding Protein (PABP) in the yeast species Saccharomyces cerevisiae. Using this domain as a reference, the authors generated a library of 110,745 protein variants, including 1,246 single amino-acid substitutions and 39,912 double amino-acid substitutions. Each of these variants was experimentally scored for function, by monitoring the growth of mutant yeast and finally, a "fitness score" was attributed to each mutant sequence in the experiment [37]. Within our models, the inferred Hamiltonian H(S) in Eq. ( 1) is also interpreted as a sequence-fitness score. Hence, a good test of the generative property of our models is to check whether the energy differences ∆H = H(mutant) -H(reference) between mutant sequences and the PABP reference correlate well with the experimental fitness variations. Because the mapping between experimental and model fitness may be non-linear, in Fig. 5 we show the Spearman's rank correlation between these two variables, both for single and double mutants. In the dense d = d max case, we reproduce the reference values already given in Ref. [20]. We also observe that upon reducing density, once again the model quality is not degraded, down to d ∼ 10%. Even for d = 1.6% the model performs quite well, and much better than a profile model, which coincides with the limit d → 0 of our decimation procedure.

IV. CONCLUSIONS

We introduced a general parameter reduction scheme for Boltzmann Machine Learning, and we applied it to Potts models for protein sequence data, i.e. for the learning of highly accurate and generative, but sparse DCA models. Our strategy makes use of a rigorous information-based criterion to select couplings that are iteratively pruned. Intuitively, removed couplings are either statistically unsupported, i.e. they correspond to pairs of amino acids that are almost never or almost always observed, similarly to the color-compression scheme [29], or they are small, i.e. they correspond to pairs that are only weakly correlated, or a combination of both. The statistical significance of a coupling is precisely quantified by the symmetric KL divergence between the Boltzmann measure of the Potts model with and without this coupling, which is exactly computable from the model or the empirical statistics.

While our method is fully general for learning Boltzmann machines from high-dimensional categorical data, here we focused on its application to model protein families via Potts models, in which strong couplings are usually associated with physical contacts in the threedimensional protein fold. We stress that the aim of this work is not to provide a sparse graph topology underlying the true interaction network, and indeed the pruning is not performed block-wise but at the level of the individual coupling entries, but to provide a general framework of parameters reduction strongly based on informationtheoretic assumptions. We have shown that the model can be decimated down to less than 10% of the original couplings, while losing neither its generative quality, nor its accuracy in contact prediction. However, it has to be noted that many couplings not corresponding to structural contacts remain non-zero even in the lowest-density models. The interpretation of such couplings remains unclear. They may result from subtle effects due to the phylogenetic relations between the training sequences [38,39], but also from extended functional constraints as those found by Restricted Boltzmann Machines or Hopfield-Potts models [25,26]. As a result, further work is needed to make DCA-type modeling fully interpretable.

The sparse models resulting from our decimation procedure are also far away from criticality: they do not display the specific-heat peak close to the formal temperature T = 1 that characterizes the dense models. Hence, we attribute the criticality observed in dense models to over-fitting, and conclude that our decimation procedure makes model learning more robust to finite-sample noise. Finally, the model maintains its performance in predicting the fitness of mutations around a reference sequence, i.e. it is capable of predicting the local shape of the fitness landscape after having been trained on a global alignment of distantly diverged amino-acid sequences.

Our decimation procedure solves the first two problems mentioned in the introduction: we can eliminate small and noisy couplings, and the resulting model maintains its fitting and generative qualities, while being statistically more robust. Unfortunately, we were unable to solve the third problem, namely the strong dependence on the initial condition of the training: different initial conditions (zero couplings and fields, profile model, plmDCA) produce fully-connected PMs of equal fitting quality but with slightly different performances in predicting contacts and mutational effects. This difference does not disappear after decimation (Appendix C 3). In other words, our decimation procedure remains sensitive to the initial fully-connected model from which it is started.

The resulting sparse PMs attempt to fit the data by using the minimal number of two-site couplings, i.e using coupling matrices that are as sparse as possible. In the context of proteins (or RNA), it is natural to think that the sparse couplings identified by the model are the most relevant to describe the physical two-site correlations that arise from the need to maintain the threedimensional folded structure. This strategy is complementary to collective-feature learning, e.g. via RBM or Hopfield-Potts models [25,26], in which the number of parameters in the coupling matrix is reduced by assuming it to be of low-rank. The features learned by these machines are associated with global sequence motifs, related, e.g., to protein function or its interactions, but the accuracy of contact prediction is reduced. An interesting and natural perspective would be to combine these two strategies into a general "sparse plus low-rank" scheme, cf. [40] for a related idea, which could lead to an accurate description of protein families in an easily interpretable way, with sparse two-site couplings describing physical constraints coming from structural contacts, and lowrank couplings describing biological features associated with protein function and its evolutionary history.

To conclude, we would like to stress once more that our information-based decimation strategy is not specific to the application of Potts models to protein sequence data. It can directly be used in other applications of inverse statistical physics and Boltzmann machine learning, as e.g. in modeling neural or socio-economic data [1], and may be adapted to more general network reconstruction schemes.

This second quantity also coincides with the variation of the likelihood of data under the change of model,

∆L = 1 M M m=1 [log P (S m ) -log P (S m )] = 1 M M m=1 J ij (a, b)δ a,s m i δ b,s m j + log Z Z = J ij (a, b)f ij (a, b) + log[p ij (a, b)e -Jij (a,b) + 1 -p ij (a, b)] , (

Data set

In the following, we report the details of the five protein families analyzed in our work, identified as PF00014, PF00072, PF00076, PF00595, and PF13354 in the Pfam database (https://pfam.xfam.org/) [6,41]. For PF00014, PF00072, PF00076 and PF00595 we filter the full set of sequences downloaded from Pfam, keeping only those that have less than six consecutive gaps. Empirically, we have found that the presence of stretched gaps renders the training more difficult as the Markov Chain Monte Carlo (MCMC) used for sampling has difficulties in visiting both very gapped sequences and gapfree region of the sequence landscape in a proper way, i.e. proportionally to the correct Boltzmann weight. This leads to a systematic bias in the model statistics. For the Beta-lactamase2 family PF13354, we used a slightly different procedure: we downloaded the Pfam pHMM model for that family, and we scanned the NCBI database to obtain aligned sequences. We then filtered sequences according to two criteria: (i) 80% sequence coverage (i.e. less than 20% gaps) and (ii) redundancy reduction at 80% (so M eff ≈ M in this case). We also removed the sequence TEM-1 (which is used as reference in the deep mutational scanning, as discussed below), and all sequences very similar to it. Note that because there are overlapping Betalactamase families in Pfam, our procedure, based on a single pHMM, gives also sequences that would align better to some other family in Pfam, in particular to the Beta-lactamase family PF00144.

In Table I we show the name of the protein domain associated with each family, the length, i.e. the number of columns L of the multiple sequence alignment (MSA), the number of sequences M of the original MSA and M eff , the number of statistically relevant sequences after a standard re-weighting of close-by sequences [14].

Training protocol

We specify here the details of the Boltzmann learning used to train the dense Potts model and to refine the non-zero parameters within the decimation run.

First, we compute the data statistics from the input MSA as

f i (a) = (1 -α)f emp i (a) + α q , (A4) f ij (a, b) = (1 -α)f emp ij (a, b) + α q 2 , ( A5
)
with f emp i (a) and f emp ij (a, b) being the one-and twosite frequencies computed from the MSA (for all positions i, j and amino acids a, b), and with α being a pseudo-count [42] introduced to avoid divergent fields and couplings associated with poor statistics. Here we set α = 1/M eff except for the PF13354 family for which we set α = 10 -50 (we observed that other values of the pseudo-count do not lead to a significant change of the trained models). Then, we start from a profile model, i.e. all couplings are set to zero and the fields are equal to h i (a) = log[f i (a)] + H i with H i a constant ensuring a h i (a) = 0. Subsequently we iteratively refine the parameters according to Eq. ( 2), using as learning rate η J = η h = 5 • 10 -2 . We stop the algorithm when the convergence error , computed as the maximum error attained in the fitting of the two-site connected correlations,

= max i,j,a,b |f ij (a, b) -f i (a)f j (b) -p ij (a, b) + p i (a)p j (b)| , (
A6) reaches 10 -2 (this value may slightly change depending on the family, up to 5•10 -2 for the PF13354 family which is the most difficult to train). At each iteration, we use Metropolis-Hasting MCMC to compute the model statistics p i (a) and p ij (a, b). We run N chain independent MC chains, with N chain = 3000 for PF00014 and PF00595, N chain = 1000 for the longer PF13354, and N chain = 5000 for the copious families PF00072 and PF00076. The chains are initialized at the first iteration from a uniform independent random distribution over all possible amino-acids, gap included, and are then persistent over iterations, i.e. at each new iteration the chain is initialized from the last configuration of the previous iteration. Each chain runs for T eq = 20 MC sweeps (one sweep corresponds here to L single-site Metropolis-Hastings MC steps) before starting to sample 10 configurations spaced by T wait = 10 MC sweeps. Hence, the total number of generated samples in a single iteration is 10 × N chains (from 10 4 to 5 • 10 4 depending on the family), and each chain is evolved by 110 MC sweeps in a single iteration. sample, from which we compute the model statistics to be compared with the MSA statistics. As in training, the MCMC method used for the sampling is the standard Metropolis-Hasting algorithm, using N chain independent MC chains, initialized from a uniform independent random distribution over all possible amino-acids, gap included. Each chain is evolved for T eq MC sweeps in order to achieve equilibration, before we start collecting samples, the waiting time between each sampled configurations being T wait MC sweeps. We specify in Table II the values of N chain , T eq and T wait and of the total number of collected samples, M MC . Note that the conditions for the sampling are different from those used in the learning. We also compute, for each model, the Hamming distance d H (t) between an equilibrium configuration and its time evolution under the MCMC dynamics after t MC sweeps (averaged over initial configurations and over the dynamics), see Fig. 6. Obviously, d H (0) = 0 and for short times, d H (t) grows linearly, with a coefficient given by the acceptance rate of single-site mutations in the MCMC dynamics. At long times, d H (t → ∞) saturates at the average distance between two independent samples from the Potts Model equilibrium distribution. This quantity can also be computed by measuring the Hamming distance between two independent MC chains, after equilibration, and is reported as a red horizontal line in Fig. 6. The time it takes for d H (t) to reach its asymptotic value gives an estimate of the decorrelation time of the MCMC dynamics, i.e. the time needed to generate a new independent equilibrium sample.

We observe that for PF00014, PF00072 and PF00076, the decorrelation time is of the order of 10 2 MC sweeps, and independent of sparsity, which suggest that the model is sampled in equilibrium during the learning process. In fact, we obtain exactly the same model statistics upon resampling the model in different conditions.

For PF00595 the decorrelation time is ≈ 10 done with persistent chains, and a small learning rate, we still believe that proper equilibrium sampling is achieved during learning. This is confirmed by the fact that we reproduce the same model statistics under resampling. Furthermore, we observe that the decorrelation time is reduced upon sparsifying the model, which suggests that the sparse models are less critical, as we discuss below. For PF13354 the situation is radically different. In this case, the decorrelation time is huge (more than 10 4 MC sweeps for the dense model). This is likely due to the presence of multiple subfamilies, such that the MC chains take a lot of time to jump from one subfamily to another. With such a long decorrelation time, learning becomes extremely hard and we cannot guarantee that equilibration is achieved during it. In fact, we find that upon resampling the model starting from random initial states, the statistics is initially good (after ≈ 2 × 10 4 MC sweeps) but then is degraded, indicating that the model suffers from overfitting due to poor equilibration during learning. For the sparse models, the decorrelation time is substantially reduced (by almost a factor 100), and consistently we find that resampling is stable at all times.

Appendix B: Results for the other protein families

In this section we report the same type of results shown in the main text for PF00076, but for the four remaining families: PF00014, PF00072, PF00595 and PF13354.

Fitting quality

To evaluate the quality of the sparse models we compute, for each possible density, the Pearson correlation coefficients between a certain type of statistics computed from the empirical data (the MSA) and the model (via MCMC). More precisely, we focus on one-site frequencies and two-and three-site connected correlations, as defined in Eq. (7). To select the indices of the most significant three-site connected correlations we have first extensively scanned all possible triplets and computed the empirical frequencies for all possible color assignment. We then keep the elements c ijk (a, b, c) with empirical absolute values above 10 -4 : only for those elements we compute the corresponding model correlations, in order to limit the computational cost. The model correlations are computed from a set of samples generated via the MCMC Average Hamming distance Average Hamming distance Average Hamming distance FIG. 6. Averaged Hamming distances between an equilibrium sequence at time t = 0 and the evolved sequence after t MC sweeps. The average is computed using 10 4 independent MC chains. procedure described in Appendix A.

In Fig. 7, we show the Pearson correlation coefficients for the three metrics between the data and the models as a function of the model density, for the PF00014 (a), PF00072 (b), PF00595 (c) and PF13354 (d) families. For all families, the Pearson coefficient maintains almost the same value reached for the densest (fully connected) model up to a density of about 10%. When the density goes below 10%, the Pearson coefficient gradually decreases for all families; not surprisingly, the reduction associated with the three-site connected correlations is more pronounced, because this more-than-two-site correlation is not explicitly fitted by BM learning.

The case of PF13354 is special because, for the reasons discussed in Appendix A, the learning, which is done using rather short waiting times between samples, suffers from a very long decorrelation time in the dense case. Hence, the resampling degrades when MC chains are evolved for long times, which explains why the Pearson coefficients are poor for d > 20%. For d < 20%, the decorrelation time becomes much shorter, and the resampling is stable over time, but the Pearson coefficients get progressively degraded when d is reduced, as for the other families. The optimal compromise seems to be d ≈ 20% for this atypical family.

Contact prediction

The APC-corrected Frobenius norms associated with the couplings can be used for scoring each pair of sites of the MSA (cf. main text). As already explored in literature, this score correlates well with the physical distances between pairs of residues in the three-dimensional struc- ture of the protein domains. Larger Frobenius norms suggest larger probabilities of a physical interaction. As usual, we try to estimate the quality of the sparse models through a set of Positive Predictive Value (PPV) curves associated with the prediction of contacts. As reference structures we use those extracted from [34], a tool that outputs the shortest relative distance of pairs of residues over all known crystal structures registered in the Protein Data Bank (PDB) database [35]. In Fig. 8, we show the PPV curves for a sub-set of the sparse models (the density is mapped to a different color of the lines) together with the result of plmDCA [10] used here as comparison (red lines). Even keeping only 10% of the coupling parameters, i.e. when 90% of them are removed by the decimation procedure, the accuracy of the contact prediction remains stable, that is the performances are comparable to those of the fully connected models. The comparison to plmDCA is instead heterogeneous: as found in [12], the Boltzmann machine learning can have comparable performances to plmDCA as for PF13354 in panel (d), slightly worse as for PF00014 and PF00072 in panels (a), (b), or slightly better as for PF00595 in panel (c).

Pearson correlation coefficient

Coupling distribution

Because the couplings mirror a physical interaction among residues, one may guess that the more we dec-imate the model, the more we decimate the couplings not associated with residues in contact. Similarly, one may expect that the more a coupling is important in terms of three-dimensional structure, the larger will be its strength, hence it will be preserved by the decimation.

To check whether this is the case, we plot in Fig. 9, for PF00014 (a), PF00072 (b), PF00595 (c) and PF13354 (d), the distributions of the couplings linking residues in contact (panel 1) and not in contact (panel 2); the values of the corresponding densities are indicated in the legend. We note that as we reduce the density of the couplings, those corresponding to residues in contact are slightly enhanced (indeed, the original red histograms in Fig. 9 for the dense models are shifted to slightly larger values in the sparse case), but we do not observe a significant change in the tails of the distributions, as discussed in the main text.

Criticality

Dense Potts models are generally very sensitive to a perturbation of their model parameters: a slight change of the couplings or the fields leads to a dramatic transformation of the model statistics, which thus seems to be close to a phase transition, i.e. to be critical. A good measure of the criticality of statistical models is represented by the heat capacity, which is obtained by apply- ing a global variation to the parameters, J → J/T, h → h/T , and measuring the derivative of the average internal energy with respect to the temperature,

C(T ) = ∂ H T ∂T = 1 T 2 H 2 T -H 2 T . (B1)
The averages in Eq. (B1), denoted as . T , are evaluated by sampling a system with Boltzmann weight exp{-H/T }. Standard thermodynamic identities also show that T C(T ) = ∂S/∂T , where S(T ) is the entropy of the model. The model criticality is related to the magnitude of C(T ) in the vicinity of T = 1, which expresses how quickly the model entropy (or energy) varies under a small rescaling of all couplings. Fig. 10 shows the behavior of the heat capacity C(T ) as a function of the temperature T for the models associated with the four families analyzed here: the color of the lines depend on the value of the density of the corresponding model, which spans the range (1, 90)%. We observe that for all families, upon sparsifying the model, (i) the heat capacity is reduced rendering the model less sensitive to changes in the model parameters and/or (ii) the peak slightly shifts towards a temperature smaller than T = 1, the natural temperature of the learning. In all cases, the value of C(T = 1) decreases upon sparsifying the model. This observation suggests that a dense model learned by the empirical data is indeed close to a phase transition, but the criticality disappears (or decreases substantially) for the statistically equivalent sparser models. Hence, we conclude that the sensitivity of the dense model is related to over-fitting. Note that the suppression of criticality is also suggested by the reduction of the decorrelation time, as discussed in Appendix A.

Mutational landscape prediction

Similarly to the analysis we proposed in the main text for the PF00076 landscape and the experimentally determined single and double-mutants fitness, we show in Fig. 11 the Spearman correlation coefficient, as a function of the density, between the energy variation (computed according to our models) and the experimental fitness associated with single-residue mutations. Here we consider the libraries of single mutants for the Beta-lactamase2 domain of the TEM1 protein [43] (here the fitness is related to antibiotic resistance) and for the PDZ3 domain of the PSD95 protein [44] (here the fitness refers to the CRIPT ligand), which we assume to be described by the models for PF13354 and PF00595 families, respectively. As shown in Fig. 11, the correlation coefficient (spBM lines) between the experimental measures and the energy differences of our models are mostly constant as a function of the density; only a smooth increment (drop) is appreciated for densities smaller than 10 -1 for PSD95 (TEM1). We remark that even in the sparsest case, the Spearman correlation coefficient never crosses that obtained from a pure profile model (denoted as prof ) suggesting that the remaining non-zero couplings of our sparse models are fundamental for the good description of the fitness landscape.

(a1) (a2) (b1) (b2) (c2) (c1) (d1) (d2

Appendix C: Additional results on PF00076

To complete the analysis described in the main text, we propose here a set of additional results for the PF00076 family. More precisely, we compare the learning and decimation strategy used in the main text and in Appendix B (initialize the parameters in the profile model, learn a dense model until convergence, then perform decimation) to several different initializations of the learning and to other decimation strategies based on different metrics. We also investigate the nature of the decimated couplings, via the statistics of the second moments associated with them, to stress the non-trivial nature of the symmetric Kullback-Leibler based decimation.

Heat Capacity

Decimation strategies

The method presented in the main text uses as criterion (or score) for the iterative decimation an information-theory based measure, the symmetric Kullback-Leibler divergence (symKLD) between the model with or without a certain coupling. As a result, the decimation score of each coupling takes into account both its statistical relevance (related to the second moments associated with it), and the strength of the coupling alone. We compare here the results presented in the main text to two simpler strategies where at each decimation step a) we remove 1% of the weakest couplings or b) we remove 1% of the couplings associated with the lowest, hence less statistically significant, two-site frequencies.

In Fig. 12, in the left panels, we compare the three possible decimation procedures using as comparison metric the fitting quality of the sparse models. We show the Pearson correlation coefficient of the empirical data and our sparse models predictions, as a function of the density, for the first moments (panel a), the two-site (panel b) and the most relevant three-site (panel c) connected correlations, respectively. Among the three procedures, that based on the two-site frequencies gives the poorest results, as it always provides the lowest Pearson up to density ≈3% where the algorithm fails to converge, meaning that it is no more able to fit the statistics associated with the non-zero parameters. The decimation based on the coupling strength outperforms the frequencies-based one but the Pearson coefficients, for all comparison metrics, is systematically lower than that of the symKLDbased decimation.

In addition to the fitting quality, we compare the three methods looking at the contact prediction PPV curves, shown in the top panel of Fig. 13, varying the model density. It is worth noting that all procedures, for all densities (except 3.2% using a frequency-based measure) perform equally well.

We also considered a standard network selection strategy, in which we first learn a series of dense models with a 1 -norm regularization at different strength γ, i.e. Eq. ( 2) for the couplings is modified to After selection, the sparse PMs is trained again keeping the decimated couplings to zero, but without the 1 -norm regularization for the nondecimated couplings, until convergence. The results for this procedure are reported in Fig. 14, and are outperformed by the symDKL-based procedure.

δJ ij (a, b) = η J [f ij (a, b) -p ij (a, b)] -γsgn(J ij (a, b)) . ( C1

Decimated couplings

As mentioned in the previous section, the couplings that are decimated at each iteration are either associated ble couplings. As the model becomes sparser and sparser the fraction reduces and reaches about 30 % for the sparsest models: here about 70 % of the decimated couplings are associated with a rich statistics but nonetheless their contribution to the Boltzmann weight is negligible.

Initialization of the learning for the dense Boltzmann machine

An intrinsic difficulty arises when comparing statistical models for protein sequences: the set of parameters that are able to reproduce the empirical statistics well and also give a good contact prediction is not unique. Therefore, giving a clear interpretation of the fields and the couplings of the inferred Potts model, i.e. to detect which variables are sufficient to characterize the target ensemble of protein sequences, is a challenging task. When the sufficient set of observables is not known, and one attempts to fit all possible pairwise couplings and single-site statistics through the Boltzmann machine learning, it is common to encounter 'flat' directions of the log-likelihood landscape, where the learning usually converges (as any attempt at modifying the parameters does not lead to any significant improvement). The parameters found at convergence thus strongly depend on the initial conditions.

Here we evaluate how the results of the decimation procedure are affected by the dense model used as starting point, which in turn depends on the initial conditions of the parameters. For this comparison, we consider three distinct initial conditions for the initial learning of the dense model: (a) the profile model (h = h profile , J = 0, used for the results presented in the main text), (b) the parameters from pseudo-likelihood maximization (h = h plmDCA , J = J plmDCA ), as implemented in plmDCA [10], and (c) a null initial condition for all model parameters (h = 0, J = 0). We then let the Boltzmann machine learning converge, and we use the converged Potts model as the starting model of the decimation run described in the main text.

In the right panels of Fig. 12 we show the Pearson correlation coefficients between the empirical frequencies f i (a) and the model frequencies p i (a) (in panel (a)), and the two-site and three-site connected correlations of the empirical data and of the sparse models, for panels (b) and (c) respectively. When all parameters are initialized to profile we reach the larger Pearson correlation coefficients, for all the three measures and for all densities. The plmDCA and zeros initializations have comparable results, and they reach Pearson correlation coefficients equal to those of the profile initialization only for the first moment in the high density regime.

In addition to the fitting quality, we can compare the three different initializations through the contact map prediction. We observe in Fig. 13 that all the three strategies, independently of the density, provide very similar contact prediction as the associated PPV curves completely overlap.

Online learning

In our decimation protocol, we proceed with a new decimation step only when the learning has reached con- vergence. Starting from a well converged dense Potts model, and decimating only 1% of the couplings at the time, allows us to modify smoothly the remaining parameters during the decimation. Indeed, we empirically observe that most of the times two consecutive decimations are separated by just a few learning steps. However, the entire protocol requires to learn a dense model first, which can be time-consuming.

We thus explored an alternative strategy in which the decimation is performed on-line, i.e. within a unique learning run. Here the decimation step is applied either because the learning has performed K steps or because it has reached the tolerance required for convergence. In these experiments, we start from a set of parameters corresponding to the profile model (as in the protocol illustrated in the main text) for PF00076 and we proceed with the decimation step every K = 10, 20, 40 steps. In Fig. 16 we compare the Pearson correlation coefficient between the one-site frequencies (panel a), two-site (b) and three-site (c) connected correlations, of the data and the models obtained by the two different strategies: we refer to the conventional method as converged (corresponding to K → ∞) while the on-line learning method is characterized by the number K of steps. It is worth noting that, at convergence, both strategies, and independently of K, reach the same fitting quality even in the three-site connected correlations. For completeness, we show in Fig. 17 the contact prediction performance of the converged and on-line runs for densities equal to 72% (panel a) and 3.2% (panel b). In the denser case (when we consider 72% of non-zero couplings) the converged run outperforms the on-line learning for any K. This can be explained by the poor fitting quality reached by the on-line runs at the initial steps of the algorithm, that is when the model is still inaccurate in fitting the two-site frequencies. It is worth noting that, in the sparse regime, i.e. for density equals to 3.2%, all the strategies show comparable results, qualitatively similar to the performance of the dense case (Fig. 17a).

Although the results of the on-line run resemble those of the converged run for the very sparse models, the online procedure is not always advantageous from the point of view of the running time. We notice that, depending on the family, a unique learning-decimation run may have problems fitting the statistics, i.e. to converge, because the decimation affects and 'deviates' the learning of the machine, for small K. To cure this issue, one may think of increasing the number of steps between each decimation. However, this results in a very slow procedure, because we remove 1% of the couplings every (large) K steps. Instead, if the model is well converged first, then convergence is achieved quite fast after each decimation, resulting in a faster procedure overall.

Appendix D: Sequences similarity

The defining feature of generative models is the ability to generate configurations that are statistically equivalent to those used within the training process, but substantially different in the residue composition, i.e. a good generative model should not just reproduce the sequences of the training set. Hence, it is important to quantify the distances between generated samples and the training data. For this purpose, we employed the following metrics, introduced in [45,46]: A proper generative model would produce comparable D st and D ss and, concurrently, the two measures must be sufficiently large (practically 20% of sequence similarity is required for good training sets). This corresponds to a scenario where generated sequences are variable (large D ss ), and similarly distant to natural or the other generated sequences (D ss D st ). This corresponds to a scenario where the average distance between each pair of generated sequences is comparable to that obtained between the two ensembles t and s: therefore, the generated synthetic sequences are indistinguishable from the natural sequences using distance based methods (like nearest-neighbor classification, distance based clustering). A similar argument can be applied to D ts . In of similar (correlated) sequences and, as a consequence, the D tt is significantly smaller than the other distance metrics. Regarding D ts , D st and D ss we notice that, as the density of the couplings decreases, the distances remain unchanged up to a density in the range 10% -20%, depending on the family. Then the minimum average distance significantly increases which suggests that the synthetic sequences are distributed more broadly in the sequence space as the number of model parameters decreases. Besides, the difference between D ts , D st , and D ss decreases for most of the protein families, in the sparse regime, suggesting that the synthetic sequence ensembles and the set of the natural sequences become more and more statistically similar for increasing sparsity. We can conclude that, according to these metrics, the decimation improves the generative properties of the model.

D Y (x) = min y∈Y D(x, y) , D XY = 1 N X N X n=1 D Y (x n ) . (

Interpretation of couplings

Here are some additional results related to the three questions we asked at the beginning of this section.

Discrepancy of contacts and non-contacts

Since the accuracy of the contact predictions is maintained even in the case of the very low coupling densities (d ∼ 5%), the non-decimated couplings may tend to correspond with contacts. Intuitively, the difference between the probability of couplings for contacts and non-contacts is expected to becomes more noticeable as the density d decreases. Here, probabilities of couplings for contacts, p c (J), and non-contacts, p n (J), are defined as the probability densities of interactions J ij (a, b) for distances between residuepairs i < j smaller than 8 Å and greater than 8 Å , respectively. We introduce the Jensen-Shannon divergence (JSD), a distance between two probability distributions1

JSD tot = 1 2 D KL (p c q) + 1 2 D KL (p n q) = +∞ -∞ dJf (J) , (3.1)
where q = (p c + p n )/2 and f = p c log(p c /q) + p n log(p n /q). We define the relative cumulative contribution of couplings smaller than J to JSD as We can observe that JSD(J) changes more abruptly around J ∼ 0 for the dense model. In contrast, this change is smoother and in a broader region for the sparse mode. In particular, there are significant contributions to JSD(J) in the tails of the distribution. Apparently, both contact and non-contact coupling parameters could increase as decreasing the coupling density d (cf. Fig. 9 in [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF]). However, the rate of increase in the contact coupling parameters is much more pronounced than that of non-contact coupling parameters.

JSD(J) = J -∞ dJ f (J ) JSD tot . ( 3
Empirically, residue contact predictions by the standard {F AP C ij } are not substantially changed under the decimations, whereas coupling distributions between contacts and non-contacts become more distinguishable. It would be interesting to exploit this discrimination in further studies.

By removing redundant couplings, the remaining couplings tend to be more enhanced to compensate for the removed couplings. The decimated weak correlations can probably be reproduced by contact couplings, whereas remaining non-contact couplings cannot significantly affect decimated correlations. Therefore, contact couplings are enhanced while non-contact couplings remain small.

Non-decimated and non-structural couplings

As shown before and in the article, large couplings J ij (a, b) that do not correspond to contact pairs remain in the set of non-decimated couplings (even at d = 1.6%, there are non-contact couplings |J| > 0.5, cf. Fig. 3.1). Such non-decimated and non-contact couplings are also considered essential parameters in KLD-based parameter selection. Sometimes, nondecimated couplings correspond to too spatially distant residue pairs, even for long-range Coulomb interactions. Several explanations are possible for these non-decimated couplings. They could be associated with the underlying phylogeny, distant allosteric interactions and/or the molecular dynamics of the protein.

Fig. 3.2 shows some coupling matrices J ij containing particularly strong couplings J ij (a, b) in the sparse model (d = 1.6%). The coupling matrices of the sparse model show that the number of strong couplings pairs in contact is significantly greater than for pairs that are not in contact 2 . By construction, the coupling matrices in the sparse model contain a small number of finite couplings. Therefore, it is likely that those couplings that are not decimated are statistically important. However, the interpretation for the non-contact and non-decimated large couplings remains unclear.

Notably, the numbers of large entries in sparse non-contact coupling matrices are significantly fewer than the case in sparse contact coupling matrices. Therefore, instead of taking the L2 norm for converting the coupling matrices to Frobenius norm, it could be interesting to apply L1 or L0 norm to improve residue-contact predictions in further studies. ). There are a few strong coupling entries in non-contact coupling matrices. However, the number of strong coupling is much smaller than the contact coupling matrices, and the difference is more pronounced in the sparse model.

As a last remark, the coupling matrices J ij ∈ R L×L tends to have nonzero coupling parameters only between certain amino-acid states as shown in Fig. 3.2. Therefore, J ij are essentially low-rank matrices and assuming the low-rank structure for coupling matrices, such as

J ij (a, b) ∼ µ v µ ij (a)v µ ij (b), or equivalently assume a statistical model p(A) ∝ exp µ i<j v µ ij (A i )v µ ij (A j ) 3 might
reflect well features of the estimated couplings parameters for protein sequences. The following section will introduce a statistical generative model that considers a low-rank structure for the coupling matrices.

Conclusion

In this chapter, we proposed pairwise-Potts models or Boltzmann machines for protein sequences that depend only on statistically important parameters. Using this method, we can decimate coupling parameters by about 90% while maintaining the reproducibilities of statistics and diversities of the generated samples, which are the expected properties as generative models. Note that a pairwise Potts model that reduces the number of coupling parameters by introducing a strong L1 regularization does not reproduce statistics as accurately as the proposed model (cf., Sec. 3.2).

By reducing coupling parameters, the model obtains the following advantages:

1. The original models are sensitive to the global perturbations of the coupling parameters. That is, these are in a state of over-learning, but the the proposed parameter-reduced models became more robust (cf. Fig. 4 in Sec. 3.2).

2. The increasing rate of parameters involved in physical contact is significantly greater than the case of non-contact couplings (cf. Fig. 3.1).

As Fig. 3.2 shows, the distribution of a finite number of elements in each coupling can be a useful measure to distinguish whether the pair is in contact or is not in contact. Further analysis should be done based on these insights.

The accuracy of residue-residue contacts may be further improved by evaluating whether the estimated coupling parameters are significantly different from the background noize by statistical tests [START_REF] Xu | Statistical properties of interaction parameter estimates in direct coupling analysis[END_REF][START_REF] Xu | Inverse finite-size scaling for high-dimensional significance analysis[END_REF].

Chapter 4

Selection of variables and low-rank model

Motivation

Despite their great success, DCA-based methods have drawbacks. In most situations, they depend on too many model parameters that need to be learned with a limited number of protein sequences. Thus over-learning problems are likely to happen. Moreover, the number of DCA parameters is necessarily high in order for these models to be generative, but a majority of these parameters are notably small and do not correspond to spatial interactions. Lastly, the choice of statistical variables or observables in MaxEnt modeling is still subjective. It might obscure the underlying simpler model for protein sequences.

As mentioned in Sec. 3.3.4, the coupling matrices can be low-rank matrices (cf. Fig. 3.2 ). This result is consistent biologically. Amino acids located on the protein surface are typically hydrophilic, whereas, those located in the protein core are normally hydrophobic. Therefore each coupling matrix shows specific amino-acid preferences. As a consequence, coupling matrices can exhibit a low-rank structure. Regarding these problems and the nature of the coupling matrix, the naturally emerging idea is to introduce a lowrank structure into the PPM. Moreover, by assuming the low-rank coupling matrices, effective number of parameter can be reduced.

Our contributions to this study are:

1. We show that even with a number of hidden variables (e.g., P = 20 -40), which is sufficiently smaller than the coupling matrix size qL, RBMs can reproduce protein sequence statistics (Pearson values of two-point correlations are ∼ 0.9).

2. We have proposed a framework that selects patterns based on the likelihood contributions. Such selected patterns can classify protein subfamilies successfully.

3. We show RBM with Gaussian hidden variables learns patterns similar to the attractive mfHP patterns (Sec. 4.3.1).

4. We show the rotation invariance in the space of the pattern indices can be fixed by using attractive mfHP patterns as initial conditions of RBM patterns (Sec. 4.3.2).

5. We show the fewer patterns in RBMs, the more robust RBM statistics will be to the global perturbations of the parameters (Sec. 4.3.3). Statistical models for families of evolutionary related proteins have recently gained interest: In particular, pairwise Potts models as those inferred by the direct-coupling analysis have been able to extract information about the three-dimensional structure of folded proteins and about the effect of amino acid substitutions in proteins. These models are typically requested to reproduce the one-and two-point statistics of the amino acid usage in a protein family, i.e., to capture the so-called residue conservation and covariation statistics of proteins of common evolutionary origin. Pairwise Potts models are the maximum-entropy models achieving this. Although being successful, these models depend on huge numbers of ad hoc introduced parameters, which have to be estimated from finite amounts of data and whose biophysical interpretation remains unclear. Here, we propose an approach to parameter reduction, which is based on selecting collective sequence motifs. It naturally leads to the formulation of statistical sequence models in terms of Hopfield-Potts models. These models can be accurately inferred using a mapping to restricted Boltzmann machines and persistent contrastive divergence. We show that, when applied to protein data, even 20-40 patterns are sufficient to obtain statistically close-to-generative models. The Hopfield patterns form interpretable sequence motifs and may be used to clusterize amino acid sequences into functional subfamilies. However, the distributed collective nature of these motifs intrinsically limits the ability of Hopfield-Potts models in predicting contact maps, showing the necessity of developing models going beyond the Hopfield-Potts models discussed here. DOI: 10.1103/PhysRevE.100.032128

Article

I. INTRODUCTION

Thanks to important technological advances, exemplified, in particular, by next-generation sequencing, biology is currently undergoing a deep transformation towards a data-rich science. As an example, the number of available protein sequences deposited in the UNIPROT database was about 10 6 in 2004, crossed 10 × 10 6 in 2010, and 100 × 10 6 in 2018, despite an important reorganization of the database in 2015 to reduce redundancies and, thus, limit the database size [1]. On the contrary, proteins with detailed experimental knowledge are contained in the manually annotated SWISSPROT subdatabase of UNIPROT. Although their number remained almost constant and close to 500 000 over the past decade, the knowledge about these selected proteins has been continuously extended and updated.

This rapidly growing wealth of data is presenting both a challenge and an opportunity for data-driven modeling approaches. It is a challenge because for less than 0.5% of all known protein sequences, at least, some knowledge going beyond sequence is available. Applicability of standard supervised machine-learning approaches is, thus, frequently limited. However, more importantly, it is an opportunity since protein-sequence databases, such as UNIPROT, are not large sets of unrelated random sequences but contain structured functional proteins resulting from natural evolution.

In particular, protein sequences can be classified into socalled homologous protein families [2]. Each family contains protein sequences, which are believed to share common ancestry in evolution. Such homologous sequences typically show very similar three-dimensional folded structures and closely related biological functions. Put simply, they can be seen as equivalent proteins in different species or in different pathways of the same species. Despite this high level of structural and functional conservation, homologous proteins may differ in more than 70-80% of their amino acids. Detecting homology between a currently uncharacterized protein and a well-studied one [3,4] is, therefore, the most important means for computational sequence annotation, including proteinstructure prediction by homology modeling [5,6].

To go beyond such knowledge transfer, we can explore the observable sequence variability between homologous proteins since it contains its own important information about the evolutionary constraints acting on proteins to conserve their structure and function [7]. Typically, very few random mutations do actually destabilize proteins or interrupt their function. Some positions need to be highly conserved, whereas others are permissive for multiple mutations. Observing sequence variability across entire homologous protein families, and relating them to protein structure, function, and evolution, is, therefore, an important task [8].

Over the past years, inverse statistical physics [9] has played an increasing role in solving this task. Methods, such as direct-coupling analysis (DCA) [10,11] or related approaches [12,13] allow for predicting protein structure [14,15], mutational effects [16][17][18], and protein-protein interactions [19]. However, many of these methods depend on huge numbers of typically ad hoc introduced parameters, making these methods data hungry and susceptible to overfitting effects.

In this paper, we describe an attempt to substantially reduce the amount of parameters and to select them systematically using sequence data. Despite this parameter reduction, we aim at so-called generative statistical models: Samples drawn from these models should be statistically similar to the real data, even if similarity is evaluated using statistical measures, which were not used to infer the model from data.

To this aim, we first review in Sec. II some important points about protein-sequence data, maximum-entropy (MaxEnt) models of these data, in general, profile, and DCA models, in particular. In Sec. III, we introduce a way for rational selection of so-called sequence motifs, which generalizes maximumentropy modeling. The resulting Hopfield-Potts models are mapped to restricted Boltzmann machines (RBMs) (recently introduced independently for proteins in Ref. [20]) in Sec. IV to enable efficient model inference and interpretation of the model parameters. Section V is dedicated to the application of this scheme to some exemplary protein families. The conclusion and outlook in Sec. VI are followed by some technical Appendices.

II. A SHORT SUMMARY: SEQUENCE FAMILIES, MAXENT MODELS, AND DCA

To put our work into the right context, we need to review shortly some published results about the statistical models of protein families. After introducing the data format, we summarize the maximum-entropy approach typically used to justify the use of Boltzmann distributions for protein families together with some important shortcomings of this approach. Next, we give a concise overview over two different types of maximum-entropy models-profile models and direct-coupling analysis-which are currently used for protein sequences. For all cases, we discuss the strengths and limitations, which have motivated our current paper.

A. Sequence data

Before discussing modeling strategies, we need to properly define what type of data is used. Sequences of homologous proteins are used in the form of multiple-sequence alignments (MSAs), i.e., rectangular matrices (A m i ) m=1,...,M i=1,...,L . Each of the rows m = 1, . . . , M of this matrix contains one aligned protein sequence A m = (A m 1 , . . . , A m L ) of length L. In the context of MSA, L is also called the alignment width, and M is called its depth. Entries in the matrix come from the alphabet A = {-, A, C, . . . ,Y } containing the 20 natural amino acids and the alignment gap "-." Throughout this paper, the size of the alphabet will be denoted by q = 21. In practice, we will use a numerical version of the alphabet, denoted by {1, . . . , q}, but we have to keep in mind that variables are categorical variables, i.e., there is no linear order associated with these numerical values.

The PFAM database [2] currently (release 32.0) lists almost 18 000 protein families. Statistical modeling is most successful for large families, which contain between 10 3 and 10 6 sequences. Typical lengths span the range of L = 30-500.

B. Maximum-entropy modeling

The aim of statistical modeling is to represent each protein family by a function P(A), which assigns a probability to each sequence A ∈ A L , i.e., to each sequence formed by L letters from the amino acid alphabet A. Obviously, the number of sequences, even in the largest MSA, is much smaller than the number q L -1 of a priori independent parameters characterizing P. So we have to use clever parametrizations for these models.

A commonly used strategy is based on the MaxEnt approach [21]. It starts from any number p of observables,

O μ : A L → R, μ = 1, . . . , p, (1)
which assign real numbers to each sequence. Only the values of these observables for the sequences in the MSA (A m ) go into the MaxEnt models. More precisely, we require the model to reproduce the empirical mean of each observable over the data,

∀ μ = 1, . . . p: A∈A L P(A)O μ (A) = 1 M M m=1 O μ (A m ). ( 2
)
In a more compact notation, we write O μ P = O μ MSA . Besides this consistency with the data, the model should be as unconstrained as possible. Its entropy has, therefore, to be maximized

- A∈A L P(A) ln P(A) -→ max . ( 3
)
Imposing the constraints in Eq. ( 2) via Lagrange multipliers λ μ , μ = 1, . . . , p, we immediately find that P(A) assumes a Boltzmann-like exponential form

P(A) = 1 Z exp ⎧ ⎨ ⎩ p μ=1 λ μ O μ (A) ⎫ ⎬ ⎭ . ( 4
)
Model inference consists in fitting the Lagrange multipliers such that Eqs. (2) are satisfied. The partition function Z guarantees normalization of P.

MaxEnt relates observables and the analytical form of the probability distribution, but it does not provide any rule on how to select observables. Frequently, prior knowledge is used to decide which observables are important and which are not. More systematic approaches, therefore, have to address, at least, the following two questions:

(1) Are the selected observables sufficient? In the best case, model P becomes generative, i.e., sequences A sampled from P are statistically indistinguishable from the natural sequences in the MSA (A m ) used for model learning. Although this is hard to test in full generality, we can select observables not used in the construction of the model and check if their averages in the model and over the input data coincide.

(2) Are the selected observables necessary? Would it be possible to construct a parameter-reduced, thus, more parsimonious, model of same quality? This question is very important due to, at least, two reasons: (a) The most parsimonious model would allow for identifying a minimal set of evolutionary constraints acting on proteins and, thus, offer deep insight into protein evolution; and (b) a reduced number of parameters would allow to reduce overfitting effects, which result from the limited availability of data (M q L ). Although there has been promising progress in the first question, cf. the next two subsections, our paper attempts to approach both questions simultaneously thereby going beyond standard MaxEnt modeling.

To facilitate the further discussion, two important technical points have to be mentioned. First, MaxEnt leads to a family of so-called exponential models where the exponent in Eq. ( 4) is linear in the Lagrange multipliers λ μ , which parametrize the family. Second, MaxEnt is intimately related to maximum likelihood. When we postulate Eq. ( 4) for the mathematical form of model P(A), and when we maximize the logarithmic likelihood,

L {λ μ }| A m i = M m=1 ln P(A m ), (5)
with respect to the parameters λ μ , μ = 1, . . . , P, we rediscover Eqs. (2) as the stationarity condition. The particular form of P(A) guarantees that the likelihood is convex, having only a unique maximum.

C. Profile models

The most successful approaches in statistical modeling of biological sequences are probably profile models [22], which consider each MSA column (i.e., each position in the sequence) independently. The corresponding observables are simply O ia (A) = δ A i ,a for all positions i = 1, . . . , L and all amino acid letters a ∈ A with δ being the standard Kronecker symbol. These observables, thus, just ask, if in a sequence A, amino acid a is present in position i. Their statistics in the MSA is, thus, characterized by the fraction,

f i (a) = 1 M M m=1 δ A m i ,a , (6)
of sequences having amino acid a in position i. Consistency of model and data requires marginal single-site distributions of P to coincide with f i ,

∀ i = 1, . . . , L, ∀ A i ∈ A: {A j | j =i} P(A) = f i (A i ). (7)
The MaxEnt model results as P(A) = L i=1 f i (A i ), which can be written as a factorized Boltzmann distribution,

P(A) = 1 Z exp i h i (A i ) , ( 8
)
where the local fields equal h i (a) = ln f i (a). Pseudocounts or regularization can be used to avoid infinite negative parameters for amino acids, which are not observed in some MSA column.

Profile models reproduce the so-called conservation statistics of a MSA, i.e., the heterogenous usage of amino acids in the different positions of the sequence. Conservation of a single or few amino acids in a column of the MSA is typically an indication of an important functional or structural role of that position. Profile models, frequently in their generalization to profile hidden Markov models [3,4,23], are used for detecting homology of new sequences to protein families, for aligning multiple sequences, and-using the conserved structural and functional characteristics of protein familiesindirectly for the computational annotation of experimentally uncharacterized amino acid sequences. They are, in fact, at the methodological basis of the generation of the MSA used here.

Despite their importance in biological sequence analysis, profile models are not generative. Biological sequences show significant correlation in the usage of amino acids in different positions, which are said to coevolve [7]. Due to their factorized nature, profile models are not able to reproduce these correlations, and larger sets of observables have to be used to obtain potentially generative sequence models.

D. Direct-coupling analysis

The DCA [10,11], therefore, includes also pairwise correlations into the modeling. The statistical model P(A) is not only required to reproduce the amino acid usage of single MSA columns, but also required to reproduce the fraction f i j (a, b) of sequences having, simultaneously, amino acid a in position i and amino acid b in position j for all a, b ∈ A and all 1 i < j L,

f i j (a, b) = 1 M M m=1 δ A m i ,a δ A m j ,b = A∈A L P(A)δ A i ,a δ A j ,b . (9)
The corresponding observables δ A i ,a δ A j ,b are, thus, products of pairs of observables used in profile models.

According to the general MaxEnt scheme described before, DCA leads to a generalized q-state Potts model,

P(A) = 1 Z exp ⎧ ⎨ ⎩ i< j J i j (A i , A j ) + i h i (A i ) ⎫ ⎬ ⎭ , ( 10
)
with heterogeneous pairwise couplings J i j (a, b) and local fields h i (a). The inference of parameters becomes computationally hard since the computation of the marginal distributions in Eq. ( 9) requires to sum over O(q L ) sequences. Many approximation schemes have been proposed, including message-passing [10], mean-field [11], Gaussian [13,24], and pseudolikelihood maximization [12,25] approximations. DCA and related global inference techniques have found widespread applications in the prediction of protein structures, of protein-protein interactions, and of mutational effects, demonstrating that amino acid covariation as captured by f i j contains biologically valuable information. Although these approximate inference schemes do not lead to generative models-not even f i and f i j are accurately reproduced-recently, very precise but time-extensive inference schemes based on Boltzmann-machine learning have been proposed [26][27][28][29]. Astonishingly, these models do not only reproduce the fitted one-and two-column statistics of the input MSA, but also reproduce nonfitted characteristics, such as the three-point statistics f i jk (a, b, c) or the clustered organization of sequences in sequence space. These observations strongly suggest that pairwise Potts models as inferred via DCA are generative models, i.e., that the observables used in DCA-amino acid occurrence in single positions and in position pairs-are actually defining a (close to) sufficient statistics. In a seminal experimental work [30], the importance of respecting pairwise correlations in amino acid usage in generating small artificial but folding protein sequences was shown.

However, DCA uses an enormous amount of parameters. There are independent couplings for each pair of positions and amino acids. In the case of a protein of limited length L = 200, the total number of parameters is close to 10 8 . Very few of these parameters are interpretable in terms of, e.g., contacts between positions in the three-dimensional protein fold. We would, therefore, expect that not all of these observables are really important to statistically model protein sequences. On the contrary, given the limited size (M = 10 3 -10 4 ) of most input MSAs, the large number of parameters makes overfitting likely, and quite strong regularization is needed. It would, therefore, be important to devise parameter-reduced models as proposed in Ref. [31] but without giving up on the generative character of the inferred statistical models.

III. FROM SEQUENCE MOTIFS TO THE HOPFIELD-POTTS MODEL

Seeing the importance of amino acid conservation in proteins and of profile models in computational sequence analysis, we keep Eqs. (7), which link the single-site marginals of P(A) directly to the amino acid frequencies f i (a) in single MSA columns. Furthermore, we assume that the important observables for our protein-sequence ensemble can be represented as so-called sequence motifs,

O μ (A) = i ω μ i (A i ), μ = 1, . . . , p, (11)
which are linear additive combinations of single-site terms.

In sequence bioinformatics, such sequence motifs are widely used, also under alternative names, such as position-specific scoring and weight matrices, cf. Refs. [32,33]. Note that, in difference to the observables introduced before for profile or DCA models, motifs constitute collective observables potentially depending on the entire amino acid sequence.

Let us assume for a moment that these motifs, or more specifically the corresponding ω matrices, are known. We will address their selection later. For any model P reproducing the sequence profile, i.e., for any model fulfilling Eqs. (7), also the ensemble average of O μ is given

A P(A)O μ (A) = i,a ω μ i (a) f i (a). ( 12
)
The empirical mean of these observables, therefore, does not contain any further information about the MSA statistics beyond the profile itself. The key step is to consider also the variance, or the second moment,

1 M m [O μ (A m )] 2 = i, j,a,b ω μ i (a)ω μ j (b) f i j (a, b), ( 13
)
as a distinct feature characterizing the sequence variability in the MSA, which has to be reproduced by the statistical model P(A). This second moment actually depends on combinations of f i j , which were introduced in DCA to account for the correlated amino acid usage in pairs of positions. The importance of fixing this second moment becomes clear in a very simple example: Consider only two positions {1, 2} and two possible letters {A, B}, which are allowed in these two positions. Let us assume further that these two letters are equiprobable in these two positions, i.e., f

1 (A) = f 1 (B) = f 2 (A) = f 2 (B) = 1/2.
Assume further a single motif to be given by ω 1 (A) = ω 2 (A) = 1/2, ω 1 (B) = ω 2 (B) = -1/2. In this case, the mean of O equals zero. We further consider two cases:

(1) Uncorrelated positions: In this case, all words AA , AB, BA, and BB are equiprobable. The second moment of O, thus, equals 1/2.

(2) Correlated positions: As a strongly correlated example, only the two words AA and BB are allowed. The second moment of O, thus, equals 1.

We conclude that an increased second moment (or variance) of these additive observables with respect to the uncorrelated case corresponds to the preference of combinations of letters or entire words; this is also the reason why they are the called sequence motifs.

Including, therefore, these second moments as conditions into the MaxEnt modeling, our statistical model takes the shape,

P A| λ μ , h i (a), ω μ i (a) = 1 Z exp ⎧ ⎨ ⎩ p μ=1 λ μ L i, j=1 ω μ i (A i )ω μ j (A j ) + L i=1 h i (A i ) ⎫ ⎬ ⎭ , (14)
with Lagrange multipliers λ μ , μ = 1, . . . , p, imposing means (13) to be reproduced by the model, and h i (a), i = 1, . . . , L, a ∈ A, to impose Eqs. (7).

The Hopfield-Potts model: from MaxEnt to sequence-motif selection

As mentioned before, an important limitation of MaxEnt models is that they assume certain observables to be reproduced, but they do not offer any strategy on how these observables have to be selected. In the case of Eq. ( 14), this accounts, in particular, to optimizing the values of the Langrange parameters λ μ to match the ensemble averages over P(A) with the sample averages Eq. ( 13) over the input MSA. As mentioned before, this corresponds also to maximizing the logarithmic likelihood of these parameters given MSA and the ω matrices describing the motifs,

L {λ μ , h i (a)}| A m i , ω μ i (a) = M m=1 ln P A m | λ μ , h i (a), ω μ i (a) . ( 15
)
The important, even if quite straightforward, step from MaxEnt modeling to motif selection is to optimize the likelihood also over the choice of all possible ω matrices as parameters. To remove degeneracies, we absorb the Lagrange multipliers λ μ into the motif matrix ω μ , and introduce

ξ μ i (a) = λ μ ω μ i (a), i = 1, . . . , L, μ = 1, . . . , p, a ∈ A . ( 16
)
032128-4

The model in Eq. ( 14), thus, slightly simplifies into

P A| h i (a), ξ μ i (a) = 1 Z exp ⎧ ⎨ ⎩ p μ=1 L i, j=1 ξ μ i (A i )ξ μ j (A j ) + L i=1 h i (A i ) ⎫ ⎬ ⎭ , (17)
with parameters, which have to be estimated by maximum likelihood, ĥi (a), ξ μ i (a)

= argmax {h i (a),ξ μ i (a)} M m=1 ln P A m | h i (a), ξ μ i (a) . ( 18
)
Our model becomes, therefore, the standard Hopfield-Potts model, which has been introduced in Ref. [31] in a mean-field treatment, and the sequence motifs equal, up to the rescaling in Eq. ( 16), the patterns in the Hopfield-Potts model. The mean-field treatment of Ref. [31] has both advantages and disadvantages with respect to our present paper: On one hand, the largely analytical mean-field solution allows to relate the Hopfield-Potts patterns ξ μ to the eigenvectors of the Pearson-correlation matrix of the MSA, and their likelihood contributions to a function of the corresponding eigenvalues. This is, in particular, interesting since not only the eigenvectors corresponding to large eigenvalues were found to contribute-as one might expect from the apparent similarity to principal-component analysis (PCA)-but also the smallest eigenvalues lead to large likelihood contributions. However, the mean-field treatment leads to a nongenerative model, which does not even reproduce precisely the single-position frequencies f i (a). The aim of this paper is to reestablish the generative character of the Hopfield-Potts model by more accurate interference schemes without losing too much of the interpretability of the mean-field approximation.

The model in Eq. ( 17) contains now an exponent, which is nonlinear in the parameters ξ μ . As a consequence, the likelihood is not convex anymore, and possibly many local likelihood maxima exist. This is also reflected by the fact that any p-dimensional orthogonal transformation of ξ μ leaves the probability distribution P(A) invariant, thus, leading to an equivalent model.

IV. INFERENCE AND INTERPRETATION OF HOPFIELD-POTTS MODELS

A. The Hopfield-Potts model as a restricted Boltzmann machine

The question how many and which patterns are needed for generative modeling, therefore, cannot be answered properly within the mean-field approach. We, therefore, propose a more accurate inference scheme based on RBM learning [34,35], exploiting an equivalence between Hopfield models and RBM originally shown in Ref. [36]. To this aim, we first perform p Hubbard-Stratonovich transformations to linearize the exponential in ξ μ ,

P(A) = 1 Z R p p μ=1 dx μ exp ⎧ ⎨ ⎩ i,μ x μ ξ μ i (A i ) + i h i (A i ) - 1 2 μ (x μ ) 2 , ( 19
)
with Z containing the normalizations both of the Gaussian integrals over the new variables x μ and the partition function of Eq. ( 14). The distribution P(A) can, thus, be understood as a marginal distribution of

P(A, x) = 1 Z exp ⎧ ⎨ ⎩ i,μ x μ ξ μ i (A i ) + i h i (A i ) - 1 2 μ (x μ ) 2 ⎫ ⎬ ⎭ , ( 20
)
which depends on the so-called visible variables A = (A 1 , . . . , A L ) and the hidden (or latent) variables x = (x 1 , . . . , x p ). It takes the form or a particular RBM with a quadratic confining potential for x μ : The important point is that couplings in the RBM form a bipartite graph between visible and hidden variables, cf. Fig. 1. RBM may have more general potentials u μ (x μ ) confining the values of the new random variables x μ . This fact has been exploited in Ref. [20] to cope with the limited number of sequences in the training MSA. However, in our paper, we stick to quadratic potentials in order to keep the equivalence to Hopfield-Potts models, and thus, the interpretability of patterns in terms of pairwise residue-residue couplings via Eq. ( 17).

B. Parameter learning by persistent contrastive divergence

Maximizing the likelihood with respect to the parameters leads, for our RBM model, to the stationarity equations,

1 M m δ A m i ,a = δ A i ,a P(A,x) , 1 M m δ A m i ,a x μ P(x|A m ) = δ A i ,a x μ P(A,x) (21)
for all i, a, and μ; the difference of both sides equals the gradient of the likelihood in the direction of the corresponding parameter. Although the first line matches the standard Max-Ent form-sample and ensemble average of an observable have to coincide, the second line contains a mixed sampleensemble average on its left-hand side. Since the variables x μ are latent and, thus, not contained in the MSA, an average over their probability P(x|A m ) conditioned to the sequences A m in the MSA has to be taken. Having a P dependence on both sides of Eqs. ( 21) is yet another expression of the nonconvexity of the likelihood function.

Model parameters h i (a) and ξ μ i (a) have to be fitted to satisfy the stationarity conditions Eq. ( 21). This can be performed iteratively: Starting from arbitrarily initialized model parameters, we determine the difference between the left-and the right-hand sides of this equation and use this difference to update parameters (i.e., we perform gradient ascent of the likelihood); each of these update steps is called an epoch of learning. A major problem is that the exact calculation of averages over the (L + p)-dimensional probability distribution P is computationally infeasible. It is possible to estimate these averages by Markov chain Monte Carlo (MCMC) sampling, but efficient implementations are needed since accurate parameter learning requires, in practice, thousands of epochs. To this aim, we exploit the bipartite structure of RBM: Both conditional probabilities P(A|x) and P(x|A) are factorized.

032128-5

MSA (training data)

A t=0

A t=1 A t=k

x t=0 x t=1 x t=k epoch 1

A t=0 A t=1 A t=k x t=0 x t=1 x t=k epoch 2 A t=0 A t=1 A t=k x t=0 x t=1 x t=k

C. Determining the likelihood contribution of single Hopfield-Potts patterns

It is obvious that the total likelihood grows monotonously when increasing the number p of patterns ξ μ . It is, therefore, important to develop criteria, which tell us if patterns are more or less important for modeling the protein family. To this aim, we estimate the contribution of single patterns to the likelihood by comparing the full model with a model where a single pattern ξ μ has been removed, whereas the other p -1 patterns and the local fields have been retained. The corresponding normalized change in logarithmic likelihood reads

μ = 1 M M m=1 [ln P(A m ) -ln P -μ (A m )], (22)
where P -μ has the same form as given in Eq. ( 14) for P but with pattern ξ μ = {ξ μ i (a); i = 1, . . . , L, a ∈ A} removed. Plugging Eq. ( 14) into Eq. ( 22), we find

μ = 1 M M m=1 L i=1 ξ μ i A m i 2 + ln Z -μ Z . ( 23
)
The likelihood difference depends, thus, on the ratio of the two partition functions Z and Z -μ . Although each of them is individually intractable due to the exponential sum over q L sequences, the ratio can be estimated efficiently using importance sampling. We write

Z -μ Z = 1 Z A∈A L exp ⎧ ⎨ ⎩ ν =μ L i, j=1 ξ ν i (A i )ξ ν j (A j ) + L i=1 h i (A i ) ⎫ ⎬ ⎭ = A∈A L P(A) exp ⎧ ⎨ ⎩ - L i, j=1 ξ μ i (A i )ξ μ j (A j ) ⎫ ⎬ ⎭ . ( 24
)
The last expression contains the average of an exponential quantity over P(A), so estimating the average by MCMC sampling of P might appear a risky idea. However, since P and P -μ differ only in one of the p patterns, the distributions are expected to overlap strongly, and sufficiently large samples drawn from P(A) can be used to estimate Z -μ /Z. Note that sampling is performed from P, so the likelihood contributions of all patterns can be estimated in parallel using a single large sample of the full model. Once these likelihood contributions are estimated, we can sort them, and identify and interpret the patterns of largest importance in our Hopfield-Potts model.

V. HOPFIELD-POTTS MODELS OF PROTEIN FAMILIES

To understand the performance of Hopfield-Potts models in the case of protein families, we have analyzed three protein families extracted from the PFAM database [2]: the Kunitzbovine pancreatic trypsin inhibitor domain (PF00014), the response regulator receiver domain (PF00072), and the RNA recognition motif (PF00076). They have been selected since they have been used in DCA studies before; in our case, RBM results will be compared to the ones of BMDCA, i.e., the generative version of DCA based on Boltzmann machine learning [29]. MSAs are downloaded from the PFAM database [2], and sequences with more than five consecutive gaps are removed; cf. Appendix B for a discussion of the convergence problems of PCD-based inference in the case of extended gap stretches. The resulting MSA dimensions for the three families are, in the order given before, L = 52/112/70 and M = 10 657/15 000/10 000. As can be noted, the last two MSAs have been subsampled randomly since they were very large, and the running time of the PCD algorithm is linear in the sample size. The MSA for PF00072 was chosen to be slightly larger because of the longer sequences in this family.

In the following sections, results are described in detail for the PF00072 response regulator family. The results for the other protein families are coherent with the discussion; they are moved to Appendix B to seek the conciseness of our presentation.

A. Generative properties of Hopfield-Potts models

PCD is able, for all values of the pattern number p, to reach parameter values satisfying the stationarity conditions Eqs. (21). This is not only true when these are evaluated using the PCD sample propagated via learning from epoch to epoch, but also when the inferred model is resampled using MCMC, i.e., when the right-hand side of Eqs. ( 21) is evaluated using an i.i.d. sample of the RBM.

In the leftmost column of Fig. 2 [panels (a.1)-(g.1)], this is shown for the single-site frequencies, i.e., for the first of Eqs. (21). The horizontal axis shows the statistics extracted from the original data collected in the MSA, whereas the vertical axis measures the same quantity in an i.i.d. sample extracted from the inferred model P(A, x). The fitting quality is comparable to the one obtained by BMDCA as can be seen by comparison with the last panel in the first column of Fig. 2.

The other two columns of the figure concern the generative properties of RBM: connected two-point correlations [panels (a.2)-(g.2) in Fig. 2] and three-point correlations [panels (a.3)-(g.3) in Fig. 2],

c i j (a, b) = f i j (a, b) -f i (a) f j (b), c i jk (a, b, c) = f i jk (a, b, c) -f i j (a, b) f k (c) -f ik (a, c) f j (b) -f jk (b, c) f i (a) + 2 f i (a) f j (b) f k (c), ( 25
)
with the three-point frequencies f i jk (a, b, c) defined in analogy to Eqs. ( 6) and ( 9). Note that, in difference to DCA, already the two-point correlations are not fitted directly by the RBM but only the second moments related to the Hopfield-Potts patterns. This becomes immediately obvious for the case of p = 0 where RBM reduces to simple profile models of statistically independent sites but remains true for all values of p < (q -1)L. Note also that connected correlations are used since the frequencies f i j and f i jk contain information about the fitted f i and, therefore, show stronger agreement between data and model. The performance of RBM is found to be, up to statistical fluctuations, monotonous in the pattern number p. As in the mean-field approximation [31], no evident overfitting effects are observed. Even if not fitted explicitly, as few as p = 20-40 patterns are sufficient to faithfully reproduce even the nonfitted two-and three-point correlations. This is very astonishing since only about 1.7-3.5% of the parameters of the full DCA model are used: The p patterns are given by p(q -1)L parameters, whereas DCA has (q -1) 2 ( L2 ) independently inferred couplings. The times needed for accurate inference decrease accordingly: In some cases, a slight decrease in accuracy of BMDCA is observed as compared to RBM with the largest p; this could be overcome by iterating the inference procedure for further epochs.

B. Strong couplings and contact prediction

One of the main applications of DCA is the prediction of contacts between residues in the three-dimensional protein fold, based only on the statistics of homologous sequences. To this aim, we follow Ref. [25] and translate q × q coupling matrices J i j (a, b) = p μ=1 ξ μ i (a)ξ μ j (b) for individual site pairs (i, j) into scalar numbers by first calculating their Frobenius norm,

F i j = F ji = a,b∈A J i j (a, b) 2 , ( 26
)
followed by the empirical average-product correction (APC),

F APC i j = F i j - F i• F • j F •• , ( 27
)
where the • denotes an average over the corresponding index,

F i• = 1 L -1 k F ik , F • j = 1 L -1 k F k j , ( 28
)
F •• = 2 L(L -1) k<l F kl .
The APC is intended to remove systematic nonfunctional bias due to conservation and phylogeny. These quantities are sorted, and the largest ones are expected to be contacts.

The results for several values of p and for BMDCA are depicted in Fig. 3(a): The PPV is the fraction of true positives (TPs) among the first n predictions as a function of n. TPs are defined as native contacts in a reference protein structure (PDB ID 3ilh [38] for PF00072) with a distance cutoff of 8 Å between the closest pair of heavy atoms forming each residue. Pairs in the vicinity along the peptide chain are not considered in this prediction since they are trivially in contact: In coherence with the literature standard, Fig. 3 going beyond the upper edge of the blue histogram is less pronounced for small p. However, in the procedure described before, these F APC i j values provide the first contact predictions.

The reduced capacity to detect contacts for small p is related to the properties of the Hopfield-Potts model in itself. Although the residue-residue contacts form a sparse graph, the Hopfield-Potts model is explicitly constructed to have a low-rank coupling matrix [J i j (a, b)]. It is, however, hard to represent a generic sparse matrix by a limited number of possibly distributed patterns. Hopfield-Potts models are more likely to detect distributed sequence signals than localized sparse ones. However, for larger pattern numbers p, we are able to detect more and more localized signals thereby improving the contact prediction until BM and Hopfield-Potts models become equivalent for p = (q -1)L.

This observation establishes an important limitation to the generative character of Hopfield-Potts models with limited pattern numbers: The applicability of DCA for residueresidue contact prediction has demonstrated that physical contacts in the three-dimensional structure of proteins introduce important constraints on sequence evolution. A perfectly generative model should respect these constraints and, thus, lead to a contact prediction being, at least, as good as the one obtained by full DCA, cf. also the discussion in the outlook of this article.

C. Likelihood contribution and interpretation of selected sequence motifs

So what do the patterns represent? In Sec. IV C, we have discussed how to estimate the likelihood contribution of patterns thereby being able to select the most important patterns in our model. Figure 4 displays the ordered contributions for different values of p. We observe that, for small p, the distribution becomes more peaked with few patterns having very large likelihood contributions. For larger p, the contributions are more distributed over many patterns, which collectively represent the statistical features of the data set. Patterns are very distributed, both in terms of the sites and in the amino acids with relatively large entries ξ i (a). This makes a direct interpretation of patterns without prior knowledge rather complicated. The distributed nature of patterns explains also why they are not optimal in defining localized contact predictions. Rather than identifying contacting residue pairs, the patterns define larger groups of sites, which are connected via a dense network of comparable couplings. However, as we will see in the next section, the sites of large entries in a pattern define functional regions of proteins, which are important in subensembles of proteins of strong (positive or negative) activity values along the pattern under consideration. In particular, we will show that the largest entries may have an interpretation connecting structure and function to sequence in protein subfamilies.

The middle column [panels (a.2)-(e.2)] of Fig. 5 shows a histogram of pattern-specific activities of single sequences, i.e., of

x μ (A) = N i=1 ξ μ i (A i ). ( 29
)
Note that, up to the rescaling in Eq. ( 16), these numbers coincide with the sequence motifs, introduced in Eq. ( 11) at the beginning of this article. They also equal the average value of the latent variable x μ given sequence A. The blue histograms result from the natural sequences collected in the training MSA. They coincide well with the red histograms, which are calculated from an i.i.d. MCMC sample of our Hopfield-Potts model, including the bimodal structure of several histograms. This is quite remarkable: The Hopfield-Potts model was derived, in the beginning of this paper, as the maximum-entropy model reproducing the first two moments of the activities {x μ (A m )} m=1•••M . Finding higher-order features, such as bimodality, is again an expression of the generative power of Hopfield-Potts models. 22) as Hopfield-Potts models of p -1 patterns with pattern ξ μ removed (vertical axis). The coherence of the correlations is strongly reduced when compared to the full model, which was shown in Fig. 2: Removal even of a single pattern has a strong global impact on the model statistics.

D. Sequence clustering

As already mentioned, some patterns show a clear bimodal activity distribution, i.e., they identify two statistically distinct subgroups of sequences. The number of subgroups can be augmented by using more than one pattern, i.e., combinations of patterns can be used to cluster sequences.

To this aim, we have selected three patterns (numbers 6, 13, and 14) with a pronounced bimodal structure from the model with p = 20 patterns. In terms of likelihood contribution, they have ranks 8, 4, and 1 in the contributions to the logarithmic likelihood, cf. Fig. 5.

The clustered organization of response-regulator sequences becomes even more evident in the two-dimensional plots characterizing, simultaneously, two activity distributions. The results for all pairs of the three patterns are displayed in Fig. 6, panels (a.1-(a.3). As a first observation, we see that the main modes of the activity patterns give rise to one dominant cluster. Smaller clusters deviate from the dominant one in a single pattern but show compatible activities in the other patterns-the two-dimensional plots, therefore, show typically an L-shaped sequence distribution and three clusters instead of the theoretically possible four combinations of activity models. It appears that single patterns identify the particularities of single subdominant sequence clusters.

We have chosen the response-regulator protein-domain family in this paper also due to the fact that it constitutes a functionally well studied and diversified family. Response regulators are predominantly used in bacterial signaling systems:

(1) In chemotaxis, they appear as single-domain proteins named CheY, which transmit the signal from kinase proteins (activated by signal reception) to flagellar motor proteins, which trigger the movement of the bacteria. CheY proteins can be identified in our MSA as those coming from singledomain proteins, i.e., with lengths compatible to the PF00072-MSA width L = 112. We have selected a sub-MSA consisting of all proteins with total sequence lengths between 110 and 140 amino acids.

( (3) Phosphorelays are similar to TCS but consist of more complex multicomponent signaling pathways. In these systems, found in bacteria and plants, response-regulator domains are typically fused to the histidine-sensor kinases. They do not act as transcription factors but transduct a signal to a phosphotransferase, which finally activates a down-stream transcription factor of the same architecture mentioned in the last paragraph. We identified a class of response-regulator domains, which are fused to a histidine kinase domain. In terms of domain architecture and protein length, this subfamily is extremely heterogenous.

Panel columns (a)-(f) in Fig. 6 show the activities of these five subfamilies. It is evident that distinct sub-MSAs fall actually into distinct clusters according to these three patterns:

( in Fig. 6, this pattern has the largest positive entries in the region of positions 80-90 and 100-110. Interestingly, these regions define the interface of OmpR-class transcriptionfactor homodimerization, cf. Ref. [39]. In accordance with this structural interpretation, we also find a periodic structure of period 3 to 4 of the large entries in the pattern, which reflects the fact that the interface is formed by two helices, which lead to a periodic exposure of amino acids in the protein surface.

( 3)] does not show a distinct distribution of activities according to the three selected patterns. It is located together with the CheYtype sequences. However, when examining all patterns, we find that pattern number 5 (ranked sixth according to the likelihood contribution) is almost perfectly discriminating the two.

(5) Last but not least, the response regulators fused to histidine kinases in phosphorelay systems [panels (f.1)-(f.3)] of Fig. 6 show a distinct activity distribution according to the third pattern, mixing a part of activities compatible with the main cluster, and others being substantially larger (this mixing results presumably from the previously mentioned heterogenous structure of this sub-MSA). Structurally known complexes between response regulators and histidine phosphotransferases (PDB ID 4euk [40], 1bdj [41]) show the interface located in residues 5-15, 30-32, and 50-55, regions being important in the corresponding pattern. It appears that the pattern selects the particular amino acid composition of this interface, which is specific to the phosphorelay sub-MSA.

These observations do not only show that the patterns allow for clustering sequences into sub-MSA, but also show that the discriminating positions in the patterns have a clear biological interpretation. This is very interesting since the analysis in Ref. [39] required a prior clustering of the initial MSA into sub-MSA, and the application of DCA to the individual sub-MSA. Here, we have inferred only one Hopfield-Potts model describing the full MSA, and the patterns automatically identify biologically reasonable subfamilies together with the sequence patterns characterizing them. The prior knowledge needed in Ref. [39] is not needed here; we use it only for the posterior interpretation of the patterns.

It is also important to remember that sequence clustering can be obtained by a technically simpler PCA. PCA is based 032128-12 on the leading eigenvectors of the data-covariance matrix, i.e., exclusively on the largest eigenvalues. The potential differences were already discussed in Ref. [31] in the context of the mean-field approximation of Hopfield-Potts models. It was shown that not only the eigenvectors with large eigenvalues lead to important contributions in likelihood, but also those corresponding to the smallest eigenvalues. Both tails of the spectrum are, thus, important for the statistical description of protein-sequence ensembles. A second drawback of PCA as compared to our approach is the nongenerative character of PCA. No explicit statistical model is learned, but the data covariance matrix is simply approximated by a low-rank matrix.

032128-13

VI. CONCLUSION AND OUTLOOK

In this paper, we have rederived Hopfield-Potts models as statistical models for protein sequences by selection of additive sequence motifs. Statistical sequence models are required to reproduce the first and second moments of the empirical motif distributions (i.e., over the MSA of natural sequences). Within a maximum-entropy approach, these motifs are found to be (up to a scaling factor) the Hopfield-Potts patterns defining a network of residue-residue couplings. In addition to the maximum-entropy framework, which is built upon known observables, the Hopfield-Potts model adds a step of variable selection: The probability of the sequence data is maximized over all possible selections of sequence motifs.

The quadratic coupling terms can be linearized using a Hubbard-Stratonovich transformation. When the Gaussian variables introduced in this transformation are interpreted as latent random variables, the Hopfield-Potts model takes the form of a restricted Boltzmann machine. This interpretation, originally introduced in Ref. [36], allows for the application of efficient inference techniques, such as persistent contrastive divergence and, therefore, for the accurate inference of the Hopfield-Potts patterns for any given MSA of a homologous protein family.

We find that Hopfield-Potts models acquire interesting generative properties even for a relatively small number of parameters (p = 20-40). They are able to reproduce nonfitted properties, such as higher-order covariation of residues. Also, the bimodality observed in the empirical activity distributions (i.e., the projection of the natural sequences onto individual Hopfield-Potts patterns) is not automatically guaranteed when using only the first two moments for model learning, but it is recovered with high accuracy in the activity distributions of artificial sequences sampled from the model. This observation is not only interesting in the context of generative-model learning, but also forms the basis of sequence clustering according to interpretable sequence motifs in the main text.

The Hopfield-Potts patterns, or sequence motifs, are typically found to be distributed over many residues thereby representing global features of sequences. This observation explains why Hopfield-Potts models tend to lose accuracy in residue-residue contact prediction as compared to the fullrank Potts models normally used in direct coupling analysis: The sparsity of the residue-residue contact network cannot be represented easily via few distributed sequence motifs, which describe more global patterns of sequence variability. Despite the fact that Hopfield-Potts models reproduce also nonfitted statistical observables, the loss of accuracy in contact prediction demonstrates that these models are not fully generative and alternative concepts for parameter reduction should be explored.

Individual sequences from the input MSA can be projected onto the Hopfield-Potts patterns, resulting in sequencespecific activity values. Some patterns show a monomodal histogram for the protein family. They introduce a dense network of relatively small couplings between positions with sufficiently large entries in the pattern without dividing the family into subfamilies. These patterns have great similarity to the concept of protein sectors, which was introduced in Refs. [42,43] to detect distributed modes of sequence coevolution. However, the conservation-based reweighting used in determining sectors is not present in the Hopfield-Potts model, and the precise relationship between both ideas remains to be elaborated. Other Hopfield-Potts patterns show bimodal activity distributions, leading to the detection of functional subfamilies. Since these are defined by, e.g., the positive vs the negative entries of the pattern, the entries of large absolute value in the patterns identify residues, which play a role similar to so-called specificity determining residues [44,45], i.e., residues, which are conserved inside specific subfamilies, but vary between subfamilies. Both concepts-sectors and specificity-determining residues-emerge naturally in the context of Hopfield-Potts families, even if their precise mathematical definitions differ and, thus, also their precise biological interpretations.

These observations open up new ways of parameter reduction in statistical models of protein sequences: The sparsity of contacts, which are expected to be responsible for a large part of localized residue covariation in protein evolution, has to be combined with the low-rank structure of Hopfield-Potts models, which detect distributed functional sequence motifs. However, distributed patterns may also be related to phylogenetic correlations, which are present in the data, cf. Ref. [46]. As has been shown recently in a heuristic way [47], the decomposition of sequence-data covariance matrices or coupling matrices into a sum of a sparse and a low-rank matrix can substantially improve contact prediction if only the sparse matrix is used.

Combining this idea with the idea of generative modeling seems a promising road towards parsimonious sequence 032128-16 models, which, in turn, would improve parameter interpretability and reduce overfitting effects, both limiting factors of current versions of DCA. In this context, it will also be interesting to explore more general regularization strategies which favor more localized sequence motifs or Hopfield-Potts patterns thereby unifying sparse and low-rank inference in a single framework of parameter-reduced statistical models for biological sequence ensembles.

Figures 789display the major results for the PF00014 protein family. PPV curves are calculated using PDB ID 5pti [48].

APPENDIX B: NOTES AND DETAILS ON INFERENCE METHODS

Regularization

In the case of limited data but many parameters, i.e., the case (Hopfield-)Potts models for protein families are in, the direct likelihood maximization in Eq. ( 18) can lead to overfitting effects, causing problems in sampling and parameter interpretation. To give a simple example, a rare and, therefore, unobserved event would be assigned zero probability, corresponding to (negative) infinite parameter values.

To cope with this problem, regularization is used. Regularization, in general, penalizes large (respectively, nonzero) parameter values and can be justified in Bayesian inference as a prior distribution acting on the parameter values. In this paper and following Ref. [20], we use a block regularization of the form

R(ξ, h) = η 0 p μ=1 i,a ξ μ i (a) 2 + qη 0 i,a h i (a) 2 , (B1)
with η 0 being a hyperparameter determining the strength of regularization. This regularization weakly favors sparsity of the patterns. We use η 0 = α 0 L/qM with α 0 = 0.0525 as default values throughout this paper. In the last section of this Appendix, we show that Hopfield-Potts inference is robust with respect to this choice.

Contrastive divergence vs persistent contrastive divergence a. Contrastive divergence does not reproduce the two-point statistics

CD is a method for training restricted Boltzmann machines similar to persistent contrastive divergence. Initialized in the original data, i.e., the MSA of natural amino acid sequences, a few sampling steps are performed in analogy to Fig. 1, and the kth step is used in the parameter update to approach a solution of Eq. ( 21). However, rather than continuing the MCMC sampling from this sample, the sample is re-initialized in the original data after each epoch. This has, a priori, advantages and disadvantages: The sample remains close to a good sample of the model in CD but far from a sample of the intermediate model with not yet converged parameters.

As can be seen in Fig. 13, after a sufficient number of epochs the statistics of the CD sample and the training data are perfectly coherent, the model appears to be converged. However, the connected two-point correlations are not well reproduced when resampling the inferred model with standard MCMC. Part of the empirically nonzero correlations are not reproduced and mistakenly assigned very small values in the inferred model. To understand this observation, we have selected the elements of the second panel, which show discrepancies between empirical and model statistics, cf. the inset in the figure. The corresponding values of (i, j, a, b) are strongly localized in the beginning and the end of the protein chain and correspond to the gap-gap statistics c i j (-, -). This gives a strong hint towards the origin of the problems in CD-based model inference: gap stretches, which exist in MSA of natural sequences, in particular, at the beginning and the end of proteins due to the local nature of the alignment algorithm used in PFAM. Those located at the beginning of the sequence start in position 1 and continue with only gaps until they are terminated by an amino acid symbol. They never start later than in position 1 or include individual internal amino acid symbols (analogous for the gap stretches at the end, which go up to the last position i = L).

In CD, only a few sampling steps are performed, so stretched gaps in the initialization tend to be preserved even if the associated gap-gap couplings are very weak. Basically, to remove a gap stretch, an internal position cannot be switched to an amino acid, but the gap has to be removed iteratively from one of its end points, namely, the one inside the sequence (i.e., not positions 1 or L). So, in CD, even small couplings are, thus, sufficient to reproduce the gap-gap statistics.

If resampling the same model with MCMC, parameters have to be such that gap stretches emerge spontaneously during sampling. This requires quite large couplings, actually, in BMDCA, gap-gap couplings between neighboring sites are the largest couplings of the entire Potts model. Using now the small couplings inferred by CD, these gap stretches do not emerge at sufficient frequency, and correspondingly the positions at the extremities of the sampled sequences appear less correlated.

b. Persistent contrastive divergence and transient oscillations in the two-point statistics

Persistent contrastive divergence overcomes this sampling issue by not reinitializing the sample after each epoch but by continuing the MCMC exploration in the next epoch with updated parameters.

As shown in the main text, PCD can actually be used to infer parameters, which lead to accurately reproduced twopoint correlations when i.i.d. samples are generated from the inferred model. However, during inference, we have observed transient oscillations, cf. Fig. 14 for an epoch where correlations in a subset of positions and amino acids are overestimated. An analysis of the positions and the spins involved in these deviations shows that again gap stretches are responsible.

The reason can be understood easily. Initially, PCD is not very different from CD. Gap stretches are present due to the correlation with the training sample, and only small gap-gap interactions are learned. However, after some epochs, the sample will lose the correlation with the training sample. Due to the currently small gap-gap correlations, gap stretches are lost in the PCD sample. According to our update rules, the corresponding gap-gap couplings will fastly increase. However, due to the few sampling steps performed in each PCD epoch, this growth will go on even when parameters would be large enough to generate gap stretches in an i.i.d. sample. Also, in the PCD sample, gap stretches will now emerge, but due to the overestimation of parameters, they will be more frequent than in the training sample, i.e., parameters start to decrease again. An oscillation of gap-gap couplings is induced.

The strength of these oscillations can be strongly reduced by removing samples with large gap stretches from the training data and train only on data with limited gaps. If the initial training set was large enough, the resulting models are even expected to be more precise since gap stretches do contain no or little information about the amino acid sequences under study. However, if samples are too small, the suggested pruning procedure may reduce the sample to an insufficient size for accurate inference. Care has, thus, to be taken when removing sequences.

Robustness of the results

As discussed before, we need to include regularization to avoid overfitting due to limited data. In Figs. 15 and16, we show the dependence of the inference results due to changes in the regularization strength over roughly two orders of magnitude. The first of the two figures shows the results for CD: Empirical connected two-point correlations are compared with i.i.d. samples of the corresponding models. We note that the results depend strongly on the regularization strength. For low regularization, the correspondence between model and MSA is low due to overfitting. At strong regularization, only part of the correlations is reproduced, we over-regularize and, thus, underfit the data. For each protein family and each number p of patterns, the regularization strength would have to be tuned.

032128-20

For PCD, the situation is fortunately much better; results are found to be very robust with respect to regularization, cf. Fig. 16. This allows us to choose one regularization strength across protein families and pattern numbers.

Mean-field Potts model for learning RBM

It is known that the RBM with Gaussian hidden variables is equivalent to the Hopfield-Potts (HP) model [START_REF] Cocco | Inference of hopfield-potts patterns from covariation in protein families: calculation and statistical error bars[END_REF][START_REF] Barra | On the equivalence of hopfield networks and boltzmann machines[END_REF]. As shown in [START_REF] Shimagaki | Selection of sequence motifs and generative hopfield-potts models for protein families[END_REF], a HP model can be represented as the marginalization of the joint probability distribution of p(A, x), which corresponds to RBM with Gaussian hidden variables,

p(A, x) = 1 Z exp   i,µ x µ ξ µ i (A i ) + L i h i (A i ) - L 2 P µ (x µ ) 2   , (4.1)
where Z is the partition function. The probability of a sequence p(A) can be understood as the consequence of the marginalized distribution of Eq. 4.1,

p(A) = R P p(A, x)dx = 1 Z exp   i,j 1 L P µ ξ µ i (A i )ξ µ j (A j ) + i h i (A i )   . (4.2)
Although RBMs can reproduce statistics of training data, mean-field HP (mfHP) models cannot serve reasonable generative models. Note that meanfield DCA overestimates coupling parameters, thous cannot also be used as a generative model. On the other hand, mfHP models are theoretically and numerically well-investigated, and mfHP patterns directly associate with data covariance matrices [START_REF] Cocco | High-dimensional inference with the generalized hopfield model: Principal component analysis and corrections[END_REF][START_REF] Cocco | Inference of hopfield-potts patterns from covariation in protein families: calculation and statistical error bars[END_REF][START_REF] Cocco | From principal component to direct coupling analysis of coevolution in proteins: Loweigenvalue modes are needed for structure prediction[END_REF]. This section aims to understand RBM patterns by comparing mfHP patterns. Our findings are as follows:

• RBM tends to preferentially learn patterns that are similar to the attractive mfHP patterns with significant likelihood contributions in the HP model (Sec. 4.3.1).

• Attractive mfHP patterns can be close to one of the optimal patterns for RBM. In other words, when the attractive mfHP patterns are used as the initial states of RBM, they keep patterns close to the initial states (Sec. 4.3.1).

• Introducing HP patterns into RBM helps to improve learning efficiency and control each pattern (Sec. 4.3.2).

In the entire section, we assumed a Gaussian prior distribution for the hidden variables. The following experiments were performed for the response regulator domain (Pfam ID, PF00072), but qualitatively similar results are obtained for other domain families.

Comparison between mean-field Hopfield-Potts model and RBM with Gaussian hidden variables

Overlaps between mfHP and RBM patterns -In the case of the mfHP model [START_REF] Cocco | Inference of hopfield-potts patterns from covariation in protein families: calculation and statistical error bars[END_REF], there are two types of patterns: Those corresponding to large eigenvalues of the Pearson correlation matrix (λ µ > 1) and the others associated with small eigenvalues (0 < λ µ < 1). These patterns are called attractive and repulsive [START_REF] Cocco | High-dimensional inference with the generalized hopfield model: Principal component analysis and corrections[END_REF][START_REF] Cocco | Inference of hopfield-potts patterns from covariation in protein families: calculation and statistical error bars[END_REF]. As established in Ref. [START_REF] Cocco | Inference of hopfield-potts patterns from covariation in protein families: calculation and statistical error bars[END_REF], distributions of entries of attractive patterns are typically dense. On the contrary, entries of repulsive patterns are typically sparse and strongly localized in some specific positions i and amino-acids a. Such localized entries are commonly associated with structural interactions, hence necessary for accurate contact predictions [START_REF] Cocco | High-dimensional inference with the generalized hopfield model: Principal component analysis and corrections[END_REF]. Ref. [START_REF] Cocco | Inference of hopfield-potts patterns from covariation in protein families: calculation and statistical error bars[END_REF] shows that the patterns, whose eigenvalues are significantly larger than 1 or markedly smaller than 1, contribute to the log-likelihood function of the mfHP model, cf. Eq. 2.34. Fig. 4.1.b shows the log-likelihood contributions as a function of eigenvalues.

It is important to examine how far the estimated patterns of the meanfield approximation are from the patterns of the exact HP model. Note that the learning dynamics are quite different between the HP model and RBM. However, as long as RBM uses Gaussian hidden variables, these models are equivalent.

In order to investigate how similar mfHP patterns and RBM patterns Figure 4.1: (a) An attractive pattern (top) and a repulsive pattern (bottom). Distributions of entries of attractive patterns are usually dense across the site and residue types. In contrast, the distributions of repulsive patterns are typically localized at specific residues and amino acids, which are generally associated with spatial interaction. (b) Likelihood contribution as a function of eigenvalues of the Pearson covariance matrix. Patterns with eigenvalues close to 1 have a small contribution to the log-likelihood, whereas those eigenvalues that are greater than / less than 1 make a significant contribution. Note that the largest eigenvalue gives the largest influence on the likelihood, but that corresponds with conservation terms, hence associating with the single site frequencies (see the following discussion and Fig. 4.2).c. are, we introduce here the overlap, i.e., inner products of these patterns,

ξ µ , v ν = (ξ µ ) t v ν ξ µ = v ν = 1 , (4.3)
where {ξ µ } are RBM patterns and {v ν } are mfHP patterns.

Fig. 4.2.a shows the overlaps between the RBM and mfHP attractive patterns | ξ µ , v ν | . In this test, we used P = 2 hidden variables for the RBM. For attractive mfHP patterns, µ = 1, 2, . . . , 400 patterns are selected from the top in descending order of eigenvalues, that is, in descending order of contribution to the log-likelihood.

The top three attractive mfHP patterns show significant overlap with RBM patterns, which is greater than the overlaps between the two different RBM patterns (red lines) (an estimated overlap using the same qL dimension 1000 Gaussian random vectors is around 0.0165). Notably, the mfHP patterns corresponding with the second and third largest eigenvalues show a strong correlation with each RBM pattern (we hereafter call a HP pattern corresponding with the n-th largest eigenvalue as n-th mfHP pattern). The first mfHP pattern does not show a strong correlation with any RBM patterns. Instead, we found a large overlap with the local field, which associates with single-site frequency Fig. 4.2.c . In Fig. 4.2.b, the overlaps between the repulsive patterns and RBM patterns are significantly smaller than the case of the attractive patterns and the overlaps between the two RBM patterns (red lines). These results clearly illustrate the absence of the repulsive modes in RBM (with Gaussian hidden variables) patterns.

We will also investigate the comparison between a mfHP model and RBM with P = 16 hidden variable case. Note that learning RBM is a nonconvex optimization problem, and the set of RBM patterns is not unique and depends on the initial conditions. Therefore, as the number of hidden variables increases, so does the number of solutions (the optimal set of patterns). Consequently, it is nontrivial how the similarity between RBM patterns and mfHP patterns can be changed as an increase in the number of patterns. Regarding this fact, we use mfHP patterns as the initial states of RBM patterns and investigated how well RBM patterns can preserve mfHP patterns after learning. As for learning the RBM, we selected only attractive mfHP patterns as initial conditions of RBM patterns. These mfHP patterns are ranked second to 17th in the likelihood contribution. The first mfHP pattern was excluded because of its similarity with the single-site frequency. The other learning conditions are same as in the case of the random initializations (cf. Fig. 4.3) Fig. 4.3 shows the overlaps between RBM patterns with P = 16 (each panel corresponds with one of these patterns) and attractive mfHP patterns. We cannot find RBM patterns that are particularly similar to one of the mfHP patterns. On the other hand, as Fig. 4.4 shows that when a RBM pattern was initialized by one of mfHP patterns, it is specifically similar to the one used as the initial condition even after the learning. That is, the set of mfHP patterns is close to one of the stable optimal solutions for RBM (using Gaussian hidden variables).

Interestingly the overlap values are small (< 0.2) even if mfHP patterns are corresponding the top ten largest eigenvalues, except those that were used as the initialization of the learning. Therefore, these RBM patterns after training show strong specificity for the initialized mfHP pattern. This tendency becomes more pronounced as the contribution to log-likelihood increases (the left on the top panel shows a greater maximum overlap than the right on the bottom panel). It indicates that one of the optimal RBM pattern sets (there are multiple optimal solutions) is close to the mfHP pattern set.

Moreover, although the RBM patterns show specificity to the initialized mfHP patterns, they differ to some extent, and their overlap values are not very close to 1. Therefore, the difference between a mfHP pattern and the RBM pattern that uses it as the initial condition makes a difference whether it is a generative model or not.

In conclusion, using mfHP patterns as initial states of the RBM pattern, it is possible to effectively "label" each RBM pattern (hidden variable) and encourage the patterns to become independent. In addition, it is also possible to accelerate the learning time by using mfHP pattern as the initial condition of RBM.

Applications

Mean-field Hopfield-Potts pattern for a generative model

The HP Energy function in Eq. 4.2 is a rotational invariant with respect to HP patterns. That is, the energy function does not change under the following transformation,

ξ µ i (a) → ν O µν ξ ν i (a) . (4.4)
It is called a gauge invariance, and such a degree of freedom is referred to as the gauge freedom in physics (see also Sec. 2.4.2). Although RBM patterns and mfHP patterns show significant overlap, these gauge choices are different. Thus those inner product depends on the gauge choice.

In this section, we will compare RBM patterns and mfHP patterns in terms of gauge-invariant quantities. Specifically, we will investigate statistics that are generated from RBM using mfHP patterns.

To build a generative model using RBM with mfHP patterns, we choose as free parameters the number of pattern P and the scale parameters of the mfHP pattern, a parameter that multiplies the mfHP pattern in the energy function of RBMs, v µ → v µ /T * =: ξ µ , where v µ is a mfHP pattern and ξ µ is a RBM pattern. As we saw in the last section, repulsive mfHP patterns show very low similarity with RBM patterns, hence we consider only attractive patterns.

Finding the optimal scale parameters T * can be useful for learning RBMs efficiently and well-controlled manner:

1. The first advantage is the acceleration of the learning processes, by choosing patterns that are close to the final configuration of the pattern reduces the computational cost.

2. In addition, as we mentioned in the last Sec. 4.3.1, the RBM learning is a non-convex problem and usually depends on the initial conditions of the model parameters. Therefore by introducing the mfHP patterns as the initial condition, we can label for each hidden variable (using the likelihood-contribution of the mfHP patterns).

As we understand from Sec. 4.3.1, the pattern with the largest contribution to the likelihood function tends to associate with the single-site frequency. Therefore, we select patterns from the second to the (P + 1) th patterns. In this experiment we used P = 2 and P = 16. Fig. 4.5.a shows the values of Pearson coefficients for two-point connected correlations as a function of 1/T . The correlations are computed from an ensemble of sequences generated from the RBM with P = 2 mfHP attractive patterns using a scale parameter 1/T , and an MSA. We selected protein families and the MSA that we used in our article [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF]. The maximum Pearson coefficients using the attractive mfHP patterns are 0.3 -0.7, which are always smaller than the Pearson coefficients using the RBM patterns (> 0.6) as expected. ). Surprisingly, the Pearson coefficients are much smaller values than the cases of P = 2. One of the possible reasons is that the optimal scale parameter might be different for each pattern. It may depend on the eigenvalues. According to the likelihood contributions Fig. 4.1.b, the second and third eigenvalues are close. Therefore, scaling with a single parameter can achieve relatively good statistics when P = 2. However, the range of the eigenvalues up to the top 17th change substantially (λ µ = 0.3 -12). Therefore it may need to introduce scale parameters for each pattern when P = 16.

Low-dimensional analysis using P = 2 model

In the previous section, we saw that RBM patterns show substantial similarity with attractive mfHP that are significantly contributing to the likelihood. Notably, RBM patterns of P = 2 show strong specificity for a few attractive mfHP patterns (Fig. 4.2), whereas RBM patterns of P = 16 show more broad similarities among many attractive mfHP patterns (Fig. 4.3). Therefore, the RBM patterns with P = 2 would be also similar to the largest eigenmodes of the covariance matrix (hereafter we call as "principal modes"), which have a clear relation with the attractive mfHP patterns (Eq. C.5).

Moreover, RBM with P = 2 patterns can still reproduce statistical properties; the values of Pearson correlation are typically greater than 0.6. It can be even 0.8 in some cases (see the red lines in Fig. 4.5). Therefore, it is a generative model that shows the similarity to the principle modes of the Another advantage of using a smaller number of patterns is that they can be used for low-dimensional analysis by projecting sequences to the pattern space. Since RBM can be used as a generative model, projected variables in the low-dimensional space corresponding to the hidden variables are thought to capture the important features that characterize the training data.

Note that these hidden variable activities in the low-dimensional spaces can be understood in terms of the statistical modelings, which making it significantly different from commonly used low-dimensional analysis. For example, the marginalized HP distribution in Eq. 4.2 can be written as a function of the mean of hidden variables,

x µ (A) = 1 L i ξ µ i (A i ) . That is, p(A) ∝ exp L 2 P µ (x µ (A)) 2 + const. .
It shows the probability distribution of a sequence A for each pattern µ ∈ 1:P , and the larger ( i ξ µ i (A i )) 2 , the greater the dependence on this pattern µ.

We show here the low-dimensional spaces of protein sequences. As the low-dimensional spaces, we used following patterns:

(a) Principal component vectors of the data covariance matrix (we will explain the data set bellow).

(b) P = 2 RBM patterns used random initialization for learning. These patterns are obtained using RBM's standard learning method PCD, and the training data is the same as in (a).

(c) mfHP attractive patterns, correspond with the second and third most contributing patterns in the likelihood function (cf. Eq. 2.34). The dataset used is the same as in (a).

(d) P = 2 RBM patterns used the mfHP patterns used in (c) for the initial condition of learning. Other conditions are the same as in (a).

We used the data set of the response regulator domain family (Pfam ID, PF00072) for this experiment. More precisely, within protein sequences of the PF00072 domain, we constructed subgroups of sequences according to other specific protein domains adjacent to the PF00072 domain. For example, sequences belong to the other protein subfamily adjacent to a protein domain A, but sequences belong to the other protein subfamily adjacent to a different protein domain B (see a schematic of adjacent protein domains, Fig. 4.6). In each protein subfamily, slight changes can occur in the three-dimensional structure and function according to other adjacent protein domains. Consequently, the difference of sequences between different protein subfamilies is pronounced. In Table 1, we summarized the considering protein subfamilies of PF00072 protein domain. Fig. 4.7.a shows projections of PF00072 protein sequences using the lowdimensional space mentioned in (a), where the projection means mapping to a sequence to two-dimensional space such that {1:q} L → R 2 . The color of each point corresponds to each protein subfamily (the mapping between the colors and protein subfamilies is summarized in Table 1).

Similarly, Fig. 2, Fig. 3, and Fig. 4 correspond to the projections of the protein sequences to the low-dimensional spaces (b) the RBM patterns with the random initialization, (c) attractive mfHP patterns, and (d) the RBM patterns with the mfHP patterns initialization mentioned earlier, respectively. For example, variables in the two-dimensional space projected using the RBM pattern can be written as:

i ξ µ i (A i ) , µ ∈ {1, 2}.
For protein subfamilies named classes PF00072-PF00512, PF00072-PF00512 (after 500 aa), and PF00072-PF00512 (after 1000 aa) (purple, red, and orange colors are assigned, respectively). These protein families can be very similar in nature, as the only difference is how far the domains PF00072 and PF00512 are from each other on the protein sequence. In fact, they are projected on similar areas, and non of the low-dimensional space can Table 1: Summary of protein subfamilies for PF00072. The left column shows Pfam IDs of the protein subfamily (discussed below). The center and the right columns explain the colors used in Fig. 4.7 and the number of sequences included in the protein subfamily data, respectively. The protein subfamily of each PF00072 is adjacent to the protein domain shown in the leftmost column of the table. PF00512 (after 500 aa) means that there are 500 amino acids between the PF00072 domain and PF00512. Similarly, after 1000 aa, means there are 1000 amino acids between them. No pair means that the PF00072 domain does not adjacent to any protein domain. The middle column shows the color types used in Fig. 4.7. M states the number of protein sequences in each ensemble of the protein subfamilies. Sequences of protein domains that are not adjacent to any protein domain (yellow color is assigned) are projected over a large area. This tendency can be observed in all low-rank spaces. Probably because there is no adjacent protein family, the sequences belonging to this protein family are more likely to be changed by mutational drift rather than specific mutations, and the sequence variability among this protein family is high.

As explained previously in Sec. 4.3.1, RBM patterns with a small P shows significant similarity to attractive mfHP patterns. Therefore, the projection using the RBM patterns (initially random patterns) Fig. 4.7.b and the projection using the mfHP patterns Fig. 4.7.c are similar. However, the shape of the projected distribution is not exactly the same direction in Fig. 4.7.b and Fig. 4.7.c , but it is projected in rotation (Fig. 4.7.b is the shape of "+", but Fig. 4.7.c is rather the shape of "x" ). We presume that this is the direct result of the rotational invariance between the patterns described in Sec. 4.3.1 (Eq. 4.4). Note that (d) is "gauge-fixed" by using the attractive mfHP patterns as the initial state of RBM patterns, and shows the same shape of distribution as (c) (both are "x" shapes).

In the case of the low-dimensional spaces except for the PCA space, the protein subfamilies, including large populations (more than 3,000 sequences), are mostly projected into regions that differ from other protein subfamilies except for the protein families we mentioned before. In the PCA projections, the subfamilies classes PF00072-PF00158 (Green) and PF00072-PF04397 (Gray) overlap with other prominent subfamily areas and are indistinguishable.

Interestingly, the low-dimensional spaces based on RBM patterns (both (b) and (c)) tend to project these subfamilies PF00072-PF00158 and PF00072-PF04397 to regions that are different from other subfamilies. Such differences may characterize the differences between generative and non-generative models.

Criticality of RBMs

This section shows that the number of hidden variables P required to reproduce the statistical properties of protein sequences is much smaller than the number qL of the pairwise Potts model. We may think that the reduction of model parameters helps the low-rank model evade criticality, i.e., the sensitivity of the statistics against slight changes in model parameters (see Chap.3 and Ref. [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF]).

We exploit the heat capacity, which is equivalent to a variance of an energy function, to assess the robustness of the statistics against the global change of the model parameters. Similar argument can be found in our study [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF].

For the HP model, the heat capacity function C HP (T ) is defined as:

C HP (T ) = ∂ E HP T ∂T = 1 T 2 (E HP ) 2 T -E HP 2 T , E HP (A) = - i<j J HP ij (A i , A j ) - L i=1 h HP i (A i ) , J HP ij (a, b) = 1 L P µ=1 ξ µ i (a)ξ µ j (b) , h HP i (a) = h i (a) + 1 L P µ=1 (ξ µ i (a)) 2 , (4.5)
where the average in Eq. 4.5, denoted as • T is evaluated as a sample average from a RBM with Gaussian hidden variables. Therefore, although we use the energy function of the HP model, the effect on the heat capacity is purely due to the statistical properties of the RBM. In order to be equivalent to a HP model p HP (A|T ) ∝ exp(-E HP (A/T )), we scaled the energy function of RBM by h i (a) → h i (a)/T and ξ µ i (a) → ξ µ i (a)/ √ T 1 Fig. 4.8.a shows the heat capacity for different numbers of hidden variables P in the RBMs. As the data set, we used the RNA recognition motif domain (Pfam ID PF00076). The figure clearly demonstrates that as we increase the number of hidden variables, the peaks of the heat capacity become rapidly steeper. As P increases further (P > 16), it becomes difficult to 1 Introducing the temperature parameter to the RBM energy function is not unique, we can obtain equivalent result using different scaling transformation:

-E RBM (A, x) = i,µ x µ ξ µ i (Ai) + 1 T i hi(Ai) -T µ L 2 (x µ ) 2 ,
after we integrating out the hidden variables we get the same result as exp(-E HP (A/T )) . perform sampling from the equilibrium distribution in the low-temperature regime (T < 0.9). This result indicates that the influence is not because of architecture but rather the effects of the underlying features of the training data. Fig. 4.8.b shows the heat capacity using random patterns ξ rand . These random patterns are generated from Gaussian distribution with the same mean and variance values as the learned RBM patterns. Formally, we can represent ξ rand

N (µ rand , σ rand ) ∼ ξ µ, rand i (a) ∈ R 1 , ∀i, a, µ, µ rand = Mean(ξ 1:P ) ∈ R 1 , (σ rand ) 2 = Var(ξ 1:P ) ∈ R 1 , (4.6)
where µ rand and (σ rand ) 2 are a mean and variance of the Gaussian distribution (normal distribution), and these values are estimated by the RBM patterns ξ 1:P using the functions Mean(•) which returns arithmetic average of the arguments •, similarity Var(•) is a function that returns a variance. Both Pearson value and the slope increase steadily as P increases. In the case of P = 16, it already achieved the Pearson values 0.8 (it improves further as increase the P ).

We found that the temperature that achieves the highest Pearson value is below T < 1.0 (between 0.9 < T < 1.0). Considering the argument of the heat capacity, this result, therefore, indicates the presence of optimal P and T in the sense that it achieves reasonably good statistics but is less critical (low-temperature is feasible).

Conclusion

This Chapter investigated low-rank models, also named Hopfield-Potts (HP) models, for proteins sequence generative models. The HP model can be converted to a restricted Boltzmann machine (RBM) with Gaussian hidden variables using the Hubbard-Stratonovich transformation, both of which are equivalent statistical models. Sec. 4.2 (cf., [START_REF] Shimagaki | Selection of sequence motifs and generative hopfield-potts models for protein families[END_REF]) showed that RBMs with a number of hidden variables (∼ 40), which is sufficiently smaller than qL can reproduce two-point cor-relations of protein sequences (Pearson values of two-point correlations are ∼ 0.9). Note that two-point correlations, unlike in the case of bmDCA, are quantities that are not guaranteed to be reproduced by the model.

Moreover, hidden variables of RBM show specificities for each protein subfamily. Therefore, protein subfamilies can be classified by the activities of hidden variables, which are equivalent to projections of protein sequences using RBM patterns. Sec. 4.3.1 compared RBM patterns, which are equivalent to HP patterns and mean-field HP (mfHP) patterns. RBM (with Gaussian hidden variables) patterns show similarity with attractive mfHP patterns that especially contribute to the log-likelihood (cf. Fig. 4.2).

RBM contains a set of patterns similar to mfHP patterns as the optimal solutions. When mfHP patterns are used as the initial states of the RBM, they show notable similarities with the mfHP patterns used in each initial state after learning (cf. Fig. 4.3 and 4.4). Sec. 4.3.2 shows the application of observations in Sec. 4.3.1. First, we investigated how accurately the statistics could be reproduced when the mfHP patterns were used as the RBM patterns.

In the case of P = 2, some RBMs that use mfHP patterns can generate statistics as well as standard RBMs by scaling the mfHP patterns with temperature parameters (cf. Fig. 4.5.a. in Sec. Astonishingly, it achieves Pearson values of two-point correlations that are bout 0.8 in the case of PF13345). As we discussed in Sec. 4.3.1, mfHP patterns are close to the ones of the RBM's solutions. Therefore the case of P=16 RBM with mfHP patterns could also generate statistics well if these patterns were correctly scaled (cf. Fig. 4.5.b). To understand appropriate scaling for each pattern, that is to understand the dependency of eigenvalues on the optimal scaling is the objective of a future work.

As another application, we performed low-dimensional analyses of protein subfamilies using these RBM and mfHP patterns. As a further comparison, we performed the same analysis based on RBM patterns that are initialized by mfHP patterns (cf. Fig. 4.7).

We constructed the intermediate space (d) between the RBM pattern space (b) and the mfHP pattern space (c) by using the RBM patterns that are used mfHP patterns as the initial states (these labels (b), (c) and (d) correspond to the labels in Fig. 4.7).

In Fig. 4.7, the comparison between (c) and (d) demonstrates the effect of the mean-field approximation. Some protein sequences of protein subfamilies (e.g. Class PF00072-PF00158 and PF00072-PF04397) degenerate in the mfHP pattern space. In other words, the activities of hidden variables must be specific for each of the different major subgroups in order to be a generative model. Similarly, the comparison between (b) and (d) shows the effect of the rotational invariance of the HP patterns.

In Sec. 4.4, we discuss the criticality of RBMs using heat capacities [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF]. Increasing the number of hidden variables P has a more pronounced effect on the statistics of RBMs by perturbing the model parameters. It is interesting to understand how there is a difference between how the heat capacity changes with respect to change in P in RBMs and how the heat capacity changes with respect to the change in the coupling densities d in sparse BM (cf., Sec. 3.2 and [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF]).

There are many questions we couldn't address in our studies. Some of them are written as follows.

As we saw in [START_REF] Shimagaki | Selection of sequence motifs and generative hopfield-potts models for protein families[END_REF], large P improves residue contact accuracy, and also the repulsive patterns tend to associate with residue contacts [START_REF] Cocco | High-dimensional inference with the generalized hopfield model: Principal component analysis and corrections[END_REF][START_REF] Cocco | From principal component to direct coupling analysis of coevolution in proteins: Loweigenvalue modes are needed for structure prediction[END_REF]. A question considering these facts is: how increasing P influences the presence of repulsive patterns? Do the increase in the number of P induce repulsive patterns?

Other related questions concerning the emergence of repulsive patterns and the independence2 of patterns are: How does the selection of the hidden variable prior distributions affect the patterns? How does the choice of regularization affects the patterns?

Chapter 5

Models combining sparse and low-rank couplings

In the case of sparse bmDCA, we successfully reduced the number of couplings. In general, contact prediction performance remains stable even below coupling densities d ≤ 5% (cf., Fig. 2 in Chap. 3.2 ). Therefore, the remaining strong couplings after the decimation are mostly couplings associated with contact pairs. However, statistics from sparse bmDCAs are significantly degraded when we go below coupling densities d ≤ 10% (cf., Fig. 1 in Chap. 3.2 ).

On the other hand, low-rank or Hopfield-Potts (HP) models are excellent for generating statistics using a very limited number of parameters. Moreover, the HP patterns can distinguish protein subfamilies, especially those of high likelihood contributions. However, the accuracy of contact prediction of the low-rank model is not as good as plmDCA or bmDCA. The main reason is the absence of the repulsion pattern in the models [START_REF] Cocco | High-dimensional inference with the generalized hopfield model: Principal component analysis and corrections[END_REF][START_REF] Cocco | Inference of hopfield-potts patterns from covariation in protein families: calculation and statistical error bars[END_REF]. Therefore, importantly, both sparse BM and low-rank models learn different features from the same data. This chapter explores the possibilities of statistical models that combine both sparse couplings and low-rank models (Fig. 5.1). In particular, the chapter is organized as follows:

• Section 5.1 introduces some backgrounds and motivations of this study:

Residue-residue contact predictions when phylogenetic correlations are present in training data. Interactions between these variables are represented as edges, where black edges are interactions defined in RBM and red edges are interactions of the sparse bmDCA.

• Section 5.2 shows how to combine sparse couplings and low-rank couplings. We propose methods for selecting sparse couplings that are complementary parameters to low-rank couplings. Particularly, we discuss element-wise couplings (Sec. 5.2.1) and block-wise couplings (Sec. 5.2.2) individually.

• Section 5.3 shows some results of the proposed method. In Sec. 5.3.1, we discuss the residue-residue contact predictions.

Motivation

It was reported [44] that the eigenvalue distributions of the covariance matrices of biological sequences show power-law distributions. This phenomenon is due to the effects of phylogenies, which are included in protein sequence ensemble typically. Ref. [44] shows that removing the largest eigenmodes from the covariance matrices can reduce phylogenetic effects and improve residue-residue contact prediction accuracy. (Fig. 5.2 shows the eigenvalue distribution and the contact prediction's accuracy). Here we emphasize that while RBM is a generative model of protein sequences, as explained in the previous section (see Sec. 4.3.1), the RBM patterns are strongly associated with the eigenmodes of the covariance matrix. Especially when RBMs depend on a small number of hidden variables such as P = 2, this relation becomes more pronounced. On the other hand, the RBM with Gaussian hidden variables cannot treat the repulsive patterns that correspond with residue-residue contacts [START_REF] Cocco | High-dimensional inference with the generalized hopfield model: Principal component analysis and corrections[END_REF][START_REF] Cocco | From principal component to direct coupling analysis of coevolution in proteins: Loweigenvalue modes are needed for structure prediction[END_REF].

Ref. [44] also provides a simple yet very suggestive insight to the question "Why does DCA work well with non-i.i.d. samples?". Here we sketch this idea demonstrated in this reference article. Suppose the covariance matrix C can be decomposed into eigenmodes,

C = λ 1 v 1 v t 1 + • • • + λ P v P v t P .
Where, v µ and λ µ are a non-zero-mode eigenvector and eigenvalue, respectively. Here, P is the rank of the covariance matrix. In the case of mfDCA, the coupling matrix is nothing but its inverse. Hence, it can be represented as

-J = C -1 = λ -1 1 v 1 v t 1 + • • • + λ -1 P v P v t P
, therefore the coupling matrices effectively down-weights large eigenvalue modes. Combining this simple argument and the main claim in the reference article, that is the that large eigenvalues are the consequences of phylogeny, we can understand why DCA can accurately predict residue contacts even if protein sequence data is noni.i.d.

Although the largest eigenmodes of the covariance matrix may show signs of phylogenies, still, we cannot bring clear answers to the following questions:

• How to remove more effectively the large eigenmodes within the framework of statistical modelings?

• How to distinguish phylogenetically or structurally induced correlations?

Here we propose a tentative answer to these questions from a generative modeling framework.

Coupling activation

As we noted, RBMs tend to learn the patterns that correspond to the largest eigenvalue modes (hereafter, we referred to them as "principal modes") of the covariance matrix. Moreover, such principal modes presumably associate with phylogenetic effects [44]. By considering these factors, the following hypothesis can be stand: Distributions obtained by subtracting the RBMs probability distributions from an empirical distribution suppresses The precision of the residue contact predictions as a function of the number of removing the principle modes of data covariance matrix. These eigenmodes are sorted in descending order of the eigenvalues. It shows that removed some largest principle modes improves contact predictions. These figures are adapted from [44].

the influence of the principal modes, and can enhance correlations caused by spatial interactions.

To compare an empirical distribution and the RBM distribution more quantitatively, KL divergence between them is a reasonable measure. It can be formally written as

D KL (p * p RBM ) = A∈A L p * (A) log p * (A) -log p RBM (A) , (5.1)
where p * is assumed that to be an empirical distribution, and it can be represented as a pairwise Potts distribution with true model parameters {J * ij (a, b)}, and p RBM is a probability distribution depending on a small number of hidden variables (P = O(1)). Here, we assumed the empirical distribution could be fully described by a pairwise Potts distribution with the set of coupling parameters. If p RBM can adequately learn the correlations related to the principle modes, D KL (p * p RBM ) in Eq. 5.1 would be a function of the couplings {J * ij (a, b)} that are more likely associated with spatial contacts.

Although Eq. 5.1 is intractable, we may directly assess how KL divergence (KLD) in Eq. 5.1 changes by adding an individual coupling to an RBM distribution. Here we slightly generalize the problem so that the assumed model is not only RBM but also an arbitrary statistical model p(A). The difference in the KLD can be written as

∆l = D KL (f p) -D KL (f p ) = A f (A) log p (A) -log p(A) , (5.2)
where f (A) is an empirical distribution, and p (A) is a model that is added an individual coupling parameter (e.g., J ij (a, b)). Hereafter, we refer to ∆l in Eq. 5.2 as likelihood variation. As we will discuss in the following section, ∆l can be obtained analytically as a function of the empirical and the model frequencies (e.g., f ij (a, b) and p ij (a, b)).

Note that a larger ∆l value means that the added parameter can be more influential to the model p(A) . Therefore, it is possible to select sparse parameters to add to the assumed model based on ∆l.

In the next section, we will formulate the ideas presented here in a more concrete way.

Element-wise coupling activation

The key idea is to estimate relevant couplings based on a likelihood variation Eq. 5.2. In order to formulate it, we first define a modified Hamiltonian,

H (A) = H(A) -J ij (a, b)δ A i ,a δ A j ,b .
(5.

3)

The old Hamiltonian H(A) is an arbitrary function of protein sequences. If H(A) is not a pairwise Potts energy function (e.g., a profile model or RBM), the operation in Eq. 5.3 means adding a single coupling parameter

J ij (a, b)δ A i ,a δ A j ,b
, where i and a indicate the residue position being i ∈ 1:L and amino acid being a ∈ 1:q, respectively. On the other hand, if H(A) is a pairwise-Potts energy function, Eq. 5.3 means changing the existing coupling.

Using these H(A) and H (A), we can estimate the likelihood variation as following,

∆l = 1 M M m=1 log p (A m ) -log p(A m ) = 1 M M m=1 H (A m ) -H(A m ) -log Z Z = J ij (a, b)f ij (a, b) -log Z Z , (5.4)
where p(A) and p (A) are Gibbs-Boltzmann distributions characterized by the Hamiltonians, e.g., p(A) = exp(-H(A))/Z. Similarly, Z and Z are the partition functions of the Boltzmann measure, e.g., Z = A exp(-H(A)).

The two-point frequency f ij (a, b) is defined as a partial marginalization of the empirical frequency,

f ij (a, b) = A∈{1:q} L f (A)δ A i ,a δ A j ,b .
Note that the ratio of the partition function is tractable and can be easily computed:

Z Z = 1 Z A e -H(A)+J ij (a,b)δ A i ,a δ A j ,b = e
where p ij (a, b) is the marginal two-point probability distribution for the assumed model p(A). By plugging the analytical formula of Z /Z, we get the exact formula of the likelihood variation,

∆l = J ij (a, b)f ij (a, b) -log e J ij (a,b) p ij (a, b) + 1 -p ij (a, b) . (5.5)
Note that ∆l is derived without knowing the function H(A). The dependency on H(A) comes via the probability p ij (a, b) which we can estimate using MC sampling. All we have assumed is that the probability distribution is only Gibbs-Boltzmann, p(A) ∝ exp(-H(A)).

Optimize of the element-wise likelihood variation -Here, we discuss more basic properties of the likelihood variation as a function of the coupling J ij (a, b). For the sake of simplicity, we neglect the indices (ijab): ∆l(J) = Jflog(e J p + 1p) .

(

Eq. 5.6 has a simple shape,

∆l(J) =      -J(1 -f ) if J → +∞ J(f -p) if J ∼ 0 -|J|f if J → -∞ . ( 5.7)
This function goes to minus infinity as |J| → ∞ in a linear manner and approaches zero as J → 0 also linearly, hence this function has a strictly positive maximum:

J * = log f (1 -p) p(1 -f ) , (5.8)
with,

∆l * = f log f p + (1 -f ) log 1 -f 1 -p ≥ 0 .
(5.9)

We refer to J * as activating couplings, which has also been suggested in [START_REF] Franz | A fast and accurate algorithm for inferring sparse ising models via parameters activation to maximize the pseudo-likelihood[END_REF], but we can calculate the optimal value instead of a perturbative calculation.

Note that J * goes to infinity if p or f is close to zero or one, however that situation can easily be avoided by introducing a pseudo-count for frequencies1 . We also show another derivation of ∆l(J) by evaluating the likelihood of gave almost the same results. In fact, the coupling activation values for each element-wise and block-wise were very similar. By the definition of the coupling activations in Eq. 5.8 and Eq. 5.15, it can be seen that when the two-point empirical distribution p ij (a, b) is close to one or zero (that is, it corresponds to a situation sites where sites i and j are likely to be contact), both values are close to each other. Interestingly, adding a small number of hidden variables (P = 4) could improve the residue contact predictions, and adding more hidden variables (P > 4) did not show any significant improvement in accuracy. The improvement of the residue contact predictions is thought to be because RBMs have specifically learned the pairwise correlation due to the phylogenetic effects of protein sequences. Furthermore, they did not learn the correlation caused by the three-dimensional structure so strongly (Sec. 4.3.1). In Appendix F.2, we show the same analyses for other protein families. Phylogenetic effects are highly dependent on the protein family, and the efficiencies of residue contact predictions based on the likelihood variation also tend to be dependent on the protein families.

Conclusion

In this chapter, we introduced statistical models that concern both a small number of strong pairwise couplings in sparse BMs and the low-rank couplings in RBMs. These two types of parameters are complementary: the sparse couplings are typically associated with pairs of residues in spatial contact. On the other hand, low-rank couplings tend to learn features of each subfamily and phylogenies. Sec. 5.1 reviewed Ref. [44], which is about phylogenetic effects on the principal modes of data covariance matrices. Principle modes of covariance matrices tend to carry phylogenetic effects and subfamily specific features, and those eigenvalues are typically following a power-law distribution. Therefore removing the principal modes from the covariance matrices can improve residue-contact predictions (cf., Fig. 5.2). Sec. 5.2 introduced methods that select variables that are presumably significant variables to describe the training data. These methods allow us to select variables to add to an assumed statistical model by assessing the impact of virtually introducing an individual model parameter (e.g., pairwise coupling parameters J ij (a, b), higher-order interaction parameters J ijk (a, b, c), ..., etc.) into the likelihood function of the assumed model. Note that this is an opposite principle to the sparse BM, as we add the couplings instead of decimating them. More specifically, we investigated in detail how to add element-wise and block-wise coupling parameters that increase the likelihood. We can also find another approach to add coupling to a pairwise model based on pseudo-likelihood. We can also find another interesting research to add coupling parameters iteratively to a pairwise model based on pseudo-likelihood [START_REF] Franz | A fast and accurate algorithm for inferring sparse ising models via parameters activation to maximize the pseudo-likelihood[END_REF].

From the results of Ref. [44] and Sec. 4.3.1, it is expected that RBMs can efficiently learn the correlations that affect phylogenies. Therefore, assuming RBMs as statistical models to which parameters are added, the coupling parameters that increase the likelihood function correspond to pairs in spatial contact.

In Sec. 5.3, we predicted residue-residue contact based on the proposed method using RBMs. Assuming a model with a small number of hidden variables (P > 2), residue contact prediction was significantly improved. Further studies are needed to apply these methods to different protein families depending on the strength of the phylogenetic effects.

Nature uses only the longest threads to weave her patterns, so that each small piece of her fabric reveals the organization of the entire tapestry.

-Richard Feynman (1918 -1988) III Higher-order statistical modeling Chapter 6

Variational Autoencoders for Protein Sequences

Up to this chapter, we have discussed parameter decimation and dimensionality reduction. However, the objective of this chapter is to investigate other possibilities of variables that explain training data more comprehensively than pairwise interactions in stochastic modeling for protein sequences. Therefore, we will explore the possibility of other variables besides pairwise interactions.

Particularly, we aim at investigating the presence of higher-order interactions in natural protein sequences. From now on, the higher-order interactions are defined as interactions between amino acids that include three or more sites in underlying probability distributions for protein sequences. Especially, three-body interactions are considered in Sec. 6.1. Note that we need to distinguish interactions and correlations.

On the other hand, there has been extensive research on protein sequence generative models and protein structure predictions using the state of the art of deep neural networks over the last few years (DNNs) [START_REF] Andrew W Senior | Improved protein structure prediction using potentials from deep learning[END_REF][START_REF] Tsuchiya | Neural networks for protein structure and function prediction and dynamic analysis[END_REF][START_REF] Burak Yelmen | Creating artificial human genomes using generative neural networks[END_REF].

Notably, DNN-based generative models do not require to decide which variables should be considered in a priori and can take into account higherorder interactions. Variational Auto-Encoder (VAE) is a generative statistical model that can handle DNNs well within the framework of statistical modeling. VAEs have been applied as a protein sequence generative model in recent years [START_REF] Adam J Riesselman | Deep generative models of genetic variation capture the effects of mutations[END_REF][START_REF] Hawkins-Hooker | Generating functional protein variants with variational autoencoders[END_REF], and further applications are expected.

The structure of this section is as follows: First, we show a relatively straightforward experiment that studies causes of strong third-order correlations. Second, we investigate the presence of higher-order and non-linear effects by exploiting VAEs.

Decomposition of strong three-point correlations in two-point correlations

It has been said that there are higher-order interactions in protein-sequence ensembles, and taking them into account is essential for improving properties as a generative model [START_REF] Schmidt | Three-body interactions improve contact prediction within direct-coupling analysis[END_REF][START_REF] Schmidt | hodca: higher order directcoupling analysis[END_REF][START_REF] Adam J Riesselman | Deep generative models of genetic variation capture the effects of mutations[END_REF]. Besides the computational challenge to include higher-order interactions explicitly in a model, there are also statistical challenges. That is, searching the evidence of the higher-order interaction is like hearing a faint sound of insects at a construction site. Most of the signals from relevant higher-order combinations are still much smaller than the majority of lower-order statistics (single site frequencies and pairwise frequencies).

Instead of directly investigating higher-order interactions, we first explore the nature of higher-order correlations. More specifically, we search three-point correlations not explainable by two-point correlations 1 . Therefore, we focus on strong three-point correlations, especially when the twopoint correlations involved in the three-point correlation are not very strong. Such three-point correlations can be defined as follows:

|C ξηζ | ≥ θ * 3 and, |C ξη |, |C ηζ |, |C ζξ | ≤ θ * 2 , (6.2)
1 More formally, three-point correlations that are explainable by two-point correlations satisfy the following properties:

f ξηζ = f ξη f ζ . (6.1)
We can easily check that the three-point correlation can be written in terms of means and two-point correlations as follows:

C ξηζ = f ξηζ -(f ξ f ηζ + fηf ζξ + f ζ f ξη ) + 2f ξ fηf ζ = +2f ξ fηf ζ -(f ξ f ηζ + fηf ζξ ) = -f ξ C ηζ -fηC ζξ .
Therefore, three-point correlations that satisfies Eq. 6.1 can be written with the two-point correlations and means. Thus, a strong three-point correlation that involves two or more weak two-point correlations must be a different form.

where each index in (ξ, η, ζ) is supposed to specify a position and an amino acid, and C ξηζ is the three-point (connected) correlation defined in Eq. 2.12. The θ * 3 , θ * 2 are preselected thresholds for third-and second-order correlations that distinguish significance.

The question raised here is whether there are strong three-point correlations, even if the associated tow-point correlations are small. Concerning this question, we compare the following two types of three-point correlation ensembles:

• ensemble-1: an ensemble for all of the possible three-point correlations.

• ensemble-2: an ensemble of the three-point correlations, where two or more of the three two-point correlations are large (e.g., if

|C ξη |, |C ηζ | > θ *
2 , then we include C ξηζ in to ensemble-2).

For both ensembles, we treat only three-point correlations where indices are pairwise different (ξ = η, η = ζ, ζ = ξ). If there are three-point correlations not explainable by the involved two-point correlations, we cannot find them in ensemble-2.

The two-point correlations that are induced by the background noise can be estimated numerically by taking sequences from a profile model. In fact, the two-point correlations due to the background noise are quite small, 0.0057, 0.0046, and 0.0049, for protein families, PF00014, PF00072, and PF00076, respectively. Hence, we selected two-point correlations whose values are greater than θ * 2 = 0.01 as significant two-point correlations, which correspond to less than 0.3% of all two-point correlations. Based on the selected strong two-point correlations, we determined a set of pairs of indices and states Ω 2 := {(ijab)}, then construct all of the possible three-point correlations Ω 3 := {(ijkabc)}, which should include at least two out of these three pairs (ijab), (jkbc) or (kica) ∈ Ω 2 .

In Fig. 6.1, we show the distributions of the three-point correlations for the three protein families (the MSAs are those in Ref. [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF]). For comparisons, we selected the largest 10 5 correlations as particularly strong threepoint correlations (the red histogram, which corresponds to ensemble-1), which is typically less than 0.05% of the total, and three-point correlations whose associated two-point correlations are significantly strong (the blue and green histograms, which correspond with ensemble-2 with different threshold θ * 2 = 0.01 and θ * 2 = 0.015 respectively). It clearly suggests that the strong three-point correlations occur as a consequence of strong two-point correlations.

6.2 Variational Autoencoders as generative models and non-linear low-dimensional analysis for proteinn sequences

Variational Autoencoders (VAEs) are alternative generative models getting recently considerable attention. VAEs were proposed by Kingma and Welling in 2013 [START_REF] Diederik | Auto-encoding variational bayes[END_REF]. VAE can be used as a generative model, but also as low-dimensional analysis such as PCA, but in a non-linear manner. Because of the usefulness and flexibility of VAE, they have been applied for designing protein sequences and extract a lot of biologically useful information from sequence alignments [START_REF] Adam J Riesselman | Deep generative models of genetic variation capture the effects of mutations[END_REF][START_REF] Sinai | Variational auto-encoding of protein sequences[END_REF][START_REF] Davidsen | Deep generative models for t cell receptor protein sequences[END_REF]. The application of VAEs to a protein sequence generation model has attracted great attention [START_REF] Adam J Riesselman | Deep generative models of genetic variation capture the effects of mutations[END_REF][START_REF] Hawkins-Hooker | Generating functional protein variants with variational autoencoders[END_REF].

It has been known that liner systems such as PCA and linear-autoassociate model are not appropriate for certain types of distributions such as multimodal and non-linear distributions, whereas non-linear systems can classify such distributions correctly [START_REF] Japkowicz | Nonlinear autoassociation is not equivalent to pca[END_REF]. Formally, both methods can be understood as reconstruction spaces, one being a linear projection, the other a non-linear projection. In fact, the difference between the two methods becomes clear in terms of reconstruction error. For instance, if there is a nonlinear relation in a given data set (suppose the variables are continuous), a linear transformation cannot reconstruct original data [START_REF] Japkowicz | Nonlinear autoassociation is not equivalent to pca[END_REF]. Note that we can find similar situations when dealing with categorical variables, but the non-linear effect in the above example corresponds to multilinear effects, i.e., higher-order interactions. In the case of protein-sequence modeling, therefore, autoencoder-based lower-dimensional analysis might be useful if the distributions are multi-modal or have multilinear effects, not captured by pairwise interactions.

As we mentioned earlier, a VAE is a generative model. Moreover, it can provide a space where the information of raw data (e.g., protein sequences) is compressed or encoded. This space is called a hidden space or feature space. The mapping of a raw data x to hidden variables z is called encoder, q φ (z|x), the and reconstruction of the raw data from the encoded hidden variables is called a decoder, p θ (x|z), cf. Fig. 6.2.a (we show more details in Sec. 6.3.1). If the dimension of the hidden space is smaller than the dimension of the input data, it acts as a dimensionality reduction or data compression method.

Encoders and decoders are usually built by deep neural networks, as described in the following section (Sec. 6.3.2), but q φ (z|x) and p θ (x|z) can be treated as probability distributions. Hence, VAEs can be formulated in terms of statistical modeling and also explicitly represent the hidden space as a probability distribution.

The main advantages of VAE for protein-sequence modeling can be summarized as follows: First, there is no assumption such as the pairwise interactions among the amino-acid variables, so possible interactions can be multi-variable interactions. Second, as we mentioned earlier, the distribution in the hidden space is flexibly adaptable considering the characteristics of data.

The organization of this section is as follows:

• In Sec. 6.2.1, we define the objective function of VAE, and discuss how encoder and decoder depend on it.

• In Sec. 6.2.2, we review (deep) neural network algorithms regarding the application to VAEs.

• In Sec. 6.2.3, we apply a VAE as protein-sequence generative models.

• In Sec. 6.2.4, we investigate the statistical properties of the hidden spaces of protein-sequence VAEs.

• In Sec. 6.2.5, we classify protein subfamilies by exploiting the hidden space. Here we also propose another VAE-based algorithm using a generalized prior distribution for hidden variables.

Introduction

As mentioned before, VAEs are defined in terms of statistical models. Indeed, behind VAEs, there is the idea of a (log-) likelihood function L(θ) = log p θ (x).

One of the most important ideas of VAEs is that, instead of dealing with p θ (x) directly, it assumes that there is another statistical variable, a hidden variable (we will formally define it in the context of VAEs later) and its marginalized distribution is defined by the Bayesian theorem:

p θ (x) = p θ (x|z)p θ (z) p θ (z|x) . (6.3)
The benefit of adding an additional stochastic variable is the gain in the complexity of probability distribution. If p θ (x) is rather a simple model but the empirical distribution aimed to be learned is more complex, learning of parameters θ will be difficult. Simple p θ (x|z) and p(z) may lead to complex models, e.g., Gaussian mixtures instead of a simple Gaussian.

The objective function of VAE can be derived as follows:

log p θ (x) = log p θ (x) q φ (z|x) = log p θ (x|z)p θ (z) p θ (z|x) q φ (z|x) = log p θ (x|z)p θ (z)q φ (z|x) q φ (z|x)p θ (z|x) q φ (z|x) = log p θ (x|z) q φ (z|x) -D KL (q φ (z|x) p θ (z)) + D KL (q φ (z|x) p θ (z|x)) ≥ log p θ (x|z) q φ (z|x) -D KL (q φ (z|x) p θ (z)) , (6.4)
where q φ (z|x) is defined as another conditional distribution of z given x. Note that p θ (z) is an assumed prior distribution, thus given. For the second to the third equation, we used the Bayesian theorem in Eq. 6.3. For the transformation to the final inequality in Eq. 6.4, we used the non-negativity of KLD. A necessary and sufficient condition for establishing equality in the Eq. 6.4 is D KL (q φ (z|x) p θ (z|x)) = 0 .

The final result in Eq. 6.4 is a lower bound of the log-likelihood function. It is as objective function of the VAE and called Evidence Lower Bound (ELBO):

L ELBO (θ, φ) = log p θ (x|z) q φ (z|x) -D KL (q φ (z|x) p θ (z)) . (6.5)
It has to be maximized with respect to the model parameters θ (for decoder) and φ (for encoder). These probabilities can be interpreted in relation to each function of VAE as follows: p θ (x|z) reproduces a data point x from z, hence it stands as a (probabilistic) decoder. On the contrary, q φ (z|x) embeds a data point x to z, therefore it works as a (probabilistic) encoder.

ELBO learning consists of two distinctive elements. First, accurate data reconstruction (the first term in Eq. 6.5). Second, minimization of KLD between the encoder and an assumed hidden prior distribution (the second term in Eq. 6.5).

To accurately reconstruct training data, the conditional log-likelihood log p θ (x|z) is maximized with respect to θ, after marginalizing the hidden variables. Note that the marginalization is not done by p θ (z|x), but q φ (z|x). Therefore, L ELBO is a lower bound of the log-likelihood function, and q φ (z|x) is its variational function.

As the second element: KLD between the parameterized encoder distribution q φ (z|x) and assumed prior distribution p θ (z) is minimized by changing the parameter φ. In general, factorized Gaussian distribution is chosen for the encoder distribution q φ (z|x), and standard Gaussian distribution is used for the hidden prior distribution p θ (z).

Therefore, the idea of ELBO is that a method alternately optimizes between the variational function q φ (z|x) and the lower bound of the log-

Deep neural networks for VAE

In this section, we will review deep neural networks (DNNs) for VAE briefly and introduce some quantities of VAEs in terms of DNNs.

A neural network (NN) is a machine (algorithm) that has a set of variables called perceptrons or neurons as components and can learn arbitrary functional forms for a certain input data x (we assume x ∈ [0, 1] d ). Here, we denote a n-th perceptron at l-th layer as s l n , where n being n ∈ 1:d l , (as we defined later, we set l ∈ 1:l E for the encoder, and l ∈ l E :l tot (= l E +l D ) for the decoder). A perceptron s l n is multiplied by a weight parameter w l mn and is summed over all perceptrons in the same layer l, then becomes an argument of another perceptron s l+1 m in the next layer using u l (•) a non-linear function or activation function, cf., Fig. 6.2.b. As a nonlinear function, we use tanh or Rectified Linear Unit (ReLU) function [START_REF] Fred | Deep learning using rectified linear units (relu)[END_REF]. So, m-th perceptron located at (l + 1)-th layer can be written as

s l+1 m = u l d l n w l mn s l n , ∀l ∈ 1:l E . (6.6)
Here, the perceptrons in the first layer are the input values themselves, therefore d 1 = d. In the case of protein sequence data, we define each se-Figure 6.4: An expression of VAE with a factorized Gaussian distribution and DNNs. The left trapezoid represents an encoder, q φ (z|x), which is a DNN parmetrizing a factorized Gaussian distribution. The green squares represent perceptrons. Each blue rectangular is a layer of perceptrons. The perceptrons in the last layer are treated as model parameters of Gaussian distributions, N (µ φ , Σ φ ). Similarly, the right trapezoid is a decoder. Hidden variables z generated from the N (z|µ φ , Σ φ ) are used as an input variable of the decoder. z is transformed to an output variable through multiple layers of perceptrons. The output variable of the decoder and the input variable of the encoder should have the same dimension.

quence A as a one-hot sequence such that

x (i-1)q+A i = δ A i ,a . (6.7)
Therefore, a perceptron in the first layer is defined using Eq. 6.7,

s 1 n = x n , ∀n ∈ 1:d(= qL) . (6.8)
From now on, we will discuss DNNs take into account VAE implementation i.e., formally define the decoder and encoder using the DNN language (we will consider a model as shown in Fig. 6.4).

One of the key factors in determining the performance of a VAE is how and what probability distribution is constructed using DNNs. In most cases, the factorized Gaussian distribution is chosen for this purpose because it can well approximate continuous probability distributions to some extent and is theoretically well-grounded (we can analytically obtain the functional form) 3 .

Hence, we define the probability distribution for the encoder as follows:

q φ (z|x) = N (z|µ φ (x), Σ φ (x)) = 1 |2πΣ| exp -(z -µ φ (x)) t Σ(x) -1 (z -µ φ (x)) , Σ φ (x) = diag(σ 1 (x), . . . , σ P (x)) , (6.9)
where P is the dimension of the hidden space. The mean values and covariance matrices are defined by perceptrons of the last layer in the decoder DNN,

µ φ,n (x) = s l E n (x) log σ φ,n (x) = s l E P +1 (x) , (6.10)
where ∀n ∈ 1:P . Note that, although we use uniform covariances log σ φ,n (x) = const., ∀n ∈ 1:P (therefore the number of perceptrons in the l E th layer is P + 1), we can also assign different perceptrons for each covariance (in that case, the number of the perceptron would be 2P ) .

A hidden variable z is the input variable to the decoder. It is converted to an output variable through the repetition of multiple nonlinear transformations involved in the decoder,

s l+1 m = u l n w l mn s l n , ∀l ∈ (l E + 1):l tot s l E +1 n = z n , ∀n ∈ 1:P . (6.11)
Note that, in order to have an output variable compatible to an one-hot sequence, we use a Sigmoid function for the nonlinear function at the last layer,

u ltot (•) = σ(•) = (1-exp(-•)) -1
. Since these elements are continuous valuables, it is not yet a one-hot sequence. To be an one-hot sequence, the maximum argument for each residue is normally used.

For the learning, we apply the backpropagation algorithm. Other technical information is summarized in Appendix G.1.

Finally, when using it as a generative model, the model parameters are fixed, and only the decoder is used. Since the decoder is assumed to be a conditional distribution of x given z, it can generate sequences. For a DNN, it can be done by feeding a random variable (hidden variable) to the decoder neural network. Random variables were sampled from the P -dimensional standard normal distribution N (z|0, I P ) and used as input values for the decoder. Note that assuming N (z|0, I P ) for z means, the prior distribution p θ (z) is a standard Gaussian distribution in Eq. 6.5. In this case, there is no dependency on the model parameters θ, so we will ignore the subscript parameter to the prior distribution.

Protein sequence design using VAE

The first thing that has to be assessed is the capacity of VAE as a generative model, to confirm it as a model that can properly capture protein data. To this purpose, we report here the standard single-site frequencies and two-point connected correlations from VAE. Note that these are not fitted quantities unlike in pairwise-Potts models. In the following experiments, we focus on PF00072 domain data (the same data as in [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF]) because of the abundant sequences and available information about distinct protein subfamilies (cf., Sec. 4.3.2).

In this experiment, the architecture we used here is summarized in Table 2. We employed relatively simple architecture for DNNs because as the number of layers increases, required learning processes increase, and learning control also becomes difficult. As a heuristic, we used more layers in the encoder to avoid the posterior collapse, which is a well-known problem for ELBO learning and is commonly occurred [START_REF] Lucas | Mixed batches and symmetric discriminators for gan training[END_REF][START_REF] Lucas | Don't blame the elbo! a linear vae perspective on posterior collapse[END_REF]. Posterior collapse is that the decoder ignores the encoded variables, which corresponds with neglecting the second term in Eq. 6.5. This problem can happen if the encoder is not enough complex and/or encoded signals are less informative. Fig. 6.5 shows the comparison of correlations with natural sequences (training data) and generated sequences from the VAE and bmDCA. VAE sequences were generated as follows: First, we learned the decoder and encoder based on the ELBO function and backpropagation algorithm. After that, we confirmed that the reconstructed sequences are similar enough to the input sequences (property as an autoencoder). Then, by using a random variable taken from a standard Gaussian distribution z ∼ N (z|0, I P ) as an input variable to the decoder, the decoder can generate artificial sequences. Note that we assumed standard Gaussian distribution as a prior distribution p(z) in ELBO learning in Eq. 6.5 . These two-point correlations are obtained by matrix-matrix multiplication operations using q × L matrices and did not use the one-hot sequence filter. It shows that statistics come from VAE is almost the same quality with bmDCA in terms of single-site and two-point frequencies.

Latent space analysis

As mentioned earlier, PCA analysis cannot well capture the underlying data features in cases where these include non-linearities and thus higher-order effects. On the other hand, autoencoder-based low-dimensional analysis can consider non-linear and higher-order effects, and all information of the data point is encoded into the internal hidden space.

The latent space of VAE can encode the underlying statistical features in low dimensions. Hence, the embedded variables are statistically as informative as raw-data space when VAEs are generative. Moreover, both encoder and decoder are probability distributions. Not only it is theoretically easy to handle, but it also has the following practical advantages: suppose two input data x A and x B are given and we want to understand this similarity. A naive idea is a Hamming distance between x A and x B . Similarly, we can compute the distance between the corresponding hidden variables for both data points, z A and z B , in the feature space used by VAE. Furthermore, VAE can also enable us to estimate the likelihood of an encoded sequence via q φ (z A |x B )/q φ (z B |x B ), which can assess how likely a data point x A belongs to the same ensemble as x B .

As we saw in the last section, the sequence-ensemble of VAE can reproduce the connected correlations of the natural sequences when the learning of VAE is correctly done. Hence we can assume that VAE provide a statistically supported higher-order hidden space. In this section, we exploit the hidden space of VAE. More specifically, in order to investigate statistical properties of protein-sequences and some generative models, we investigate single-point mean and two-point connected correlations in hidden space,

ψ u = 1 M M m=1 µ u (x m ) ψ uw = 1 M M m=1 µ u (x m )µ w (x m ) -ψ u ψ w , u = w , (6.12)
here we omit the symbol of the model parameters φ for simplicity. As introduced in Sec. 6.2.2, µ u (x) is the mean value of u-th hidden variable. Since, µ u (x) itself is a typical hidden value given x, ψ u represent an ensemble average of hidden values. Similarly, ψ uw is a covariance of typical hidden variables. Since the hidden variables are obtained by repeating the non-linear transformation of the linear combination of perceptrons variables (including amino acids), {ψ u } contains also multilinear effects. Moreover, since the decoder can successfully reproduce the training data, the encoded variable µ u (x) and {ψ u } should be statistically relevant.

Fig. 6.6 shows comparisons of the natural sequences used for learning the VAE in Sec. 6.2.3, and (a) profile sequences, (b) bmDCA sequences, and (c) VAE sequences (generated using the same condition of Sec. 6.2.3) in hidden space.

In Fig. 6.6.a, a profile model shows strong correlations between naturals sequences in the hidden space. Notably, the hidden-covariancies can show strong correlation even if profile models cannot generate strong correlations at all in terms of two-point connected correlations in sequence space. This result is not surprising, because there are strong influences from the singlesite frequencies f i (a) in ψ uw due to the multiple non-linear transformations in the DNN.

In Fig. 6.6.b and Fig. 6.6.c, we show the comparison between natural sequences, bmDCA and VAE. It shows that the reproducabilites of statistics using the VAE is better than the bmDCA in the hidden space, it is clear by comparing the bmDCA and VAE in terms of ψ uw . However, it is not very astonishing because the model generates sequences, and the model provides the hidden space is the same VAE model, therefore the features in hidden space are characteristic variables for the VAE, which might be different from the important variables for bmDCA (single site frequencies and pairwise fre-

Protein sequence classification

In general, VAEs use a factorized Gaussian distribution for q φ (z|x) and assume a prior distribution p(z) = N (z|0, I P ) (hence we omit the parameter symbol θ in the prior distribution). Therefore the decoder can generate samples when fed with statistical variables from the standard Gaussian distribution. However the choice of the standard Gaussian distribution as a prior distribution is not a fundamental constraint of VAE. Relaxing the conditions of the standard Gaussian distribution as a prior distribution does not conflict with the derivation of the ELBO function.

It is a rather reasonable choice to assume a factorized Gaussian distribution for q φ (z|x) to describe latent space, which contains continuous variables. Due to the conditioning by x, assuming the factorized Gaussian distribution result effectively in a mixture of factorized Gaussian distributions. However, when assuming a standard Gaussian distribution as the prior distribution, the distribution of q φ (z|x) would be simple and the mean values tend to be zero because of the minimization of the KLD between q φ (z|x) and p(z) in Eq. 6.5.

A problem that occurs frequently in VAE-based classification problems, is that the hidden variables are projected into approximately the same area and that their distribution does not show a clear structure. In fact, it is an anticipated result since when we assume the standard Gaussian distribution as a prior distribution p θ (z). This problem could be more pronounced for some data structures, such as one-hot encoded sequences. This tends to make VAEs spend a lot of effort learning features of specific data-structure (e.g., one-hot sequence). Differences between sub-ensembles will be more ambiguous when compared to differences due to data structures.

VAE with structured prior distribution -

Here, it is assumed that an encoder and a decoder are learned initially using standard VAE learning. We introduce additional hidden variables, c ∈ 1:K (K is any positive integer), in order to make the hidden prior be more complex distribution. To do this, we assume the following hidden prior distribution,

p θ (z, c) = p θ (z|c)p θ (c) , (6.13)
where p θ (c) is a categorical distribution over 1:K and p θ (z|c) stands for a non-zero mean factorized Gaussian distribution, p θ (z|c) = N (z|μ c , I P ) . (6.14)

Here, μc is a parameter vector defined in the hidden space, which we will discuss it later in detail. In this manner, we modified the Decoder. Similarly, the encoder changes in the following fashion:

q φ (z, c|x) = q φ (c|z)q φ (z|x) . (6.15)
The factorized distribution q φ (z|x) is the same as the standard encoder distribution defined in Eq. 6.5. The other distribution, q φ (c|z) is a probability distribution, which generates a value for c given z.

From here, we will discuss how to obtain the probability distributions within the framework of VAE we introduced earlier.

Let us make a first point about p θ (z|c), which generates z given c from a Gaussian distribution with a mean μc , as shown in Eq. 6.14. We define μc using the information of the distribution in hidden space at the previous learning step:

μt+1 c = zp θ t (z|c) dz Data , (6.16)
were the superscript t indicates a learning epoch. Note that the data dependency result from c and z. As the initial states of the mean vectors μt=0 c , c ∈ 1:K, we assign random vectors drawn from a standard Gaussian distribution, μt=0 c ∼ N (μ|0, I P ).

Second, we define one of the factorized distribution, q φ (c|z) . As in the previous discussion, we take an iterative strategy using the previous state of model parameters,

c t+1 = argmax c ∈1:K q φ t (c |z) , (6.17)
i.e., each z is assigned to the most probable cluster c . As for the specific mathematical form of q φ (c|z), we use a Gaussian distribution with the mean vector, q φ (c|z) = N (z|μ c , I P ) ,

As an initial state of c t=0 , it is chosen from a Gaussian distribution N (z|μ t=0 c , I P ) , hence the categorical variables are assigned uniformly.

Note also that this iterative optimization of {μ c , c} defined in Eq. 6.16, Eq. 6.17 and Eq. 6.18, is equivalent to the K-means clustering algorithm, an supervised classification algorithm.

Last, in order to assess and control effects of the assumed generalized prior distribution for hidden variables, we introduce a control parameter β ∈ [0, 1] that changes the importance of the conditional log-likelihood term and the KLD term in Eq. 6.5. Note, a VAE including such a parameter β is known as β-VAE [START_REF] Durk P Kingma | Semi-supervised learning with deep generative models[END_REF]. Finally, we define the VAE objective functions with the generalized prior distribution regarding all the quantities we defined here,

L β,K (θ, φ) = (1 -β) log p θ (x|z, c) q φ (z,c|x) -D KL (q φ (z, c|x) p θ (z, c)) . (6.19)
By definition, as we increase β, we can enhance the effect of the non-zero mean Gaussian distribution via the KLD. This approach is closely related with the conditional VAE (CVAE), which is a semi-supervised learning algorithm using label information (cf., Appendix G.1.3). The major difference is, although this method can use labeled variables, it is not necessary; we perform unsupervised learning, whereas CVAE is a semi-supervised learning algorithm.

Moreover, the cluster information of c has in effect via the prior distribution of hidden variables p θ (z, c) . However, CVAE introduce additional perceptrons to the encoder and decoder to take into account the label information, and the dependency of the label is absorbed in the DNNs. Hence, how c depends analytically on the probability distribution is not clear.

In the next section, we will see the results of the classification of protein sequence subfamilies based on this method, and compare it to an existing functional protein classification based on domain architecture, as already used in [START_REF] Shimagaki | Selection of sequence motifs and generative hopfield-potts models for protein families[END_REF].

Results

Here we apply this method to the classification of protein domain subfamilies, response regulator domain (Pfam ID, PF00072). The data set is the cf., Fig. 4.7. That is, subfamilies that are indistinguishable in RBM-based low-dimensional space are also indistinguishable in VAE hidden space. For example, Class PF00072-PF00512 (purple), PF00072-PF00512(after 500 aa) (red) and PF00072-PF00512(after 1000 aa) (orange) are projected into same areas.

Conclusion

In this chapter, we investigated the effects of higher-order statistics/interactions in protein sequence data. We also discussed the Variational Auto-Encoder (VAE) as a generative model considering the influence of higher-order interactions.

Sec. 6.1 discussed the presence of significant three-point correlations that are not explainable by the two-point correlations as a model-independent analysis. Our experiments found that all significant three-point correlations always involve more than two significantly large two-point correlations. Although further numerical experimental evaluation is required, it would be interesting to directly compare large three-point correlations with a combination of two-point correlations and means as a supplementary study.

In Sec. 6.2, we examined VAEs as protein-sequence generative models that can consider higher-order interactions. Sec. 6.2.1 reviewed the principle of VAEs which are formulated by the objective function, Evidence Lower Bound (ELBO). We also confirmed the relations between the main components of VAE: encoder, decoder, and hidden space. In Sec. 6.2.2, we defined deep neural networks (DNNs) formally and explained how to construct probability distributions based on DNNs. In Sec. 6.2.3, we demonstrated an experiment of generative model for protein sequences based on a VAE (cf. Table 2). VAE can reproduce statistics of a protein family (PF00072), both single-site frequencies and two-point connected correlations are as good as the bmDCA (cf. Fig. 6.5 ).

In Sec. 6.2.4, we investigated statistical properties in the hidden variable space, defined by variables including higher-order effects. For this investigation, we exploited the hidden space provided by the experiment in Sec. 6.2.3. Note, since the decoder of the VAE can reproduce statistics accurately from encoded variables, the hidden space provides statistically significant variables. Notably, generated sequences from bmDCA can reproduce hidden statistics (hidden-means and hidden-covariance) accurately (cf. Fig. 6.6). In other words, no significant difference can be found between the natural protein sequences and the sequence generated from the pairwise coupling model, even in the space where non-linear and higher-order interaction effects exist. Sec. 6.2.5 proposed a VAE method based on structured prior distributions, based on the insight that distributions of hidden variables tend to be unimodal Gaussian distributions, which may prevent maximizing efficiencies of VAEs encoding and decoding. Here, the structured prior distribution is defined as a Gaussian mixture distribution with different means that are expected to reflect underlying data clusters.

As expected from the construction of the structured prior distributions, characteristic structures of hidden variable distributions were enhanced (cf. Fig. 6.8). Protein subfamilies that were distinctive in the results of lowdimensional analysis using the Hopfield-Potts model (cf. Fig. 4.7 in Sec. 4.3.2) have emphasized their cluster structures more in this VAE hidden space (c.2). However, some protein subfamilies still share areas, that is, such protein families are intrinsically indistinguishable. The same type of experiment should be conducted on more protein families to ensure the reproductibility.

Thanks to the development of next-generation sequencing machines, technology for extracting meaningful information from vast amounts of genomic data has become increasingly important over the last years. Consequently, machine learning technologies that can extract useful information concerning data without understanding the physical mechanisms of the data become more and more necessary. Such technologies as Deep neural networks, machine learning have come to astonishing results in structural biology in the last years [START_REF] Andrew W Senior | Improved protein structure prediction using potentials from deep learning[END_REF].

While the developing machine learning technologies are crucially important, the traditional modeling approaches that aware of understanding the physical phenomena behind the data are just as crucial for further evolutions of biology and genomics.

In this dissertation, we discussed protein sequence generation models based on statistical modelings and machine learning methods. Especially, we focused our studies on constructing biophysically understandable statistical models and selecting variables that are inherently important for describing protein sequences. Chapters 3, 4, and 5 are based on DCA [42,45].

Chapter 3 aimed to construct minimally constraint pairwise Potts models so that the remaining coupling parameters correspond to structural contacts by decimating coupling parameters. This sparse pairwise Potts models can remove more than 90% of coupling parameters without degrading the statistical properties expected as generative models. Moreover, even though coupling parameters are removed by more than 95%, the accuracy of residueresidue contact is maintained [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF].

Chapter 4 investigated the HP models, a model that imposes a lowrank structure on pairwise coupling parameters. These HP models can reproduce the statistics reasonably well, even with about 95% reduction in model parameters. Furthermore, the HP model can provide low-dimensional spaces that are able to capture the characteristics of protein subfamilies [START_REF] Cocco | From principal component to direct coupling analysis of coevolution in proteins: Loweigenvalue modes are needed for structure prediction[END_REF][START_REF] Tubiana | Learning protein constitutive motifs from sequence data[END_REF][START_REF] Shimagaki | Selection of sequence motifs and generative hopfield-potts models for protein families[END_REF].

Chapter 5 proposed methods for selecting statistically significant variables that increase the likelihood function for any generative model. Using these methods, we specified pairwise coupling parameters to enhance the likelihood of HP models, thus combining sparse couplings and low-rank couplings. Note that the sparse couplings tend to associate with spatial contacts, whereas low-rank couplings typically learn global correlations that are supposed to be due to phylogenies. Therefore, this chapter served as further analysis to distinguish correlations that are due to spatial contacts or phylogenies in protein sequences [44].

Lastly, chapter 6 explored the presence of higher-order statistics. Section 6.1 investigated significantly large three-point correlations that are not explainable by the two-point correlations. It revealed that all large threepoint correlations are consequences of the large two-point correlations in our experiments. In section 6.2, we employed variational autoencoders (VAEs) [START_REF] Diederik | Auto-encoding variational bayes[END_REF][START_REF] Diederik | Adam: A method for stochastic optimization[END_REF] as protein sequence generative models to consider nonlinear and higher-order effects. We proposed a framework to investigate higher-order effects in protein sequences by exploiting hidden spaces of VAEs. According to these methods, the protein sequences generated from the pairwise Potts model were not statistically significantly different from the natural protein sequences. In other words, up to second-order interactions are sufficient as a generative model of protein sequences.

The last two chapters need further researches in the future. For chapter 5, we should examine generative models that are re-trained after learning sparse spatial couplings and low-rank couplings. As a result of the relearning, these generative models could improve residue contact prediction further. Chapter 6 proposed the idea to exploit VAE hidden spaces for verification of statistical reproducibility of generative models. This direction has just been opened, many fundamental questions to be addressed are leaving. Some of the intriguing questions are: Can we construct a situation in which the hidden variables of VAE can have physical meanings? Can we understand VAE hidden spaces by comparing them with other hidden variable models, such as the HP models? There many things to be done. also submitted from the EMBL-Bank/GenBank/DDBJ nucleotide sequence resources (International Nucleotide Sequence Database Collaboration (INSDC)).

2. UniProtKB/Swiss-Prot: a high-quality manually annotated1 and nonredundant protein sequence database. All registered sequences are reviewed. It can also contain experimental conditions, functional information, and mechanical and thermodynamic features determined in silico.

3. UniProtKB/TrEMBL: a protein sequence database, which contains automatically annotated information, and these are unreviewed information. It also contains automatically generated annotations and functional characterization.

PDB -Protein Data Bank (PDB) is a repository of information regarding three-dimensional biological molecules such as proteins and nucleic acids. It can provide information about 3D structures of molecules, and experimental conditions that are used to determine the coordinates of atoms in molecules [START_REF] Helen M Berman | The protein data bank[END_REF]. Protein structure information is obtained by X-ray crystallography, NMR spectroscopy, or cryo-electron microscopy. Structural/ functional information for proteins is maintained by a web-based data server (e.g., PDBe [11], PDBj [START_REF] Akira R Kinjo | Protein data bank japan (pdbj): maintaining a structural data archive and resource description framework format[END_REF], and BMRB [START_REF] Bmrb Mallesham | Highly efficient cui-catalyzed coupling of aryl bromides with oxazolidinones using buchwald's protocol: a short route to linezolid and toloxatone[END_REF]).

The majority of protein structures are determined by X-Ray crystallography (around 146,000 entries) [START_REF] Helen M Berman | The protein data bank[END_REF]. The second and third most commonly used protein structure determination methods are NMR (around 13,000 entries) and electron microscopy (around 5,000 entries). Only 500 entries are determined by other methods. X-ray crystallography is almost inapplicable for some types of proteins due to technical and experimental limitations. The main factor is the difficulty in creating protein crystals of sufficient quality for analysis. For example, large proteins and membrane proteins are extremely difficult to make protein crystals in general. Therefore, some classes of protein structure information are significantly insufficient. lems. Indeed, bmDCA initialized by small couplings and another bmDCA initialized by the plmDCA couplings converge to different couplings, especially small couplings after the learning are substantially different.

One of the reasons is that the objective functions of pairwise Potts models involve an almost flat direction in the space of coupling parameters. Therefore, they tend to need long learning epochs to satisfy the fixed point equations Eq. 2.10 using gradient-based optimization methods. The situation becomes more severe when the number of model parameters is significantly greater than the number of training data, which is the typical case of our problems.

We typically use small random parameters as initial conditions of coupling parameters because small parameters correspond to a high-temperature regime and thus the mixing time is generally fast and the ergodicity breaking does not occur.

One could use optimized coupling parameters using CD-based learning as initial conditions of the coupling parameters (the similar idea to optimize bmDCA can be found in [START_REF] Ahmed A Quadeer | Mpf-bml: a standalone gui-based package for maximum entropy model inference[END_REF]). It can be written as follows using the eigenvalues and eigenvectors of the Pearson covariance matrix. When the eigenvalue is greater than 1, it is called attractive patterns ξ +,µ , and when the eigenvalue is less than 1, it is called repulsive patterns ξ -,ν . P + attractive mfHP patterns are:

ξ +,µ = 1 - 1 λ µ 1/2
ṽµ , µ ∈ 1:P + , (C.5)

where attractive eingenvalues are λ 1 ≥ λ 2 , ≥, . . . , ≥, λ P + > 1 .

Similarly, P (= P + + P -) repulsive mfHP patterns are:

ξ -,ν = 1 λ ν -1 1/2
ṽν , ν ∈ (P + + 1):P , (C.6)

where repulsive eigenvalues are 0 < λ P ≤ λ P -1 , . . . , ≤ λ P + +1 < 1 . Note the data in terms of statistical modeling. That means to construct a certain probability density p(x|θ), which carries the same statistical properties of the training data (as following we will denote vectors without bold for simplicity such as θ → θ) . Here, θ is a set of model parameters that we want to optimize so that the probability distribution can reproduce statistics of the training data. Up to here, the assumed setup is the same as the MLE-based learning.

The significant difference with the maximum likelihood (ML) method is that ML based methods assume equilibrium distribution for estimating gradients of the objective functions, i.e., log-likelihood functions, whereas CD-based methods use non-equilibrium distribution to obtain gradients.

Suppose that the probability distribution of interest belongs to an exponential family, p(x|θ) = exp(-E(x|θ))/Z , where Z is the partition function. Accordingly, the log-likelihood function can be written as

l(θ) = E(x|θ) p data -log Z(θ) , (E.1)
where, p data denotes an empirical distribution. Similarly we denote p data = p(x|θ) hereafter. Suppose we optimize the model parameters θ using the standard ML approach. Therefore, we perform the differential of the loglikelihood function l(θ) with respect to the model parameters θ, ∂l(x .

(E.3)

Where, q (k) (x|θ; x 1:N ) is a probability distribution, after k step transitions with an appropriate transition kernel p(x|x ; θ) from an empirical distribution p data . We call Eq. E.3 as the update equation for CD learning. From now on, we call this probability distribution as the distribution after k step transitions. Formally, we can define q (k) (x|θ; x 1:N ) as following, q (k) (x|θ; x 1:N ) =   k t=1 x (t) ∈X p(x (t) |x (t-1) ; θ)   p data (x (0) |x 1:N ) , (E.4) where x (k) = x, and x (0) ∼ p data (x|x 1:N ). Ref. [START_REF] Baroni | Don't count, predict! a systematic comparison of context-counting vs. contextpredicting semantic vectors[END_REF] shows that this fixed point equation can be derived as a consequence of an approximation of the log-likelihood function by expanding it. Historically, the CD method was introduced to minimize a contrastive divergence [START_REF] Manning | Foundations of statistical natural language processing[END_REF][START_REF] Bengio | A neural probabilistic language model[END_REF], therefore its objective function is assumed to be different from the log-likelihood function.

The contrastive of the divergence is defined as D KL (p data p model ) -D KL (q (k) p model ) , (E.5)

If 1 k, it leads to q (k) ∼ p model , therefore the objective function becomes the same as the one of the MLE.

The differentiation of Eq. E.5 with respect to the parameter θ cannot be exactly the same as Eq. E.2, but assuming the parameter θ dependency on q (k) is not so large i.e., |∂q (k) /∂θ| 1. This assumption is usually satisfied unless the set of parameters is at a critical point.

E.1.2 Contrastive Divergence for latent variable models

CD methods are used for Restricted Boltzmann Machines (RBMs) as the standard learning algorithm. Suppose a probability distribution p(v, h) is characterized by two types of variables, namely visible variables v and hidden or latent variables h. Here we assume there are corresponding observables in an assumed data set for the visible variables, and the probability we can obtain is only a probability after marginalizing the hidden variables, p(v) = h p(v, h).

The CD learning algorithm for hidden variable models is as follows:

1. Initialize the visible variables using one sample in the training data at the first MC transition step t = 0, v (t=0) ← x ∈ x 1:N .

2. Then, the hidden variables are obtained from the conditional probability distribution, h (t=0) ∼ p(h|v (t=0) ; θ) . Note that the conditional distribution in the case of RBMs becomes independent among each hidden variable h because each hidden variable has interacted only via the visible variables (there is no interaction between hidden variables).

3. The visible variables are updated by taking them from the conditional distribution, v (t=1) ∼ p(v|h (t=0) ; θ) .

-E(v, h) = ij w ij v i h j + i a i v i + j b j h j (E.6)
For simplicity, we assume the visible variables are binary variables v i ∈ {0, 1}. Where, {w ij } are model parameters for interactions between visible and hidden variables. {a i } and {b j } are other model parameters for visible and hidden variables, respectively.

These model parameters are optimized using a gradient-based learning algorithm. Eq. E.5 and Eq. E.6 lead to the following equations for updating the model parameters. Learn the RBM so that δw ij , δa i , δb j , ∀i, j will be small. A more appropriate learning evaluation method is explained in the following section, E.3.

E.2 Persistent Contrastive Divergence

CD methods can remarkably reduce the computational processes. However, CD samples that are generated from q (k) (x|θ; x 1:N ) tend to depend strongly on the training data. As a result, the energy function (probability distribution) using the CD learning assigns a significantly lower value (high probability) to state points that are included in the training data. On the contrary, state points that are not included in the training data, tend to have high energy values (low probabilities).

Besides, the probability distribution constructed using MCMC with the model parameters estimated by the CD method tends not to reproduce the training data correctly. Therefore the data statistics are not accurately reproduced (cf. [START_REF] Shimagaki | Selection of sequence motifs and generative hopfield-potts models for protein families[END_REF][START_REF] Berglund | Stochastic gradient estimate variance in contrastive divergence and persistent contrastive divergence[END_REF]).

One of the reasons for this phenomenon is that all data points are used as initial states of the probabilistic transitions p(x (t) |x (t-1) ; θ) at each learning epoch, and the samples used for estimating the update equations Eq. E.3 are constructed without sufficiently exploring the state space. Thus, the CD learning minimizes the flow of the transition probability that escapes from the data points. This observation is closely related to Minimum Probability Flow learning [START_REF] Sohl-Dickstein | Minimum probability flow learning[END_REF][START_REF] Sohl-Dickstein | New method for parameter estimation in probabilistic models: minimum probability flow[END_REF] .

Considering the drawbacks of the CD learning due to the lack of sufficient state space search, T. Tieleman Ref. [START_REF] Tieleman | Training restricted boltzmann machines using approximations to the likelihood gradient[END_REF] proposed Persistent Contrastive Divergence (PCD) learning. CD methods and PCD methods are equivalent in terms of computational complexity, but the PCD methods can estimate the update equations with samples that are similar to those generated from the model. Thus, it is closer to the ML learning. When using protein family data, empirically, the probability distributions learned by the PCD methods reproduce the training data well.

The significant difference between the PCD method and the CD method is the initialization of the visible variables: Instead of initializing the visible variables by samples in the training data at the starting point of the transition of each learning epoch, reuse the final states of the previous CD chains.

More precisely, PCD learning consists in the following steps:

1. Initialize the visible variables using one of the training data (same as CD)

2. Apply MCMC k times, then keep samples that are used to estimate the update equations.

3. Use the samples generated and stored during the previous learning epoch as the initial states for the current learning epoch.

4. Repeat processes 2. and 3.

Note that if a learning ratio is sufficiently small (slight changes in model parameters between epochs), the PCD samples would be equivalent to the samples from the equilibrium distribution i.e., the PCD learning gives an equivalent solution to the MLE.

E.3 Convergence criterion for Contrastive Divergence based learning

The update equations are used to evaluate the learning processes in general. However, these quantities cannot be used as appropriate measures to assess the CD-based learning as showed in [START_REF] Hinton | A practical guide to training restricted boltzmann machines[END_REF]. Note that the update equation for the CD learning Eq. E.3 is not the "reconstruction error", i.e., the difference between statistics based on the training data and samples from the model. Hence the update equation Eq. E.3 cannot be used to assess the learning processes (e.g., assessing adequate learning epoch). Particularly learning hidden variable models require cautious treatments [START_REF] Fischer | Empirical analysis of the divergence of gibbs sampling based learning algorithms for restricted boltzmann machines[END_REF][START_REF] Hinton | A practical guide to training restricted boltzmann machines[END_REF].

To evaluate CD-based learning processes, Annealed Importance Sampling (AIS) [START_REF] Radford | Annealed importance sampling[END_REF][START_REF] Fischer | Empirical analysis of the divergence of gibbs sampling based learning algorithms for restricted boltzmann machines[END_REF][START_REF] Salakhutdinov | On the quantitative analysis of deep belief networks[END_REF]] can be used. The AIS is defined as a ratio between two partition functions named Z(θ (t) ), Z(θ (t+1) ). It can be estimated ) is defined as an expected value of the probability distribution p(x|θ) = e -E(x|θ) /Z(θ). In the last equation we assume that the expected value • p(x|θ (t) ) can be approximated as an ensemble average and that those samples come from the probability x m ∼ p(x|θ (t) ).

If p(x|θ (t+1) ) and p(x|θ (t) ) are quite different, the ratio cannot be estimated accurately. In such a situation, we can introduce intermediate probability distributions between these two probability distributions and estimate is exactly the same form as we saw in Eq. 5.9.

If there is a significant discrepancy between the empirical distribution f ij (a, b) and the assumed model p ij (a, b), in this case p M [f ij (a, b), p ij (a, b)] becomes particularly small. That is exactly the case in which we need to make corrections by introducing the conjugate parameters J ij (a, b).

The same argument can hold not only for two-point frequencies, but also for single-site frequencies, and even three-or more-point frequencies.

F.2 Residue contact for additional protein families

As we showed in Sec. 5.3.1, the likelihood-variation-based residue-residue contacts prediction can improve contact as the number of hidden variables P increases in the RBMs. In this section, we will report residue contact prediction results using the same methods for other protein families, PF00072 and PF13354. We used identical data sets that are used in Ref. [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF]. ) itself has already been predicted as accurately as plmDCA, the effect of the phylogenetic effect on this protein family is probably not so substantial. ELBO should also be slightly modified (we don't rewrite the objective function but it is just plug Eq. G.1).

It is easy to check that the derivative with respect to parameters of the expectation of this stochastic objective function is equivalent to the derivative of the original ELBO objective function, ∇ θ,φ E N (ξ;0,I) [ Lθ,φ (x, ξ)] = ∇ θ,φ L(x).

Regarding the reparametrization, we also need to take into account Jacobian, q φ (z|x) ∂z ∂ξ = N (ξ; µ, Σ) , (G.2)

where A is determinant of a matrix A, and ∂z ∂ξ is the Jacobian, ∂z ∂ξ = ∂(z 1 , z 2 , . . . , z P ) ∂(ξ 1 , ξ 2 , . . . , ξ P ) = ∂z i ∂ξ j i,j∈1:P . (G.3)

In the case of the above example, we supposed that the hidden variable comes from Gaussian distribution,

∂z i ∂ξ j = ∂ ∂ξ j (µ + Σξ) i = Σ ij . (G.4)
The conditional probability distribution of hidden valiables given visible variables can be represented as following, log q φ (z|x) = log N (z|µ(x), diag(σ(x))) = P u log N (z u ; µ u (x), σ 2 u (x))-log σ u (x) .

Put everything together, the objective function we used is represented as follows, Lθ,φ (x, ξ) = log p θ (x|z))p θ (z)log q φ (z|x) = log p θ (x|z)) + P u log N (z i ; 0, 1)log N (z u ; µ u (x), σ 2 u (x))log σ u (x) .

(G.5)

Here, we assumed z = z(ξ, x; φ). Note that we have not explicitly stated the index of the data points. To be the objective function, need to sum Eq.

200

G.5 for all data points1 .

G.1.2 Other types of VAEs

Here we show some variants of VAE that are closely related to our study.

β-VAEβ-VAE is a simple powerful effective method that aims to resolve the frequently occurred problem, posterior collapse [START_REF] He | Lagging inference networks and posterior collapse in variational autoencoders[END_REF], where the decoder learns to ignore the hidden variables.

The β-VAE can be realized by introducing a hyper-parameter β to ELBO objective function2 .

L βVAE (θ, φ) = (1β) log p θ (x|z) q φ (z|x) -βKL(q φ (z|x) p θ (z)) , β ∈ (0, 1) , (G.6) if β = 0.5, β-VAE recovers original VAE. The small β imposes less constraints to the latent space. On the other hand, large β induces strong constraints so that the hidden variables are independent when it is assumed standard Gaussian p θ (z) ∼ µ p θ (z µ ) .

In order to avoid the posterior collapse, we can push VAE to learn KLD terms more by increasing β. That is, the posterior of the encoder becomes similar with the distribution of hidden prior [START_REF] Christopher P Burgess | Understanding disentangling in \beta -vae[END_REF]. Moreover, by controlling the β, we can enforce features or hidden variables [START_REF] Higgins | beta-vae: Learning basic visual concepts with a constrained variational framework[END_REF].

Conditional VAE -The standard VAE learning is unsupervised, which means there is no additional label to train VAEs. The conditional VAE (CVAE) [START_REF] Sohn | Learning structured output representation using deep conditional generative models[END_REF][START_REF] Lecun | MNIST handwritten digit database[END_REF] is a semi-supervised VAE that can generate a certain class of variables according to a label variable. For example, it can specifically generate images of "7" when the model learns images of handwritten numbers (0-9) with the labels (also 0-9) (cf. MNIST data set).

Formally, the objective function of CVAE is defined as the variational lower bound of the conditional probability distribution of data points given class variables. Thereby, the hidden prior distribution also depends on the class variables.

L CVAE = log p θ (x|z, c) q φ (z|x,c) -KL(q φ (z|x, c) p θ (z, c)) , (G.7)

where, c is the conditional variable. The class variables are assumed as additional input variables, so they are assigned to additional perceptrons. Therefore, CVAE can be learned by standard ELBO learning.

G.2 Conditions of the experiments

Batch learning -We also used batch-learning and batch-normalization methods. Batch-learning is a method to optimize model parameters using a sub-ensemble of training data for each learning epoch. The gradients of the objective function are obtained using sub-ensembles that contain B sequences.

Our study constructed a batch ensemble by resampling using the reweighting parameters (cf. Sec. 2.4.3) from all sequences in the training data for each learning epoch. We didn't find any significant difference by changing the batch size B between 300 and 1000.

Advantages of batch learning are:

1. Reducing requiring computational processes.

2. Various combinations of samples can be provided to DNN for each learning epoch.

3.

Reducing the sample size enhances the stochastic effect and helps avoid local minima. Note that DNN-based problems usually involve many local minima solutions.

The second reason can be useful for VAE learning, but particularly for Generative Adversarial Network (GAN) algorithm [START_REF] Goodfellow | Generative adversarial nets[END_REF][START_REF] Repecka | Expanding functional protein sequence space using generative adversarial networks[END_REF][START_REF] Burak Yelmen | Creating artificial human genomes using generative models[END_REF], to overcome so-called mode collapse [START_REF] Salimans | Improved techniques for training gans[END_REF][START_REF] Yi | Generative adversarial network in medical imaging: A review[END_REF][START_REF] Metz | Unrolled generative adversarial networks[END_REF][START_REF] Creswell | Generative adversarial networks: An overview[END_REF]. Batch normalization is a method for removing gradient bias, which is equivalent to using normalized training data as input. By using this method, learning is expected to be stable and fast [START_REF] Santurkar | How does batch normalization help optimization[END_REF].

Optimization of DNNs -For optimization methods, we employed a stochastic gradient-based optimization method Adam [START_REF] Diederik | Adam: A method for stochastic optimization[END_REF] (adaptive moment estimation), which is a commonly used optimization method for DNNbased algorithms. To apply Adam, only the first and second moments of gradients are needed (these are obtained as a by-product while optimizing the DNN) and are computationally efficient.

We have selected the weight decay parameters for the first and second moments, which are Adam's hyper-parameters, to 0.99 and 0.9999, respectively.

One-hot sequence -To learn protein sequence data using a neuralnetwork-based algorithm, we map categorical variables representing amino acids to binary variables, {1:q} L → {1, 0} qL . We refer to this binarized sequences as one-hot sequences.

In general, initial conditions of input neurons are symmetric; all input neurons connect to the next layer's all neurons. Asymmetricity of the neural networks due to the one-hot data structure emerges as a consequence of learning.

We often represent a one-dimensional vector containing qL elements as a q × L matrix. Here, it corresponds to each residue site in the column, and in the case of one-hot sequence, only one of the q elements is 1 and the others are 0. Fig. G.1.a shows a typical PF00072 protein sequence using one-hot representation (cf. Sec. 6.2). These white and black elements in the matrix correspond to one and zero, respectively.

Output sequences from the VAEs are continuous variables R qL because the activation functions of the output layer are the Sigmoid function. Therefore, it is necessary to convert it into a one-hot sequence by some filter. One naive way is to take the argument with the largest output value of all the q-state amino acids.

APC

  Average Product Correction BM Boltzmann Machine CD Contrastive Divergence DCA Direct Coupling Analysis DNN Deep Neural Network HMM Hidden Markov Model HP Hopfield model KL Kullback-Leibler MaxEnt Maximum-Entropy MC Monte Carlo MCMC Markov Chain Monte Carlo mfHP mean-field Hopfield model MI Mutual Information ML Maximum Likelihood MLE Maximum Likelihood Estimation MSA Multiple Sequence Alignment PCD Persistent Contrastive Divergence PDB Protein Data Bank PPV Positive Predictive Value RBM Restricted Boltzmann Machine

Figure 1 . 4 :

 14 Figure 1.4: Top: An MSA is a rectangular matrix containing aligned (amino acid) sequences. Some sites (columns) show conservation, particularly sites involved in Cysteine (C) conserved clearly, in the case of the protein family PF00014 shown here Fig. 1.4. Notably, there are strong pairwise correlations between several sites, which presumably contain information of the co-evolution. Bottom: HMM logo, a sequence motif to characterize the protein family based on single-site frequencies and entropy measures. Typically conserved sites are more emphasized.

Figure 1 . 5 :

 15 Figure 1.5: A typical profile HMM architecture. Shown squares, diamonds, and circles are denoted as match, insertion, and deletion states, respectively. An edge between the states is denoted as one of the transition probabilities t(s |s). (Source [14])

Figure 2 . 1 :

 21 Figure 2.1: Two-point connected-correlations between the natural and artificially generated sequences for the Pfam protein family PF00072. The ensemble of the artificial sequences reproduces the statistics of natural sequences in terms of two-point correlations with high accuracy. The result is expected in BM learning, but technically it is difficult to reproduce the statistics.

Figure 2 . 2 :

 22 Figure 2.2: Left: Contact prediction for a protein family, sigma-70 factor as an example. (a) shows the top 20 DI predictions, which are essentially the DCA prediction, and (b) shows the top MI predictions. Each pair of residues less than the 8 Å are linked with the red edges, and pairs other than that linked with the green edges. Right: Mean true positive (TP) rate for 131 domain families, as a function of top-ranked contacts. The curve of the DCA (denoted as DI) outperform the other methods MI and Bayesian dependency tree [41]. Both of the figures are adapted from [42].

Figure 2 . 3 :

 23 Figure 2.3: Top: The three-point correlations. (a) The three-point correlation distribution of the profile model (vertical axis) is almost zero, and there is no association with the three-point correlations of natural MSA (horizontal). However, the three-point correlation from sequences generated by a bmDCA can well reproduce the three-point distribution from the MSA. Bottom: PCA projection for (c) the MSA, (d) a profile model, and (e) a PPM. The profile model has a completely different sequence distribution in the PCA space compared with the result of the natural sequence. In contrast, the PPM almost perfectly reproduces the distribution of it. Figures were adapted from[43].

  Figure 2.3: Top: The three-point correlations. (a) The three-point correlation distribution of the profile model (vertical axis) is almost zero, and there is no association with the three-point correlations of natural MSA (horizontal). However, the three-point correlation from sequences generated by a bmDCA can well reproduce the three-point distribution from the MSA. Bottom: PCA projection for (c) the MSA, (d) a profile model, and (e) a PPM. The profile model has a completely different sequence distribution in the PCA space compared with the result of the natural sequence. In contrast, the PPM almost perfectly reproduces the distribution of it. Figures were adapted from[43].

Figure 2 . 4 :

 24 Figure 2.4: Left: Pie charts showing the results of folding experiments for N (n = 42), CC (n = 43), IC (n = 43), or random (n = 19) WW domain sequences. The experiments shown that none of the sequences folds into the native structure for both R sequences and IC sequences. On the contrary, 28% of the CC sequence realized the native structure. Considering that a 67% of the N sequences could be natively folded in the experimental conditions, the result of the CC sequences is astonishing. Right: The top panel shows the Potts-energy distribution of generated sequences for the WW domain (parameters were obtained by the ACE algorithm [64]). The histograms correspond to MC samples of the pairwise model (blue), which should be the best-fit distribution to the natural WW sequences of the singlesite model (profile model) (green), and random model (red). (Source [65, 66])

Figure 2 . 5 :

 25 Figure 2.5: (a) Schematic figure of the workflow to characterize CM activities. (b) Relationship between r.e. and log k cat /K m . It shows as reaction proceed E. coli strains with defective CM decrease in the minimal medium relative to the wild type reference. (c) Relation between the DCA energy and r.e.indicates that no functional sequence can exist above a certain DCA energy value. These figures were adapted from[START_REF] William P Russ | An evolution-based model for designing chorismate mutase enzymes[END_REF]

Figure 2 . 6 :

 26 Figure 2.6: The right column shows frequencies of the statistical energy (measured relative to EcCM) for natural sequences, sequences from MC sampling (T ∈ {1.0, 0.33, 0.66}), and sequences from the profile model. Sequences with the negative statistical (DCA) energy are observed in the natural and T = 0.66 and T = 0.33 sequences. However, none of the sequences has a negative energy for T = 1 or profile sequences. The left column shows frequencies of the r.e. for these ensembles of sequences. It shows that the natural sequences and bmDCA sequences with T = 0.66, 0.33 are successfully adapted in the selective conditions since the normalized r.e. is shifted toward positive values and show bimodal distribution (the left peak in the histogram corresponds with null E. coli, and the right peak corresponds with EcCM). Notably, sequences from bmDCA with T = 0.33 give higher enrichment of EcCM compare to the natural sequences. On the other hand, it also shows that both bmDCA with T = 1.0 and the profile model generate rarely produced EcCM sequences. The figure was taken from [67].

Figure 2 . 7 :

 27 Figure 2.7: Comparison between the statistical scores (∆φ = ∆E) and the experimental fitness, the minimum inhibitory concentration (MIC) of the antibiotic [69]. The five highest-scoring mutations shown in the figure (M182T, H153R, I247V, T265L/Q, and N276D) are reported as stabilizing mutants (red crosses). (Source [70])

Figure 2 . 8 :

 28 Figure 2.8: The left and right figures show the inferred interaction networks for the small ribosomal subunit (SRU) and large ribosomal subunit (LRU).The edge colors are assigned as follows: Green for true prediction. Red for false prediction (there is no such interaction in the experimental PPI networks). Gray for the interaction that is determined by experiment but not found in the prediction. (Source[START_REF] Feinauer | Inter-protein sequence co-evolution predicts known physical interactions in bacterial ribosomes and the trp operon[END_REF])

Figure 2 . 9 :

 29 Figure 2.9: (A). Architecture Boltzmann machine (BM), edges, and nodes are denoted as coupling interactions {J ij } and amino-acid variables {A i }. (B). Sparse BM, the architecture is represented by the same convention in Fig. A, redundant couplings interactions are diluted (see Chap. 3) (C). Restricted BM (RBM) or low-rank model, the squares in the green squares are represented as "hidden variables", which effectively induce interaction among amino-acid variables (see Chap. 4). (D). Variational Autoencoder (VAE), it is a generative model constructed by a multilayer neural network (see Chap. 6).

FIG. 1 .

 1 FIG. 1. Fitting and generative quality -Pearson correlation coefficient between model and data frequencies as a function of the model density. The one-site frequencies fi(a) are directly fitted. The two-site connected correlations cij(a, b) are fully fitted by the densest model, while only a fraction of them are fitted for the sparse models at d < 1. The three-site connected correlations c ijk (a, b, c) are never fitted.The generative performance of the model is essentially unchanged down to a density of 10%, and slowly decays for even sparser couplings. However, even down to d = 1%, the Pearson coefficients remain at remarkably high values above 95% for the two-site correlations, and above 84% for the three-site correlations.

FIG. 2 .

 2 FIG. 2. Contact prediction -Positive predictive values (PPV) for several model densities, i.e. the fraction of true positives among the highest-ranking k pairs (i, j) of sites, when ordered by decreasing F APC ij .Even the most sparse model, with only 1.6% of couplings, shows an excellent performance at contact prediction. The curve for plmDCA, a standard DCA approach for contact prediction, is shown for reference and gives comparable results.

FIG. 3 .

 3 FIG. 3.Coupling distributions -Distribution of couplings corresponding to true contacts (top) and to noncontacts (bottom) in the three-dimensional protein fold, for the initial PM with density dmax = 91% and a sparser model having density d = 7% associated with a reasonably accurate contact prediction.

FIG. 5 .

 5 FIG. 5. Single and double mutations -Spearman correlation between the experimental fitnesses and the model predictions as a function of the model density, both for single and double mutants of the PABP, a member of the PF00076 family. The dashed lines show the same correlations for a profile model (i.e. d = 0) as a reference.

  A3) which coincides with (A2) when the model is well converged and p ij (a, b) = f ij (a, b). Note that the qualitative form of D ab ij and D ab ij as a function of f ij (a, b) and J ij (a, b) is very similar, and D ab ij is a monotonous function of D ab ij . Using D ab ij or D ab ij in the decimation procedure thus leads to fully equivalent results.

FIG. 7 .

 7 FIG. 7. Pearson correlation coefficients between the three chosen metrics (first moments, two-site and three-site connected correlations) of the data and the sparse models as a function of the density. Each panel (a), (b), (c) and (d) is associated with a different family, respectively, PF00014, PF00072, PF00595, PF13354.

FIG. 8 .

 8 FIG.8. Positive predictive value (PPV) curve for the other protein families associated with the contact prediction of several sparse models, from yellow to black lines. As a comparison we show the PPV curve (red line) obtained by the state-of-the-art method for this task, plmDCA.

)FIG. 9 .

 9 FIG. 9. Distribution of the couplings associated with residues physically in contact (labeled '1' histograms) and with residues not in contact (labeled as '2' histograms) for two different densities. Panels (a), (b), (c) and (d) refer to PF00014, PF00072, PF00595 and PF13354 respectively.

FIG. 10 . 8 FIG. 11 .

 10811 FIG. 10. Heat capacity as a function of the temperature T for the other protein families PF00014, PF00072, PF00595 and PF13354 in panels (a), (b), (c) and (d) respectively.

FIG. 12 .

 12 FIG. 12. Left panels: Pearson correlation coefficients between (a) the first moments, (b) the two-and (c) three-site connected correlations of each model (varying the density) compared to the empirical data. The lines are colored according to the metrics used within the decimation procedure: the symmetric Kullback-Leibler distance, the strength of the couplings or the two-site frequencies. Right panels: same plots, but varying the initial condition of the dense model learning. All data are for the PF00076 family.

)

  At convergence, all couplings such that |f ij (a, b)p ij (a, b)| < γ thus have zero gradient and are considered as decimated. In this way one can obtain PMs of different density d by tuning γ.

FIG. 13 .

 13 FIG. 13. Positive predictive value for each decimation procedure (top) and for each initial condition (bottom) for the PF00076 family.

FIG. 15 .

 15 FIG. 15. Cumulative density function of the logarithms of the two-site frequencies associated with the decimated couplings for the sparse models having densities d = 90.7%, d = 69.9%, d = 49.7%, d = 12.2% and d = 3.2%. The data refers to the PF00076 and the decimation is performed according to the standard protocol described in the main text.

FIG. 16 .

 16 FIG. 16. Pearson correlation coefficients for the online learning (blue line), for K = 10, 20, 40, compared to the converged run (red line) as a function of the density, for the one-site frequencies (a), two-site (b) and three-site (c) connected correlations.

FIG. 17 .

 17 FIG. 17. Comparison between the Positive Predictive Value (PPV) curve obtained for the online (for K = 10, 20, 40) and converged runs at two different densities, 72% and 3.2%. All data are for the PF00076 family.

  D1) where X and Y are ensembles of the generic statistical variables x and y, D(x, y) is a certain distance defined for the sequences x and y. The metric D Y (x) computes the minimum distance of the sequence x reached when compared to each of the possible sequences in the ensemble Y ; the quantity D XY is instead the average value of D Y (x) over the ensemble of X. In our problem, we choose D(x, y) as the Hamming distance between sequence x and sequence y and the ensembles X and Y are respectively t (the training set) and s, the synthetic sequences generated from the sparse Boltzmann machines.

Fig. 18 ,

 18 we show the average distances D ts , D st and D ss for each protein family; we do not show the D tt measure which is obviously constant for all densities, and takes values D tt (PF00076) = 0.308, D tt (PF00014) = 0.0917, D tt (PF00072) = 0.421, D tt (PF00595) = 0.295, and D tt (PF00076) = 0.445. Because of the phylogenetic relationship among sequences, the training set is composed

FIG. 18 .

 18 FIG. 18. Sequence variability for all the protein families considered in this work. We plot Dts, Dst and Dss as a function of the density using blue, orange and green lines, respectively, for PF00076 (panel a), PF00014 (b), PF00072 (c), PF00595 (d), PF13354 (e).

. 2 )Fig. 3 .

 23 Fig. 3.1.a and Fig. 3.1.b show histograms of the coupling for both contact and non-contact couplings for a dense and a sparse model. Both probability densities, p c and p n change around J ∼ 0 while diluting the couplings. Interestingly, contact density p c becomes slightly more pronounced by the coupling decimation at both tails (0 |J|).

Figure 3 . 1 :

 31 Figure 3.1: (a) Distributions of couplings for contact (red) and non-contact (blue) for a fully connected PPM with density d = 90.7% (some couplings are zeros due to the gauge choice as mentioned Sec. 2.4.2). The blue line shows cumulative JSD(J). (b) Same for a sparse model with d = 1.6%. (c) It shows the JSD(J) as a function of the density d. We used the data set of PF00076, which is also used in the article [92].

Figure 3 . 2 :

 32 Figure 3.2: (a) Contact map (left) and the APC score matrix (right) for protein doamin, RNA biding domain (Pfam ID, PF00076). In the right panel, the upper left triangle and the lower right triangle are results for sparse model (d = 1.6) and dense model (d = 90.7), respectively. (b) Heat maps of coupling matrices J ij for pairs that are close (< 3 Å) and have strong couplings (J ij (a, b) > 0.5) in the sparse model. (c) The same type of heat maps but for non-contact pairs (> 11 Å). There are a few strong coupling entries in non-contact coupling matrices. However, the number of strong coupling is much smaller than the contact coupling matrices, and the difference is more pronounced in the sparse model. 86

Selection of sequence motifs

  and generative Hopfield-Potts models for protein families Kai Shimagaki and Martin Weigt Sorbonne Université, CNRS, Institut de Biologie Paris-Seine, Laboratoire de Biologie Computationnelle et Quantitative-LCQB, Paris, France (Received 28 May 2019; published 19 September 2019)

FIG. 1 .

 1 FIG.1. Panel (a) represents the (Hopfield-)Potts model as a statistical model for sequences A ∈ A L , typically characterized by a fully connected coupling matrix J and local fields h (not represented). The model can be transformed into a RBM by introducing Gaussian hidden variables x ∈ R p with p being the rank of J. Note the bipartite graphical structure of RBM, which causes the conditional probabilities P(A|x) and P(x|A) to factorize. Panel (b) shows a schematic of persistent contrastive divergence (PCD). Initially, the sample is initialized in the training data (the MSA of natural sequences), and then, k alternating steps of sampling from P(A|x), respectively, P(x|A)'s are performed. Parameters are updated after these k sampling steps, and sampling is continued using the updated parameters.

FIG. 2 .

 2 FIG. 2. Statistics of natural sequences (PF00072, horizontal axes) vs MCMC samples (vertical axes) of Hopfield-Potts models for values of p ∈ {5, 10, 20, 40, 80, 160} and for a full-rank Potts model inferred using BMDCA. The first column [panels (a.1)-(g.1)] shows the one-point frequencies f i (a) for all pairs (i, a) of sites and amino acids; the other two columns show the connected two-and three-point functions c i j (a, b) [panels (a.2)-(g.2)] and c i jk (a, b, c) [panels (a.3)-(g.3)]. Due to the huge number of combinations for the three-point correlations, only the 100 000 largest values (evaluated in the training MSA) are shown. The Pearson correlations and the slope of the best linear fit are inserted in each of the panels.

  FIG. 3. Panel (a) shows the positive predictive value (PPV) for contact prediction as a function of the number of predictions for various values of the pattern number p and for BMDCA. Panels (b)-(d) show, for p = 20, 160, and BMDCA, the distribution of coupling scores F APC i j . All residue pairs are grouped into contacts (red) and noncontacts (blue). The best contact predictions correspond to the positive tail of the red histogram, which becomes more pronounced when increasing p or even going to BMDCA. observables are reproduced with good accuracy, the PPV curves depend strongly on the pattern number p. Up to statistically probably insignificant exceptions, we observe a monotonous dependence on p, and none of the RBM-related curves reach the performance of the full-rank J i j matrices of BMDCA. Even large values of p where RBM have more than 30% of the parameters of the full Potts model show a drop in performance in contact prediction. Can we understand this apparent contradiction: similarly accurate reproduction of the statistics but reduced performance in contact prediction? To this end, we consider, in Figs. 3(b)-3(d), the histograms of coupling strengths F APC i j divided into two subpopulations: Values for sites i, j in contact are represented by red, and values for distant sites are represented by blue histograms. It becomes evident that the rather compact histogram of noncontacts remains almost invariable with p (even if individual coupling values do change), but the histogram of contacts changes systematically: The tail of large F APC i j

Figure 5 9 FIG. 4 .

 594 FIG. 4. Likelihood contribution of the individual patterns for pattern numbers p = 20, 40, 80, and 160.letter being proportional to |ξ μ i (a)|, according to the sign of ξ μ i (a), letters are represented above or below the zero line. The alignment gap is represented as a minus sign in an oval shape, which allows to represent its size in the current pattern.Patterns are very distributed, both in terms of the sites and in the amino acids with relatively large entries ξ i (a). This makes a direct interpretation of patterns without prior knowledge rather complicated. The distributed nature of patterns explains also why they are not optimal in defining localized contact predictions. Rather than identifying contacting residue pairs, the patterns define larger groups of sites, which are connected via a dense network of comparable couplings. However, as we will see in the next section, the sites of large entries in a pattern define functional regions of proteins, which are important in subensembles of proteins of strong (positive or negative) activity values along the pattern under consideration. In particular, we will show that the largest entries may have an interpretation connecting structure and function to sequence in protein subfamilies.The middle column [panels (a.2)-(e.2)] of Fig.5shows a histogram of pattern-specific activities of single sequences, i.e., of

Figures ( 5

 5 Figures (5.a.3)-(5.e.3) prove the importance of individual patterns for the inferred model. The panels show the two-point correlations c i j (a, b) of the natural data (horizontal axis) vs the one of samples drawn from the distributions P -μ (A), introduced in Eq. (22) as Hopfield-Potts models of p -1 patterns with pattern ξ μ removed (vertical axis). The coherence of the correlations is strongly reduced when compared to the full model, which was shown in Fig.2: Removal even of a single pattern has a strong global impact on the model statistics.

10 FIG. 5 .

 105 FIG. 5. The five patterns of highest likelihood contribution for p = 20, the eighth ranking is added since used later in the text. The left panels (a.1)-(f.1) show the patterns in logo representation, the letter size is given by the corresponding element ξ i (a). The middle panels (a.1)-(f.2) show the distribution of the activities, i.e., the projections of sequences onto the patterns. The blue histogram contains the natural sequences from the training MSA, and the red histogram contains sequences sampled by MCMC from the Hopfield-Potts model. The right-hand side [panels (a.3)-(f.3)] shows the connected two-point correlations of the natural data (horizontal axis) vs data sampled from P -μ (A), i.e., a Hopfield-Potts model with one pattern removed. Strong deviations from the diagonal are evident. DNA-binding domains with the response-regulator domain in the same protein. The different DNA-binding domains are indicative for distinct homodimer structures assumed by the active transcription factors; DCA run on the sub-MS identifies their specific subfamily interfaces [39].(3) Phosphorelays are similar to TCS but consist of more complex multicomponent signaling pathways. In these systems, found in bacteria and plants, response-regulator domains are typically fused to the histidine-sensor kinases. They do not act as transcription factors but transduct a signal to a phosphotransferase, which finally activates a down-stream transcription factor of the same architecture mentioned in the last paragraph. We identified a class of response-regulator domains, which are fused to a histidine kinase domain. In terms

1 )

 1 FIG. 6. Patterns with multimodal activity distributions for the set of all MSA sequences can be used to cluster sequences. The rows show combinations of patterns 6-13 [panels (a.1)-(f.1)], 6-14 [panels (a.2)-(f.2)], and 13 to 14 [panels (a.3)-(f.3)]. Each sequence corresponds to a density-colored dot. A strongly clustered structure is clearly visible. When dividing the full MSA into functional subclasses, we can relate clusters to subclasses and, thus, patterns to biological function.

  ) The GerE class [panels (d.1)-(d.3)] of Fig. 6 differs in activities in the direction of the first pattern, only GerE-class proteins have positive, and all others have negative activities. Dominant positive entries are found in regions 5-15 and 100-105, again identifying the homodimerization interface, cf. Ref. [39]. (4) The Sigma54 class [panels (e.1)-(e.

FIG. 7 .

 7 FIG. 7. The same as Fig. 2 but for the protein family PF00014.

FIG. 9 . 1 FIG. 10 .

 9110 FIG.8. The same as Fig.3but for the protein family PF00014.

  FIG.11. The same as Fig.3but for the protein family PF00076.

FIG. 12 .

 12 FIG.12. The same as Fig.5but for the protein family PF00076.

Figures 10 -FIG. 13 .

 1013  display the major results for the PF00076 protein family. PPV curves are calculated using PDB ID 2x1a[49].

FIG. 14 .

 14 FIG. 14. The upper two panels show the statistics (two-point connected correlations) of the training data (PF00014) against an i.i.d. MCMC sample extracted from the inferred model and the PCD sample used for inference. As in the CD case, the MCMC sample shows larger deviations from the empirical observations than the PCD sample, but correlations appear overestimated and contrast in the PCD plot is sufficient to drive further evolution of parameters. To understand these observations, we have selected those (i, j, a, b)'s with substantial deviations, cf. the inset in the first panel and analyzed their location in the protein (first panel) and their amino acid composition (second panel, amino acids in alphabetical order of one-letter code [-, A, C, . . . ,Y ]), densities are represented via heat map plots. Locations at the extremities in the sequences and in gap-gap correlations emerge clearly.

16 FIG. 15 .

 1615 FIG. 15. Regularization dependence for CD inference, empirical two-point connected correlations (PF00014) are plotted against those estimated from the model using an i.i.d. MCMC sample. The regularization strength is varied over almost two orders of magnitude with η 0 = α 0 L/qM and α 0 = 0.0525, going from a zone of overfitting to one of over-regularization. Results are shown for various values of p, illustrating a strong p dependence of the optimal coupling strength.

16 FIG. 16 .

 1616 FIG. 16. Regularization dependence for PCD inference, empirical two-point connected correlations (PF00014) are plotted against those estimated from the model using an i.i.d. MCMC sample. The regularization strength is varied over almost two orders of magnitude with η 0 = α 0 L/qM and α 0 = 0.0525; results are shown for various values of p. We find a strong robustness of results with respect to regularization.

Fig. 4 .

 4 Fig. 4.1.a shows typical attractive and repulsive patterns. These are the second most significant attractive pattern (top) and the most significant repulsive pattern (bottom).

Figure 4 . 2 :

 42 Figure 4.2: (a) Overlap between RBM patterns (P = 2) and attractive HP patterns (P = 400), | ξ µ , v ν |. mfHP patterns are sorted in descending order according to the eigenvalues (the likelihood contribution decrease in the same manner). The red lines in each panel are an overlap value between the RBM patterns, | ξ 1 , ξ 2 |. The second and third mfHP patterns show significant correlations with RBM patterns. (b) The same types of figures as (a) but for repulsive patterns. None of the repulsive patterns show similarity with the RBM patterns. (c) Overlap between between local field h and the attractive mfHP patterns. The first attractive mfHP pattern shows substantial similarity with the local field, which also associate with the single site frequency f i (a) (See Appendix B.2).

  The similarities between RBM patterns are shown in Appendix D. None of the RBM pattern pairs show an overlap greater than 0.2. Even for pairs of 0.1 or more, the pair is only about 15% Fig. D.1 and Fig. D.2.

Figure 4 . 3 :

 43 Figure 4.3: In contrast to the case of RBM with P = 2, each RBM pattern overlaps with multiple HP patterns. The overlap greater than around 0.1 (below of which corresponds to estimation noise) is across the top around ten patterns. The RBM patterns with P = 16 are expressed by superimposing multiple mfHP patterns.

Figure 4 . 4 :

 44 Figure 4.4: Overlap values between the attractive mfHP patterns and RBM patterns using the mfHP patterns as initial states. In the order of the upper left to lower right panel, it corresponds to the attractive mfHP patterns with the rank according to the likelihood contribution. The left, on the top to the bottom: 1st to 8th likelihood contribution. The right, on the top to the bottom: 9th to 16th.

Fig. 4 .

 4 Fig. 4.5.b shows the same types of figure with Fig. 4.5.a, but with many mfHP attractive patterns (P = 16). Surprisingly, the Pearson coefficients are much smaller values than the cases of P = 2. One of the possible reasons is that the optimal scale parameter might be different for each pattern. It may depend on the eigenvalues. According to the likelihood contributions Fig.4.1.b, the second and third eigenvalues are close. Therefore, scaling with a single parameter can achieve relatively good statistics when P = 2. However, the range of the eigenvalues up to the top 17th change substantially (λ µ = 0.3 -12). Therefore it may need to introduce scale parameters for each pattern when P = 16.

Figure 4 . 5 :

 45 Figure 4.5: (a) Pearson coefficients of two-point connected correlation C ij (a, b), computed from the natural MSA and samples from the RBM with mfHP attractive patterns for as a function of a parameter 1/T (represented as blue circles and labeled as RBM(ξ HP )). As comparisons, we also included the Pearson values using sequences generated from the RBM with P = 2 RBM patterns, which are represented as red lines (denoted as RBM(ξ)). (b) the same types of plot with the (a), but Pearson values using sequences generated from the RBM with the P = 16 mfHP patterns (Pearson values using RBM with P = 16 RBM patterns are typically greater than 0.9).

Figure 4 . 6 :

 46 Figure 4.6: Schematic of two protein subfamilies of RR protein domain, those are distinguished according to the adjoining protein domains. Class RR-A protein family, which adjacents to domain A. Class RR-B protein family adjacents to domain B.

Figure 4 . 7 :

 47 Figure 4.7: (a) Projection of the protein sequences to the PCA space. Each color represents different protein subfamilies (cf. Table 1). we can see three distinct regions in the PCA space. (b) Projection using the RBM patterns (initial states of learning were random patterns). (c) Projection using the attractive mfHP patterns. (d) Projection using the RBM patterns (initial states were the mfHP patterns used in (c)).

Figure 4 . 8 :

 48 Figure 4.8: (a) Heat capacity C(T ) of the RBMs using a protein family data, the RNA recognition motif domain (Pfam ID PF00076) as the training data. We used P ∈ {1, 2, 4, 8, 16} patterns cases. There are the peaks below T = 1 and as P increase the peak becomes more pronounced that means the statistics of the RBMs become more sensitive against a slight perturbation on the parameters. (b) Heat capacity of the RBMs with randomized patterns (ξ rand ). The means µ rand were estimated as (-0.234, -0.152, -0.0291, 0.00391, -0.00509) for each case of P → (1, 2, 4, 8, 16). Similarly, the standard deviations σ rand were estimated as (0.423, 0.438, 0.4331, 0.406, 0.369).

Figure 4 . 9 :

 49 Figure 4.9: (a) Pearson coefficient of two-point connected correlations comparing the protein sequences (Pfam ID PF00076) and sequences generated from the RBMs. As P increases, the Pearson value increases in all temperature regions T and the P dependence becomes more pronounced around T ∼ 0.95. (b) Similarly, it shows the slope of the comparison of the twopoint correlations between natural sequences and sequences generated from RBMs. Qualitatively similar to the result of Pearson values in Fig. 4.9.c, the optimal temperature is slightly below T = 1.0.

Fig. 4 .

 4 Fig. 4.9 shows comparison of two-point correlation C ij (a, b) of training data (PF00076, the same data used in Fig. 4.8) and sequences generated from the model using Pearson (a) values and slope (b).Both Pearson value and the slope increase steadily as P increases. In the case of P = 16, it already achieved the Pearson values 0.8 (it improves further as increase the P ).We found that the temperature that achieves the highest Pearson value is below T < 1.0 (between 0.9 < T < 1.0). Considering the argument of the heat capacity, this result, therefore, indicates the presence of optimal P and T in the sense that it achieves reasonably good statistics but is less critical (low-temperature is feasible).

Figure 5 . 1 :

 51 Figure 5.1: A graphical representation of the model we propose in this section: Combining sparse couplings and low-rank couplings. Blue circles and green squares denote amino acids and hidden variables respectively. Interactions between these variables are represented as edges, where black edges are interactions defined in RBM and red edges are interactions of the sparse bmDCA.

Figure 5 . 2 :

 52 Figure 5.2: (a) An eigenvalue distribution of protein sequences covariance matrix, which shows power-law. (b)The precision of the residue contact predictions as a function of the number of removing the principle modes of data covariance matrix. These eigenmodes are sorted in descending order of the eigenvalues. It shows that removed some largest principle modes improves contact predictions. These figures are adapted from[44].

Figure 5 . 3 :

 53 Figure 5.3: (a) PPV curves based on element-wise coupling activation F E.C. . As a model to add coupling parameters, we used RBM with several different P , the number of hidden variables. There is not much difference in the results between different P . the accuracies of these results are between the PPV curve of MI (red line) and PPV curve of plmDCA (blue line), which is the best residue contact prediction from only MSAs. (b) Same as (a) but these predictions are based on block-wise coupling activation F B.C. . These PPV curves are very similar to the results in Fig. 5.3.a.

  ij

Fig. 5 .

 5 Fig. 5.4 shows PPV curves using the likelihood variation values for both element-wise F E.L.ij

Figure 5 . 4 :

 54 Figure 5.4: (c) PPV curves based on element-wise likelihood variation F E.L. . These residue-contact accuracies are systematically better than the result based on MI (shown as a red line). If P is greater than some values (P > 4), predictions based on F E.C. ij are almost the same with plmDCA results (blue line) until around 300. (d) Same as (c) but for the block-wise likelihood variation F B.L. . These show almost the same curves as the element-wise cases, but the block-wise predictions are slightly more accurate (the point the PPV curve based on plmDCA drops is almost the same point as some of the cases for F B.L. ).

Figure 6 . 1 :

 61 Figure 6.1: (a) Distribution of the three-point correlations for PF00014. We show the largest 10 5 |C ijk (a, b, c)| (red histogram), which corresponds to 0.045% of all three-point correlations (ensemble-1). We also show threepoint correlations that are associated with two-point correlations (ensemble-2) whose values are greater than θ * 2 = 0.01 (blue) and θ * 2 = 0.015 (green). The three amino-acid positions corresponding to the largest and the second-largest correlations are highlighted on the 3D structure. The largest positive three-point correlations emerge at (i, j, k) = (10, 32, 34) with states, (a, b, c) = (G,G,C), (C,G,C). Similarly, the largest negative threepoint correlations correspond with (i, j, k) = (10, 14, 34) with (a, b, c) = (G,L,C),(C,L,C). (b) The same type of figure but for PF00076. (c) The same type of figure but for PF00072.

Figure 6 . 2 :

 62 Figure 6.2: (a) Schematic representation of VAE. An encoder q φ (z|x) transforms an input sample x to a hidden variable z. A decoder p θ (x|z) transforms the hidden variable z to a reconstructed variable x.

Figure 6 . 3 :

 63 Figure 6.3: Schematic of ELBO learning. The log-likelihood function is bounded by the variational lower bound. Each time optimize variational function via φ, then optimize lower bound function via θ.

Table 2 :

 2 Table shows the parameters of the architecture of the encoder and the decoder.l indicates the ID of layer. d l indicates the number of perceptrons contained in the layer. u l (•) is a nonlinear function (activation function) used in the layer. ReLU(•) is defined as Rectified linear units function. tanh(•), is the hyperbolic tangent function. σ(•) is the Sigmoid function.

Figure 6 . 5 :

 65 Figure 6.5: (a.1) Comparison of single-site frequencies for natural sequences (horizontal) and natural sequences from bmDCA (vertical). At the bottom right corner, we show Pearson correlation values (denoted as "corr.") and the slope of a linear fitting. (a.2) Comparison of two-point connected correlations for natural sequences (horizontal) and natural sequences from bmDCA (vertical). (b.1) and (b.2) are the sames plot as (a.1) and (a.2) respectively, but using the sequences from VAE (vertical).

Figure 6 . 6 :

 66 Figure 6.6: (a.1) Comparison of {ψ u }, hidden-means between the natural sequences (horizontal) and sequences generated from the profile model (vertical). At the bottom right corner, we also show the Pearson correlation and slope of correlation. (a.2) Comparison of {ψ uw }, hidden-covariance distribution for both the natural and profile sequences. (b.1, c.1) Same type of plots as (a.1) but comparing bmDCA sequences and VAE sequences (vertical) respectively. (b.2, c.2) Same type of plots as (a.2) but comparing bmDCA sequences and VAE sequences (vertical) respectively.

Figure 6 . 7 :

 67 Figure 6.7: Projections of hidden variables in a PCA space of hidden variables of the natural sequence (sequences that were used for learning the VAE). The ensemble sequences projected to the hidden PCA space are: (a) Natural sequences (used in PCA space and learning the VAE). (b) Sequences from a profile model. (c) Sequence form a bmDCA. (d) Sequences from the VAE, which also used for providing the hidden space.

Figure 6 . 8 :

 68 Figure 6.8: Projection of training sequences to the hidden PCA space. using the non-mean Gaussian hidden prior. Show several conditions of β, β = 0.1 (a), β = 0.5 (b), and β = 0.9 (c). As beta increases, the constraint of the KLD increases, thus a stronger effect of the non-mean Gaussian hidden prior distribution. In the left column, colors are assigned using the hidden variable c, which is obtained during VAE learning (a.1, b.1, and c.1). In the right column, on the other hand, colors are assigned using the true labels, therefore classifying based on the information of protein subfamilies (a.2, b.2, and c.2).

  )C ii (a, b)ξ µ i (b) , (C.2)whereC is covariance matrix C ij (a, b) = f ij (a, b)f i (a)f j (b).Since Eq. C.2 is a convex function of the pattern from the equation, the maximum likelihood estimation solution can be obtained easily. There are two types of solutions, depending on the eigenvalues of the Pearson covariance matrix. Here, Pearson covariance matrix is defined as follows,Γ ij (a, b) = C ij (a, b) f i (a)f j (b) . (C.3)Suppose we can easily get eigenvalues and eigenvectors as follows, Γv µ = λ µ v µ . (C.4)

Figure D. 1 :

 1 Figure D.1: Absolute value of inner products among RBM patterns, sorted but their value, using L2 (a) or L1 (b) regularization. The self overlaps are excluded and assumed to be zero.

Figure D. 2 :

 2 Figure D.2: Same data as in Fig. D.1, but shown as a function of sorted ranks. The blue and green markers correspond to the absolute overlap values for L2 and L1 regularization, respectively. The red line is the overlap values of RBM patterns for the P = 2 model.

4 .

 4 Repeat process 1. and 2. t = k times. If we assume RBM as an example of a latent variable models, p(v, h|w, a, b) ∝ e -E(v,h|w,a,b))

  δw ij = v i h j p(h|v;w,a,b)p data (v) v i h j p(h|v;w,a,b)q k (v|w,a,b) δa i = v i p data (v) v i q k (v|w,a,b) δb j = h j p(h|v;w,a,b)p data (v) h j p(h|v;w,a,b)q k (v|w,a,b) (E.7)

p

  statistically rigorously:Z(θ (t+1) ) Z(θ (t) ) = p * (x|θ (t+1) )dx Z(θ (t) ) = p * (x|θ (t+1) ) p * (x|θ (t) ) p(x|θ (t) ) * (x m |θ (t+1) ) p * (x m |θ (t) ),(E.8)where p * (x|θ) denotes the unnormalized probability distribution of the model, and • p(x|θ (t)

Fig. F. 1

 1 Fig. F.1 shows PPV curves for PF13354. Similarly, Fig. F.2 shows PPV curves for PF00076. Likelihood-variation-based methods F E.L. and F B.L. show accurate results for both families. Especially in Fig. F.1.c and Fig. F.1.d, the PPV values are higher than the result of plmDCA above around 200 contact predictions.

Figure F. 1 :

 1 Figure F.1: (a) PPV curves based on the element-wise coupling activation F E.C. . (b) The same types of plots as (a) but for the block-wise coupling activation F B.C. . (c) PPV curves based on the element-wise likelihood variation F E.L. . Fig. F.1.c and Fig. F.1.d show a clear dependency on the number of hidden variables in RBM P . Increasing P improves these PPV curves, with almost the same accuracy as contact predictions based on plmDCA. Especially, when P ≥ 8, above a certain number of predictions, they achieve better accuracy than plmDCA accuracy. (d) The same types of plots as (d) but for the block-wise likelihood variation F B.L. .

Figure F. 2 :

 2 Figure F.2: PPV curves for F E.C. (a), F B.C. (b), F E.L. (c), F B.L. (d). Since the MI result (corresponding to P = 0 in the case of F B.L.) itself has already been predicted as accurately as plmDCA, the effect of the phylogenetic effect on this protein family is probably not so substantial.

Figure F. 3 :

 3 Figure F.3: (a) PPV curves based on the coupling matrix constructed by mfHP patterns (the same definition of couplings shown in Eq. 4.5). The accuracy of contact prediction increase as increases P, which is expected result and is coherent with Ref.[START_REF] Shimagaki | Selection of sequence motifs and generative hopfield-potts models for protein families[END_REF]. The same types of plots as (a) are shown for PF00076 (b) and for PF13354 (c). For all of these families, the accuracy of contact predictions improves as increase P .

Fig. G. 1 .

 1 Fig. G.1.b shows an output sequence from VAE (cf. Sec. 6.2.3) using the filter. Fig. G.1.c shows the same output sequence without the filter.

Figure G. 1 :

 1 Figure G.1: (a) One-hot encoded matrix (q × L, q=21, L=112) of a protein sequence (PF00072). White elements and black elements are assigned as 1 and 0 respectively. (b) One-hot encoded matrix generated from the VAE using the argument-max filter. (c) One-hot encoded matrix generated from the VAE.

TABLE I .

 I Once the training is complete, for the final set of parameters of the Potts Model, we need to generate a new We show here the Pfam identifier, the name of the protein domain, the length, the number of sequences and the effective number of sequences for the families analyzed in our work.

	Identifier	PF00014	PF00072	PF00076	PF00595	PF13354
	Protein domain Kunitz domain Response regulator receiver domain RNA recognition motif PDZ domain Beta-lactamase2
	L	53	112	70	82	202
	M	13600	823798	137605	36690	7515
	M eff	4364	229585	27785	3961	7454
				4. Sampling protocol

TABLE II

 II 3 MC sweeps for the dense model. Because our training is

	Identifier PF00014 PF00072 PF00076 PF00595 PF13354
	MMC	30000	30000	30000	30000	30000
	Twait	60	80	60	90	100
	Teq	10000	50000	30000	50000	50000
	N chain	100	100	100	100	300

. We report here the details of the MC sampling performed to evaluate the model statistics.

  Where, • p data and • p model are expected values with the probability distribution of the data p data and the model p model . Eq. E.2 is the fixed point equation of MLE learning to be solved. The corresponding fixed point equation using CD-based learning can be written similarly as follows,

	1:N ) ∂θ	=	-∂E(x|θ) ∂θ	p data	-	-∂E(x|θ) ∂θ	p model	.	(E.2)
	∂l(x 1:N ) ∂θ	=	-∂E(x|θ) ∂θ	p data	-	-∂E(x|θ) ∂θ

q (k) (x|θ;x 1:N )

Other commonly used methods are NMR spectroscopy, and with impressive recent progress, cryo-electron microscopy.

Random sequences have ∼ 5% sequence identity.

States are categorical as known q-state Potts spins

It is a protein that regulates the rate of transcription of genetic information from DNA to mRNA

Normally, the pseudocount method is required.

Obtaining a profile HMM from an unaligned sequence is an open issue as it requires simultaneous optimization of the model and multiple alignments.

In recent years, inference problems to predict effective Hamilton has attracted significant attention in material informatics[35, 36].

This problem may more severe if the underlying interaction networks of residues have many loops.

The very beginning attempt of structure information which evolved in to DCA was[40], this method used MC to estimate the partition function but different method from bmDCA.

This is a peculiarity of the pseudo-likelihood function, not plmDCA.

Technical points to derive the Eq. 2.32 is the estimation of the partition function based on mean-field approximation. The outline of the derivation is shown as follows: First, applying Hubbard-Stratonovich transformation on the partition function. Second, apply the saddle point expansion of the joint distribution and approximate it as a multivariate Gaussian distribution considering the mean and covariance. Last, integrate the Gaussian distribution while imposing a particular gauge choice.

The sum of the number of true positives and the number of false positives corresponds to the total predicted number.

The JSD is definable for all domains R even if the support of two distributions namely p A and p B are different, therefore it meets our study.

It is suggestive that using the L1 norm as the Frobenius norm may give better contact predictions than in the case of the L2 norm.

The fixed point equation for a paramter v µ ij (a) becomes a simple modification of the case of PPM, bfij(a, b)v µ ij (b)b pij(a, b)v µ ij (b) = 0 ,where pij(a, b) is a two-point frequency of the assumed model.

Kunitz-bovine pancreatic trypsin inhibitor domain PF00014

RNA recognition motif PF00076

For this purpose, we need to look at the collective activity of hidden variables, x µ x ν , where • means the average of hidden space gives the training data.

One easily finds that this function becomes ∆l(J) → 0 as f → 0, 1 for arbitrary J (suppose f ∼ p).

The validity of this formulation could be that, as in the derivation of the likelihood function, adding two couplings such as Jij(a, b) and Jij(c, d) gives an arithmetic sum if the entropic term is negligible in Eq. 5.5.

By definition, they are non-negative values, ∀i, j .

We can find a similar idea in the Expectation-Maximization algorithm.

As we saw the derivation of the ELBO function, however, there are no restrictions on the functional form of p φ (z|x) (we will discuss this point later in Sec. 6.3.4).

Here, manual annotation means reviews of experimentally proven facts or computationally predicted features.

Note also that need to marginalize the hidden variables to be the objective function in practice.

The original β-VAE in[START_REF] Higgins | beta-vae: Learning basic visual concepts with a constrained variational framework[END_REF] has β dependency only on the second term in eq.(G.6) and the β > 0, but essentially the same.

Acknowledgements ACKNOWLEDGMENTS We would like to thank Matteo Bisardi, Simona Cocco, Yaakov Kleeorin, Rémi Monasson, Rama Ranghanatan, Olivier Rivoire and Jeanne Trinquier for discussions related to this work. We acknowledge funding by the EU H2020 research and innovation programme MSCA-RISE-2016 (grant 734439 InferNet, to MW), by the Simons Foundation (#454955, to FZ) and by the Honjo International Scholarship Foundation (PhD grant to KS). P.B.-C., A.P.M. and K.S. contributed equally to this work. Author contributions: P.B.-C. and M.W. designed the research; P.B.-C., A.P.M., K.S., and F.Z. performed the research; A.P.M. and K.S. analyzed the data; A.P.M., K.S., M.W., and F.Z. wrote the paper. ACKNOWLEDGMENTS We are particularly grateful to P. Barrat-Charlaix, G. Croce, C. Lucibello, A.-P. Muntoni, A. Pagnani, E. Sarti, J. Tubiana, and F. Zamponi for numerous discussions and assistance with data. We acknowledge funding by the EU H2020 Research and Innovation Programme MSCA-RISE-2016 under Grant Agreement No. 734439 InferNet. K.S. acknowledges an Erasmus Mundus TEAM (TEAM-Technologies for information and communication, Europe-East Asia Mobilities) scholarship of the European Commission in 2017 and 2018 and a doctoral scholarship of the Honjo International Scholarship Foundation since 2018.

The first Appendix is dedicated to other protein families. As discussed in the main text, we have analyzed three distinct families and discussed only one in full detail in the main text. Here, we present the major results-generative properties, contact prediction, and selected collective variables (patterns)-for two more families. These results show the general applicability of our approach beyond the specific response-regulator family used in the main text.

p given an observation of f (Appendix F.1).

Note that the above arguments can be held whenever an adding parameter is introduced individually. The same result can obtain by introducing other types of parameters such as external field h i (a) and many-body interactions parameters (e.g., J ijk (a, b, c)).

Block-wise couplings activation

Similar to the previous discussion, we can estimate the likelihood change due to adding an entire coupling matrix J ij ∈ R q×q .

H (A) = H(A) -J ij (A i , A j ) , ∀A = (A 1 , . . . , A L ) ∈ {1:q} L .

(5.10) Note that Eq. 5.10 is not an elementwise coupling addition on Eq. 5.3 but blockwise. Accordingly, we get the following relation for the likelihood variation:

Optimize the block-wise likelihood variation -Here we determine the properties of the likelihood variation in the case of block-wise likelihood variations. In the following analysis we neglect site indices and represent states (ijab

Here the Hessian matrix of the likelihood variation is,

where r µ = exp(Jµ)pµ ν exp(Jν )pν , hence ν r ν = 1 2 .

2 Thus rν can be regarded as a probability measure.

Note that the inner-product for an arbitrary vector a and Ha is

where a * r = µ a µ r µ . Thus, the block-wise ∆l is a convex function of J ∈ R q×q and there is only one maximum.

Therefore, we can find the coupling matrix that realizes the maximum, by solving the saddle point equation,

Eq. 5.14 can be easily solved and we get a solution,

with,

Note that when the frequency of model p ij (a, b) is given by the profile model i.e., p ij (a, b) = f i (a)f j (b), the right most side of Eq. 5.16 becomes the mutual information.

Applications

This section discusses residue-residue contact predictions using the likelihoodvariation methods for both element-wise and block-wise. Assume multiple RBMs with different hidden variables P as generative models to be added coupling parameters. The RBM patterns are closely associated with the correlations associated with the principal mode ( [START_REF] Cocco | High-dimensional inference with the generalized hopfield model: Principal component analysis and corrections[END_REF][START_REF] Cocco | From principal component to direct coupling analysis of coevolution in proteins: Loweigenvalue modes are needed for structure prediction[END_REF] and Sec. 4.3.1), which is significantly influenced by phylogenies [44]. Therefore, it is expected that the residue-residue contact prediction can be improved when correlations of RBM sequences efficiently reproduce non-contact correlations.

Here, increasing the number of hidden variables P is associated with increasing the number of principal modes to be removed from a covariance matrix for residue contact predictions (see Fig. 5.2). The effect of removing the patterns is nontrivial.

The learning protocol is the following. First, we initialize the RBM parameters based on the mfHP patterns as we discussed in Sec. 4.3.1. Second, we learn the RBMs using the standard learning algorithm (PCDbased learning algorithm with Gaussian hidden variables, see Appendix E.2). Lastly, we apply the likelihood-variation methods and obtain the coupling activation values J * ij (a, b), J * ij and the maximized likelihood variations ∆l(J * ij (a, b)), ∆l(J * ij ) for both elementwise and blockwise cases.

For obtaining the scoring function for contact predictions, we apply different strategies for both element/block-wise and coupling/likelihoodvariation methods:

• ∆l * ij (a, b) ∈ R 1 : For element-wise optimal likelihood variation, we use the sum for the overall state, F E.L.

For block-wise coupling activation, we take its Frobenius norm,

• ∆l * ij ∈ R 1 : For Block wise optimal likelihood variation, we use that value directly,

Fig. 5.3 shows the contact predictions i.e., positive predictive value (PPV) curves using the coupling activation values for both element-and block-wise cases. For RBMs, we tested multiple different number of hidden variables, P ∈ {0, 1, 2, 16, 32}. Note that a P = 0 RBM is equivalent to the profile model. For comparison, we also included results using MI and plmDCA. As for the MSA to predict residue contacts, we use the data set of PF00072 employed in Ref. [START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF].

quencies).

The more important observation here is that bmDCA produces statistics almost as good as the VAE in hidden space. As supplementary quantities to assess the statistics, we also examine the distributions of hidden variables using PCA. The PCA space is constructed by the ensemble of hidden variables for the the natural sequences that are used for learning.

In Fig. 6.7, we show the projections of hidden variables to PCA space. The distribution of the profile model (b), does not have a cluster structure and is different from the case of the natural sequences (a). On the contrary, the distributions of bmDCA (c) and VAE (d) can reproduce the similar distribution of the natural sequence in the hidden space. same as in Sec. 4.3.2 (see also Table 1 in Sec. 4.3.2).

In this experiment, the architecture of the VAE is the same as Table 2 in Sec. 6.2.3. As mentioned earlier, we assigned initial states of mean vectors μt=0 c , c ∈ 1:K, from a standard Gaussian distribution. For the initial class label, we assigned the true labels (correspond with the protein subfamilies) to make it easier to compare with the true labels. However, the classification properties, such as the cluster structure of hidden variables and assignment of the label did not change much by the initial state of the labels. Fig. 6.8 shows results of sequence projections in to hidden PCA spaces, which are constructed form the hidden variables of training data (including all of the protein subfamilies). To demonstrate the effect of the constraint on the hidden prior, we examined several, β ∈ {0.9, 0.5, 0.1}. Sequences are colored according to their VAE-estimated label (left) or the true label (right). Note that the sequences of the PF00072 domains themselves were not used for the labeling but for the other adjacent domains in the same protein.

As we increase β, i.e., we impose stronger constraints on the conditional posterior for hidden variables, the hidden spaces show more and more complex structures, and clusters become more distinctive. Some subfamilies are correctly classified in any β, but some subfamilies always seem to be difficult to classify. Probably these protein subfamilies are inherently very similar.

When β = 0.1 (a.1 and a.2), the constraint on the hidden space is very weak by this construction. Therefore the distribution of hidden variables is similar to that pre-learned with standard VAEs, cf., Fig. 6.7. As β increases, the heterogeneity of the hidden-variable distribution increases.

On the other hand, when the hidden space is imposed greater constraint, i.e., β = 0.9 (c.1 and c.2), distribution of the hidden variable projections becomes more inhomogeneous and demonstrates characteristic structures. In the case of the labels that are estimated by the VAE (c.1), label (color) assignments differ from projected area to area; it shows the one-to-one correspondence between label types and regions. However, in the case of the true labels (c. 2), some regions involved hidden variables that are assigned different labels, i.e., those labels are indistinguishable in the hidden space.

Interestingly, the subfamilies projected onto the characteristic regions are the same as the low dimensional analysis based on RPM and mfHP, Appendix A

Databases and Data Format A.1 Database

Pfam -Pfam contains multiple sequence alignments (MSAs) of protein domain families and their hidden Markov models (HMMs). E-value (expect value) is also calculated according to an MSA construction [START_REF] Bateman | The pfam protein families database[END_REF]. Pfam consists of the following six types of sequences: family, domain, repeat, motif, coiled-coil, and disordered.

To make a large MSA, it needs a curated small number of sequences pre-aligned (the number of sequences is around 100). Based on an MSA that contains a small number of alignments, construct a HMM to search reference protein sequences from external databases such as UniProt. The full alignments can improve the HMMs further by reselecting the seed alignments accordingly, then repeating the same process using the new HMMs. This iterative construction of HMMs and seed alignments are repeated until there is no new entry of sequences from the database.

UniProt -Universal Protein resource (UniProt) [20] is a comprehensive resource of protein sequences and annotated data such as types of proteins and taxonomy. It has been managed by Swiss Institute of Bioinformatics (SIB), European Bioinformatics Institute (EBI), and Protein Information Resource (PIR). UniProt contains the following databases; 1. UniProtKB: a central hub for collections of functional protein information. These entries come from coding sequences (CDS) that are

A.2 Data Format

In general, protein sequence files we use are in FASTA format. In the FASTA format, sequences are represented as a series of lines, each of which is no longer than 80 characters. The first line of each protein sequence in a FASTA file, which is also called the header, starts as a later ">", then follows a unique name and/or a unique identifier of the sequence, and there may be additional information. NCBI provides identifiers for protein sequences. The after first lines are the series consist characters that correspond with one of 20 amino acids or gap "-" . Note that the character " Z " means glutamic acids, "X" means any amino acids, and "*" means translation stop.

Appendix B

Other modeling techniques

B.1 Sequence gap filtering

Amino acid sequences tend to have consecutive gap symbols at the beginning and the end of sequences typically. These gaps cause some learning problems. Especially quantities that directly depend on gaps cause slow learning convergences and unstable learnings [START_REF] Shimagaki | Selection of sequence motifs and generative hopfield-potts models for protein families[END_REF]. A method to remedy this problem is to exclude sequences that contain too many consecutive gaps. We typically keep sequences that have less than six consecutive gaps.

B.2 Initialization of model parameters

The reasonable choice of initial states for the local-field parameters {h i (a)} in Eq. 2.8 are solutions of the profile model :

Thus, we can assume the local-field parameters as following

Empirically, solutions of {h i (a)} in bmDCA in Eq. 2.8 do not change much after the learning when these parameters are initialized based on Eq. B.2.

Especially, efficient initializations for coupling parameters is needed because learning of {h i (a)} are much faster than {J ij (a, b)}. Furthermore, solutions of bmDCA depend heavily on initializations in practice [START_REF] Barrat-Charlaix | Understanding and improving statistical models of protein sequences[END_REF][START_REF] Barrat-Charlaix | Sparse generative modeling of proteinsequence families[END_REF], even if optimizations of pairwise Potts models, including bmDCA are convex prob-Appendix C

Hopfield-Potts model and Restricted Boltzmann machines

Here, we summarize the solutions of mean-field Hopfield-Potts (HP) patterns based on Ref. [START_REF] Cocco | Inference of hopfield-potts patterns from covariation in protein families: calculation and statistical error bars[END_REF].

The key steps for obtaining the mfHP solutions are as follows:

(1) Apply the Harvard Stratonovich transformation by introducing continuous variables to unwind interactions between categorical variables for the partition function. (2) Execute Gaussian integration, including the effects of second-order fluctuations, i.e., variance. (3) Fix the gauge to remove redundant gauge freedom due to rotational invariance in the space of the pattern (cf., Eq. 4.4), i,a

These arguments lead to the following equation of log-likelihood function that these mfHP patterns consider only non-zero eigenmodes therefore P ≤ L(q -1) .

Here we denote ṽµ as the scaled eigenvectors:

Appendix D

RBM pattern orthogonality and regularization effect

This section shows the overlap or inner products of RBM patterns with the number of hidden variables P = 16. The learning protocols are the same as in Sect. 4.3. As the data set, we used the response regulator domain (Pfam ID, PF00076). MCMC is a flexible and relatively fast method to realize probability distribution. However, most of the probability distributions we want to infer require iterative optimization of model parameters and MCMC execution each time. Repeating such a process thousands of times is computationally too demanding, even if running MCMC on a set of model parameters is relatively light.

CD methods are stochastic learning methods for exponential families that can remedy such a computational burden due to the repetitive executions of MCMC. CD-based methods are widely used as common methods for learning exponential families (e.g, Boltzmann machines, Restricted Boltzmann machines). However, there is not yet a sufficient theoretical understanding of the basic properties of learning based on CD methods. The followings are examples:

• Are solutions using CD methods the same as the MLE solutions? (it is generally accepted that CD is a different estimation method from MLE methods [START_REF] Miguel A Carreira-Perpinan | On contrastive divergence learning[END_REF]).

• Guarantee of convergence [START_REF] Sutskever | On the convergence properties of contrastive divergence[END_REF], speed for convergence, the dependence of estimation bias Etc. have not been understood.

E.1.1 Contrastive Divergence learning

Suppose there is an ensemble of statistics x 1:N = (x 1 , x 2 , ..., x N ) ∈ X N that we use as a set of training data, and our primary objective is to describe the ratio as follows:

where β k = k/K, k = 1, 2, .., K, and Z β (θ (t+1) , θ (t) ) is a normalization factor of the probability distribution p β (x|θ (t) , θ (t+1) ).

It is reported that AIS values are good measurements to evaluate learning processes and to determine the optimal learning stopping epochs [START_REF] Fischer | Empirical analysis of the divergence of gibbs sampling based learning algorithms for restricted boltzmann machines[END_REF] and it also works for protein sequence modeling [START_REF] Tubiana | Learning protein constitutive motifs from sequence data[END_REF].

Appendix F

Other remarks of likelihood-variation method F.1 Another derivation element-wise likelihood variation

Here, we show another derivation of the optimal likelihood variation ∆l(J * ij (a, b)) in Eq. 5.9.

Suppose that M × f ij (a, b) out of M sequences in a given MSA are drawn from a probability distribution, p ij (a, b) = A∈A p(A)δ A i ,a δ A j ,b , where p(A) ∝ e -H(A) .

Therefore, the binomial distribution to consider can be written as:

For simplicity, we ignored site and amino-acid indexes.

Eq. F.1 can be simplified to an exponential form with a large M limit, The encoder and decoders are represented as the neural network architectures, therefore, the objective function, ELBO is differentiable. However, problems arise when parameters are stochastics: the dependency of φ on the expectation E q φ (z|x) [•] comes via the hidden variable, which has a stochastic quantity, so it is not differentiable.

An important idea to solve this problem is to make all variables including hidden variables differentiable by separating the deterministic parts and stochastic parts [START_REF] Diederik | Adam: A method for stochastic optimization[END_REF]. Note that variables z are assumed as functions of random variables, and dependency of x and φ are absorbed in the function. Therefore, hidden variables can be written as z = z(ξ, x; φ) = µ φ (x) + Σ φ (x)ξ ξ ∼ N (ξ; 0, I) .

(G.1)

This method to separate stochastic factors and deterministic factors is called reparameterization trick [START_REF] Diederik | Auto-encoding variational bayes[END_REF]. Accordingly, the objective function