Chitra Narayanan

David N Bernard

Nicolas Doucet
email: nicolas.doucet@iaf.inrs.ca

David D Boehr

Role of Conformational Motions in Enzyme Function: Selected Methodologies and Case Studies

Keywords: enzyme catalysis, conformational dynamics, nuclear magnetic resonance, molecular dynamics simulations, HIV-1 protease, DNA polymerase β

It is now common knowledge that enzymes are mobile entities relying on complex atomic-scale dynamics and coordinated conformational events for proper ligand recognition and catalysis. However, the exact role of protein dynamics in enzyme function remains either poorly understood or difficult to interpret. This mini-review intends to reconcile biophysical observations and biological significance by first describing a number of common experimental and computational methodologies employed to characterize atomic-scale residue motions on various timescales in enzymes, and second by illustrating how the knowledge of these motions can be used to describe the functional behavior of enzymes and even act upon it. Two biologically relevant examples will be highlighted, namely the HIV-1 protease and DNA polymerase β enzyme systems.

Introduction

Proteins and enzymes are essential components of living cells. Among a variety of other functions, they act as hormones in cell signaling, protein transporters, antibodies in host defense, etc. Concomitantly, enzymes are some of the most efficient catalysts known to date, enhancing biochemical reaction rates up to 19 orders of magnitude relative to uncatalyzed This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/). reactions [START_REF] Garcia-Viloca | How enzymes work: Analysis by modern rate theory and computer simulations[END_REF][START_REF] Wolfenden | Degrees of difficulty of water-consuming reactions in the absence of enzymes[END_REF]. However, factors contributing to these large rate enhancements in enzymecatalyzed reactions remain largely uncharacterized. The original structure-function paradigm, popular for many decades, portrayed enzymes as rigid structures with shapes that facilitate substrates, ligand and/or cofactor binding. This original model evolved over time, and theories were developed that led to the now-accepted induced fit, conformational selection, and transition-state stabilization models to explain the behavior of protein-ligand recognition and catalysis in the molecular function of enzymes [START_REF] Hammes | Conformational selection or induced fit: A flux description of reaction mechanism[END_REF]. Increasing evidence suggests that proteins sample a variety of distinct conformations (or sub-states) enabled by concerted atomic-scale dynamical fluctuations occurring over a wide range of timescales and acting on the primary, secondary, tertiary, and quaternary organization of their molecular structure [START_REF] Henzler-Wildman | Dynamic personalities of proteins[END_REF]. Conformational transitions between highly and rarely populated states have been shown to play important roles in substrate recognition, binding, and product release, among others [START_REF] Watt | The mechanism of rate-limiting motions in enzyme function[END_REF][START_REF] Venkitakrishnan | Conformational changes in the active site loops of dihydrofolate reductase during the catalytic cycle[END_REF][START_REF] Boehr | Promiscuity in protein-RNA interactions: Conformational ensembles facilitate molecular recognition in the spliceosome[END_REF][START_REF] Berlow | Substrate-dependent millisecond domain motions in DNA polymerase β[END_REF]. Advances in experimental and computational methodologies continue to offer new insights into enzyme conformational motions over functionally relevant timescales [START_REF] Kay | NMR studies of protein structure and dynamics[END_REF][START_REF] Vallurupalli | Structures of invisible, excited protein states by relaxation dispersion NMR spectroscopy[END_REF][START_REF] Ramanathan | Protein conformational populations and functionally relevant substates[END_REF][START_REF] Vanwart | Exploring residue component contributions to dynamical network models of allostery[END_REF]. Interestingly, experimental [START_REF] Gagné | Conservation of flexible residue clusters among structural and functional enzyme homologues[END_REF][START_REF] Eisenmesser | Intrinsic dynamics of an enzyme underlies catalysis[END_REF], computational [START_REF] Agarwal | Protein dynamics and enzymatic catalysis: Investigating the peptidyl-prolyl cis-trans isomerization activity of cyclophilin A[END_REF][START_REF] Goodey | Allosteric regulation and catalysis emerge via a common route[END_REF], and sequencebased [START_REF] Suel | Evolutionarily conserved networks of residues mediate allosteric communication in proteins[END_REF][START_REF] Reynolds | Hot spots for allosteric regulation on protein surfaces[END_REF] approaches have also revealed functional networks of concerted residue motions distant from the active site in selected enzyme systems. Correlations between the timescale of conformational fluctuations and that of catalytic turnover have been well established in a variety of enzyme systems, including cyclophilin A [START_REF] Eisenmesser | Intrinsic dynamics of an enzyme underlies catalysis[END_REF], RNase A [START_REF] Watt | The mechanism of rate-limiting motions in enzyme function[END_REF][START_REF] Cole | Evidence for flexibility in the function of ribonuclease A[END_REF][START_REF] Gagné | Structural and functional importance of local and global conformational fluctuations in the RNase A superfamily[END_REF], triosephosphate isomerase [START_REF] Pareek | Connecting active site loop conformations and catalysis in triosephosphate isomerase: Insights from a rare variation at residue 96 in the plasmodial enzyme[END_REF], and HIV-1 protease [START_REF] Torbeev | Protein conformational dynamics in the mechanism of HIV-1 protease catalysis[END_REF], among others. Further, the rate of conformational exchange has been shown to coincide with the rate-limiting step, such as product release, in some of these systems [START_REF] Cole | Evidence for flexibility in the function of ribonuclease A[END_REF][START_REF] Gagné | Structural and functional importance of local and global conformational fluctuations in the RNase A superfamily[END_REF]. Conformational exchange between sub-states allows enzymes to sample higher energy conformations with structural and dynamical properties important for function such as ligand binding and allosteric regulation [START_REF] Ramanathan | Protein conformational populations and functionally relevant substates[END_REF][START_REF] Goodey | Allosteric regulation and catalysis emerge via a common route[END_REF][START_REF] Kar | Allostery and population shift in drug discovery[END_REF]. In addition to the millisecond conformational exchange, dynamics on faster timescales modulate the chemical environment through rearrangements in the active site, thus affecting enzyme function [START_REF] Agarwal | Enzymes: An integrated view of structure, dynamics and function[END_REF]. Taken together, a view is emerging whereby conformational fluctuations occurring over a range of timescales can affect enzyme function through conformational sampling along preferred pathways.

Over the past few years, a number of controversial statements relating 'catalysis' with 'dynamics' have been published in the literature, often giving rise to heated debates between experimentalists and theoreticians [START_REF] Kohen | Role of dynamics in enzyme catalysis: Substantial versus semantic controversies[END_REF][START_REF] Kamerlin | At the dawn of the 21st century: Is dynamics the missing link for understanding enzyme catalysis?[END_REF]. As recently outlined, these debates are often semantic in nature and can be traced back to actual definitions (and research field perceptions) of what 'protein dynamics' represents, in addition to which atomic-scale events are being observed and/or over which timescales they occur during enzyme-catalyzed reactions [START_REF] Kohen | Role of dynamics in enzyme catalysis: Substantial versus semantic controversies[END_REF]. In the current report, we consider motions affecting enzyme function in a broad sense, i.e., "any motion that, if impeded, would reduce the ability of the enzyme to function by the mechanism that it has evolved to execute" [START_REF] Nagel | A 21st century revisionist's view at a turning point in enzymology[END_REF]. As such, we take into account motions on a wide range of timescales, from the fast, local picosecond-nanosecond (ps-ns) fluctuations of individual residues to the slower microsecond-millisecond-second (μs-ms-s) global conformational exchange experienced by enzymes. Conserved motional events occurring on a broad range of timescales could be involved in many and/or a selected subset of events occurring along the catalytic cycle of an enzyme, including ligand binding, recognition, substrate discrimination, structural rearrangements leading to the transition state, product release, etc. [START_REF] Kohen | Role of dynamics in enzyme catalysis: Substantial versus semantic controversies[END_REF]. To clarify and prevent confusion, we never imply that 'motions contributing to catalysis' exclusively describe the fast femtosecond atomic motions involved in transition-state chemistry.

In the current account, we draw attention to a number of experimental and computational methodologies that have recently improved our understanding of catalytic and functional properties in two enzyme systems that closely rely on conformational dynamics for proper biological function. We do not pretend to cover the overwhelming number of theoretical approaches, methodologies, or enzymatic systems that previously illustrated the role of protein dynamics in enzyme function, but instead focus on recent reports where the combination of experimental NMR and computational techniques have emphasized the role of conformational exchange in inhibitor binding (HIV-1 protease) and fidelity in DNA repair through the selection of correct nucleotides (DNA polymerase β).

Selected Experimental and Computational Approaches for Sampling Conformational Motions in Proteins

A variety of approaches, including (but not limited to) NMR spectroscopy, X-ray diffraction, single molecule FRET, and computational simulations, have been used to probe conformational dynamics in proteins over nanosecond to millisecond timescales, offering the means to extract motions potentially relevant to biological function [START_REF] Henzler-Wildman | Dynamic personalities of proteins[END_REF][START_REF] Hanoian | Perspectives on electrostatics and conformational motions in enzyme catalysis[END_REF]. One of the most commonly used experiments to characterize local and global conformational events experienced by enzymes on the timescale of their catalytic rate k cat -typically occurring on the order of milliseconds-is the NMR Carr-Purcell-Meiboom-Gill (CPMG) relaxation dispersion experiment [START_REF] Grutsch | NMR methods to study dynamic allostery[END_REF][START_REF] Palmer | Chemical exchange in biomacromolecules: Past, present, and future[END_REF][START_REF] Kleckner | An introduction to NMR-based approaches for measuring protein dynamics[END_REF][START_REF] Loria | A trosy cpmg sequence for characterizing chemical exchange in large proteins[END_REF][START_REF] Loria | A relaxation-compensated carr-purcell-meiboom-gill sequence for characterizing chemical exchange by NMR spectroscopy[END_REF]. This methodology garnered considerable attention in recent years since motions experienced by enzymes on this physiologically relevant timescale can improve our understanding of catalysis and allostery, as the flexibility events uncovered can be analyzed in light of catalytic events. The CPMG method enables the detection of atomicscale millisecond dynamics of enzymes in solution, either in the free form or bound to biologically significant ligands/analogues that mimic intermediate states along the chemical reaction [START_REF] Gagné | Conservation of flexible residue clusters among structural and functional enzyme homologues[END_REF][START_REF] Oyen | Cofactor-mediated conformational dynamics promote product release from Escherichia Coli dihydrofolate reductase via an allosteric pathway[END_REF][START_REF] Bhabha | Keep on moving: Discovering and perturbing the conformational dynamics of enzymes[END_REF]. On this time frame, molecular flexibility may involve the rearrangement of single residues and/or entire secondary structures moving in a coordinated fashion (for reviews see [START_REF] Kay | NMR studies of protein structure and dynamics[END_REF][START_REF] Doucet | Can enzyme engineering benefit from the modulation of protein motions? Lessons learned from NMR relaxation dispersion experiments[END_REF][START_REF] Palmer | NMR probes of molecular dynamics: Overview and comparison with other techniques[END_REF][START_REF] Palmer | NMR characterization of the dynamics of biomacromolecules[END_REF]) (Figure 1). In its typical form, the method involves the characterization of an equilibrium exchange process experienced by the N-H amide bond of amino acid residues, performed by recording the transverse relaxation rate constant (R 2 ) as a function of v CPMG , the frequency with which refocusing pulses are applied in a CPMG pulse train [START_REF] Palmer | Chemical exchange in biomacromolecules: Past, present, and future[END_REF][START_REF] Manley | NMR insights into protein allostery[END_REF]. These refocusing pulses are effectively invisible to amino acid residues that do not undergo conformational exchange in the protein, but they add an additional contribution (R ex ) to R 2 for exchanging residues-those experiencing motions on this millisecond time frame-and thus causing a decrease in NMR resonance intensity. Plotting this decrease in signal intensity as a function of v CPMG not only allows the qualitative identification of residues experiencing conformational exchange in the enzyme, but fitting the curves to relaxation dispersion equations also allows the quantitative extraction of structural information related to the excited (less populated) sub-states (differences in chemical shifts, Δω), in addition to their population dynamics in solution (conformational exchange rates, k ex , and equilibrium populations, p A and p B ) (see [START_REF] Manley | NMR insights into protein allostery[END_REF] and references therein).

With regards to the timescale of protein motions, NMR is not limited to the μs-ms range. To this day, the model-free formalism developed by Lipari and Szabo in the early 1980s remains one of the preferred approaches to determine the atomic-scale dynamics of a protein in solution [START_REF] Lipari | Model-free approach to the interpretation of nuclear magnetic resonance relaxation in macromolecules. 1. Theory and range of validity[END_REF][START_REF] Lipari | Model-free approach to the interpretation of nuclear magnetic resonance relaxation in macromolecules. 2. Analysis of experimental results[END_REF]. Using spin relaxation NMR data, the method primarily provides information on residue dynamics occurring on the ps-ns timescale, which are most often attributed to single residue side-chain rotations or local residue motions. The method measures spin relaxation rates for each residue amide N-H pair to extract order parameters (S 2 ), conformational exchange parameters (R ex ), and the correlation time (τ c ), offering an interpretation of the internal amplitude of the N-H bond vector in solution [START_REF] Morin | A practical guide to protein dynamics from 15N spin relaxation in solution[END_REF][START_REF] Fisette | Synergistic applications of MD and NMR for the study of biological systems[END_REF][START_REF] Mandel | Backbone dynamics of Escherichia Coli Ribonuclease HI: Correlations with structure and function in an active enzyme[END_REF]. The order parameter S 2 provides an easily interpretable measure of the rigidity of each protein residue, where the S 2 of a completely unrestricted N-H bond would theoretically correspond to a value of 0, while that of a fully rigid residue would be 1. For a number of residues that do not fit to simple calculation models, an R ex exchange parameter can be estimated to achieve an adequate fit to the model-free analysis [START_REF] Mandel | Backbone dynamics of Escherichia Coli Ribonuclease HI: Correlations with structure and function in an active enzyme[END_REF].

While the aforementioned NMR relaxation analyses provide detailed information on the molecular flexibility experienced by enzymes in solution, other experimental methodologies offer additional details pertaining to potentially significant biological motions. Singlemolecule Förster Resonance Energy Transfer (single-molecule FRET) [START_REF] Henzler-Wildman | Dynamic personalities of proteins[END_REF][START_REF] Stephenson | Characterizing 3D RNA structure by single molecule FRET[END_REF] and sitedirected spin labeling Double Electron-Electron Resonance (DEER) spectroscopy [START_REF] Huang | Inhibitor-induced conformational shifts and ligand-exchange dynamics for HIV-1 protease measured by pulsed epr and NMR spectroscopy[END_REF]also called pulsed Electron Paramagnetic Resonance (pulsed-EPR)-are both used to determine the distance between two labels, ranging up to a few tens of Ǻ. In DEER, a pair of cysteine residues is chemically modified with electronic spin labels, namely the nitroxide radical of a methylthiosulfonate moiety [START_REF] Torbeev | Protein conformational dynamics in the mechanism of HIV-1 protease catalysis[END_REF][START_REF] Huang | The role of select subtype polymorphisms on HIV-1 protease conformational sampling and dynamics[END_REF]. The magnitude of the dipolar coupling between the two spins can be extracted from the measured DEER echo curves, and this magnitude is inversely proportional to the cube of the distance between the two spins [START_REF] Jeschke | Distance measurements on spin-labelled biomacromolecules by pulsed electron paramagnetic resonance[END_REF], thus producing a distance profile. Populations of the different states of the protein can then be evaluated by deconvoluting the distance profile. Much like with protein crystallography, a disadvantage of the technique is the requirement of cryogenic temperatures [START_REF] Huang | Inhibitor-induced conformational shifts and ligand-exchange dynamics for HIV-1 protease measured by pulsed epr and NMR spectroscopy[END_REF], which does not provide an accurate description of biological conditions. On the other hand, singlemolecule FRET can be used in solution and at physiologically relevant temperatures, much like NMR. This technique also requires labeling of two distinct sites on a protein, this time using two different fluorophores. Since these fluorophores are selected so that the absorption spectrum of the first (the acceptor) matches the emission spectrum of the second (the donor), the fluorescence energy can be transmitted from the donor to the acceptor if they are in close proximity to each other. Thus, the emission wavelength of the donor will disappear and the emission wavelength of the acceptor will be detected as a function of distance. By measuring the FRET efficiency, the inter-fluorophore distance can be calculated. The single-molecule version of this experiment can thus allow one to witness whole domain reorientations in a protein. This technique also has the advantage of being performed in real-time [START_REF] Henzler-Wildman | Dynamic personalities of proteins[END_REF][START_REF] Stephenson | Characterizing 3D RNA structure by single molecule FRET[END_REF]. While most experimental methodologies yield quantitative information on flexibility events, they often fail to provide a clear visual representation of the actual atomic-scale motions experienced by an enzyme in solution [START_REF] Fisette | Synergistic applications of MD and NMR for the study of biological systems[END_REF]. In contrast, computational molecular dynamics (MD) simulations trace the position and interaction of atoms over time based on classical mechanics principles, to provide an atomic-scale time evolution of conformational properties in proteins and other biomolecules. Until recently, MD simulations typically sampled over hundreds of nanoseconds, while protein motions span a much wider range of timescales [START_REF] Henzler-Wildman | Dynamic personalities of proteins[END_REF], with many catalytically relevant motions occurring on the same timescale as the catalytic constant k cat , i.e., typically milliseconds or slower [START_REF] Cole | Evidence for flexibility in the function of ribonuclease A[END_REF][START_REF] Kohen | Role of dynamics in enzyme catalysis: Substantial versus semantic controversies[END_REF][START_REF] Boehr | Millisecond timescale fluctuations in dihydrofolate reductase are exquisitely sensitive to the bound ligands[END_REF][START_REF] Boehr | Targeting structural dynamics of the RNA-dependent RNA polymerase for anti-viral strategies[END_REF]. This presents an existing limitation of computational MD simulations. Advances in software and hardware capabilities have facilitated access to longer timescales, with several recent studies reporting ms timescale simulations [START_REF] Voelz | Molecular simulation of ab initio protein folding for a millisecond folder NTL9(1-39)[END_REF][START_REF] Bowman | Atomistic folding simulations of the five-helix bundle protein λ6-85[END_REF][START_REF] Voelz | Slow unfolded-state structuring in Acyl-coA binding protein folding revealed by simulation and experiment[END_REF]. While a single ms trajectory has been achieved using Anton [START_REF] Shaw | Atomic-level characterization of the structural dynamics of proteins[END_REF], a specialized supercomputer for protein simulations, access to the ms timescale with traditional, unbiased MD simulations still remains limited.

A variety of enhanced sampling techniques such as accelerated MD (aMD) [START_REF] Bucher | Accessing a hidden conformation of the maltose binding protein using accelerated molecular dynamics[END_REF][START_REF] Hamelberg | Accelerated molecular dynamics: A promising and efficient simulation method for biomolecules[END_REF], umbrella sampling [START_REF] Wojtas-Niziurski | Self-learning adaptive umbrella sampling method for the determination of free energy landscapes in multiple dimensions[END_REF], steered MD [START_REF] Da | Dynamics of pyrophosphate ion release and its coupled trigger loop motion from closed to open state in RNA polymerase II[END_REF], and metadynamics [START_REF] Tagle | Percutaneous encoscopic gastrostomy[END_REF][START_REF] Sutto | Effects of oncogenic mutations on the conformational free-energy landscape of EGFR kinase[END_REF][START_REF] Abrams | Enhanced sampling in molecular dynamics using metadynamics, replicaexchange, and temperature-acceleration[END_REF], among others, have been developed to improve sampling of the conformational landscape of proteins. In aMD, the potential energy landscape is modified by raising energy minima that are below a threshold, therefore minimizing the energy barriers separating states, and facilitating access to conformational states not easily accessible through traditional MD [START_REF] Hamelberg | Accelerated molecular dynamics: A promising and efficient simulation method for biomolecules[END_REF]. This approach has been used to characterize dynamics on the μs-ms timescale in a variety of protein systems [START_REF] Doshi | Towards fast, rigorous and efficient conformational sampling of biomolecules: Advances in accelerated molecular dynamics[END_REF]. A combination of normal and accelerated MD simulations was recently used to characterize the role of conformational dynamics in stabilizing the transition state in the enzyme cyclophilin A [START_REF] Doshi | Resolving the complex role of enzyme conformational dynamics in catalytic function[END_REF]. Umbrella sampling uses a biasing potential to characterize the higher energy states along a reaction coordinate [START_REF] Torrie | Nonphysical sampling distributions in monte carlo free-energy estimation: Umbrella sampling[END_REF]. Metadynamics involves the use of a history-dependent biasing potential that acts on collective variables corresponding to select degrees of freedom [START_REF] Tagle | Percutaneous encoscopic gastrostomy[END_REF]. A combination of NMR and metadynamics was recently used to characterize the conformational ensemble and the free energy landscape of the highly dynamic helix 1 of the prion protein [START_REF] Camilloni | Energy landscape of the prion protein helix 1 probed by metadynamics and NMR[END_REF]. While these approaches have clear benefits from enhanced sampling, the requirement of the knowledge of start and end structures (umbrella sampling), or the selection of suitable reaction coordinates (metadynamics), among others, limit their application [START_REF] Andersen | Toward an enhanced sampling molecular dynamics method for studying ligand-induced conformational changes in proteins[END_REF].

Other advanced sampling methods, such as the Markov State Model (MSM, also known as the kinetic network model) [START_REF] Singhal | Using path sampling to build better markovian state models: Predicting the folding rate and mechanism of a tryptophan zipper β hairpin[END_REF][START_REF] Bowman | Progress and challenges in the automated construction of Markov state models for full protein systems[END_REF][START_REF] Noé | Transition networks for modeling the kinetics of conformational change in macromolecules[END_REF] and Quasi Anharmonic Analysis (QAA) [START_REF] Ramanathan | Protein conformational populations and functionally relevant substates[END_REF][START_REF] Ramanathan | Evolutionarily conserved linkage between enzyme fold, flexibility, and catalysis[END_REF], facilitate the identification of conformational sub-states, equilibrium populations of substates, and transition probabilities between these sub-states. MSMs are constructed from trajectories obtained from MD simulations. In these models, the conformational space is discretized into multiple states, and a network connecting these states is created from the MD simulations. A transition matrix corresponding to the probability of transitions between different states is constructed to characterize the kinetics of transition between states. MSMs have been applied to sample a wide range of timescales associated with protein folding, functional dynamics, ligand binding and for characterizing intrinsically disordered proteins [START_REF] Shukla | Markov state models provide insights into dynamic modulation of protein function[END_REF][START_REF] Chodera | Markov state models of biomolecular conformational dynamics[END_REF][START_REF] Morcos | Modeling conformational ensembles of slow functional motions in Pin1-WW[END_REF]. QAA uses higher order statistics to describe positional fluctuations and the coupling of these fluctuations in different regions of the protein [START_REF] Ramanathan | Evolutionarily conserved linkage between enzyme fold, flexibility, and catalysis[END_REF]. It allows the characterization of conformational sub-states along a reaction pathway and the identification of sub-states with structural and dynamical properties important for function [START_REF] Ramanathan | Discovering conformational sub-states relevant to protein function[END_REF]. This approach was used to identify the hierarchical organization of conformational sub-states in ubiquitin and the T4 lysozyme, and the conformational sub-states associated with the cis/ trans isomerization catalyzed by cyclophilin A were also characterized [START_REF] Ramanathan | Discovering conformational sub-states relevant to protein function[END_REF]. This technique was recently used to characterize conformational sub-states associated with the interconversion between reactant and product states in wild-type and mutant forms of RNase A [START_REF] Gagné | Perturbation of the conformational eynamics of an active-site loop alters enzyme activity[END_REF]. Recent years have also seen advances in the integration of experimental and computational approaches for characterizing the dynamics of proteins. NMR chemical shifts and residual dipolar couplings have been used as replica-averaged structural restraints to characterize the conformational fluctuations in RNase A and the hen lysozyme, respectively [START_REF] Camilloni | Characterization of the conformational equilibrium between the two major substates of RNase A using NMR chemical shifts[END_REF][START_REF] Simone | Characterization of the interdomain motions in hen lysozyme using residual dipolar couplings as replica-averaged structural restraints in molecular dynamics simulations[END_REF][START_REF] Camilloni | Assessment of the use of NMR chemical shifts as replicaaveraged structural restraints in molecular dynamics simulations to characterize the dynamics of proteins[END_REF].

Case Studies

HIV Protease

The protease of the type 1 human immunodeficiency virus (HIV-1 PR) is an aspartyl proteinase that cleaves the polyprotein product of the gag-pro-pol gene and the Gag protein itself, both of which are vital steps in the maturation of the virus [START_REF] Sundquist | HIV-1 assembly, budding, and maturation[END_REF]. The three most prevalent subtypes of the virus are those whose proteases have been the most studied: subtypes B, C, and CRF_01 A/E, which are the prevalent HIV subtypes in North America and Europe, Africa, and Asia, respectively. Even though all three subtypes have a distinct wild-type (WT) protease, subtype B is the most widely characterized [START_REF] Huang | Inhibitor-induced conformational shifts and ligand-exchange dynamics for HIV-1 protease measured by pulsed epr and NMR spectroscopy[END_REF]. To date, nine Food and Drug Administration (FDA)-approved inhibitors targeting this protein have been used to treat HIV/acquired immune deficiency syndrome (AIDS)-infected patients. However, multidrug resistance to protease inhibitors has sparked renewed interests in uncovering distinct inhibitors targeting HIV-1 PR, and thus understanding drug resistance acquisition mechanisms is of utmost importance [START_REF] Ghosh | Recent progress in the development of HIV-1 protease inhibitors for the treatment of HIV/AIDS[END_REF].

HIV-1 PR is only active in its homodimeric form. The 99-amino-acid protomers assemble in a symmetric configuration, forming a tunnel. The protomers each provide one of the catalytic aspartate residues (Asp25, located in the tunnel), while access to the catalytic site is allowed through the opening and closing of a pair of β-hairpins, termed "flaps" (residues 43-58) (colored red in Figure 2). The protomers interact with each other through their Nand C-termini, as well as at the tip of the flaps [START_REF] Spinelli | The three-dimensional structure of the aspartyl protease from the HIV-1 isolate BRU[END_REF]. A previous report briefly discusses the atomic-scale dynamics experienced by WT HIV-1 PR, describing how movements of the flaps originating from the isomerization of Gly51 are transmitted to the active site [START_REF] Pastor | Information flow and protein dynamics: The interplay between nuclear magnetic resonance spectroscopy and molecular dynamics simulations[END_REF]. This vital conformational transition, identified as early as 1995, involves the 180° reorientation of the Gly51 backbone N-H vector, exchanging between β-turn types I and II, a process taking place over a 10 μs timescale, as determined from R ex values extracted using the Lipari-Szabo model-free formalism mentioned above [START_REF] Nicholson | Flexibility and function in HIV-1 protease[END_REF]. This unique residue reorganization represents the sole significant structural difference between otherwise symmetrical protomers in the crystal structures. Most commonly, Gly51 is found in the L-conformation in one protomer and in the D-conformation in the opposing protomer [START_REF] Torbeev | Protein conformational dynamics in the mechanism of HIV-1 protease catalysis[END_REF]. Model-free analyses also confirmed that the flap region, along with the hinge loop (residues 34-42), are the primary regions in which sub-nanosecond timescale residue dynamics are localized [START_REF] Ishima | A diverse view of protein dynamics from NMR studies of HIV-1 protease flaps[END_REF].

NMR relaxation and computational MD simulations demonstrated that flaps can adopt a variety or conformations as the enzyme proceeds through its catalytic cycle [START_REF] Huang | Inhibitor-induced conformational shifts and ligand-exchange dynamics for HIV-1 protease measured by pulsed epr and NMR spectroscopy[END_REF][START_REF] Xia | NMR relaxation in proteins with fast internal motions and slow conformational exchange: Model-free framework and Markov state simulations[END_REF][START_REF] Ishima | Solution structure of the mature HIV-1 protease monomer: Insight into the tertiary fold and stability of a precursor[END_REF]. These flaps alternate between a "closed" conformation, a "semi-open" conformation, a "wide open" conformation, and a "curled/tucked" conformation (Figure 2). The flaps close over the active site to maintain the substrate inside the catalytic pocket and to provide a gating mechanism for substrate binding, but they also interact with each other in the absence of substrate or inhibitor [START_REF] Ishima | A diverse view of protein dynamics from NMR studies of HIV-1 protease flaps[END_REF]. A combination of DEER measurements, NMR relaxation experiments, and MD simulations provided information on Asp25 and flap dynamics in each protomer during catalytic turnover. The μs-ms timescale motions of Asp25 and Gly27 were found to correlate with those of the flaps [START_REF] Torbeev | Protein conformational dynamics in the mechanism of HIV-1 protease catalysis[END_REF]. It was also observed that the more flexible L-Gly51 must be located on the same protomer as the protonated general acid Asp25-COOH for efficient catalysis to occur, while the more rigid D-Gly51 must be located on the protomer bearing the unprotonated general base Asp25-COO -. The rate-limiting step of catalytic turnover was shown to be linked to isomerization of the flap tips towards the type I and type II β-turn configurations [START_REF] Torbeev | Protein conformational dynamics in the mechanism of HIV-1 protease catalysis[END_REF], further illustrating the functional importance of conformational exchange in this enzyme system.

The kinetic network model, constructed by discretizing the conformational space of atomistic simulations into a series of related sub-states, was used to determine the transitions between the semi-open, closed, and open conformations of HIV-1 PR [START_REF] Deng | Insights into the dynamics of HIV-1 protease: A kinetic network model constructed from atomistic simulations[END_REF]. This approach facilitates the characterization of interconverting rates between kinetically relevant substates, and can predict kinetic quantities on slower (millisecond) timescales [START_REF] Shukla | Markov state models provide insights into dynamic modulation of protein function[END_REF][START_REF] Shukla | Activation pathway of SRC kinase reveals intermediate states as targets for drug design[END_REF]. Deng et al. combined replica exchange molecular dynamics simulations with transition path theory to characterize the diversity and temperature-dependence of kinetic pathways between the three conformational states of HIV-1 PR. They showed that transitions between semi-open and closed states occur on a fast timescale (~33 ns), while transitions between semi-open and open states were infrequent (~375 ns), consistent with NMR observations showing motions on the fast sub-nanosecond and slower microsecond timescales [START_REF] Deng | Insights into the dynamics of HIV-1 protease: A kinetic network model constructed from atomistic simulations[END_REF]. Not all studies agree on which conformation is the most likely populated form of the apo WT protease in solution. Studies using the DEER method concluded that the semi-open conformation is the most populated state, independent of the subtype [START_REF] Torbeev | Protein conformational dynamics in the mechanism of HIV-1 protease catalysis[END_REF][START_REF] Huang | Inhibitor-induced conformational shifts and ligand-exchange dynamics for HIV-1 protease measured by pulsed epr and NMR spectroscopy[END_REF], while NMR residual dipolar coupling (RDC) measurements suggested that the closed conformation was predominant [START_REF] Roche | Conformation of inhibitor-free HIV-1 protease derived from nmr spectroscopy in a weakly oriented solution[END_REF]. However, all studies agree that the populations of these conformers change upon inhibitor binding, with the most potent inhibitors favoring the "closed" conformation [START_REF] Huang | Inhibitor-induced conformational shifts and ligand-exchange dynamics for HIV-1 protease measured by pulsed epr and NMR spectroscopy[END_REF][START_REF] Yamazaki | Three-dimensional solution structure of the HIV-1 protease complexed with DMP323, a novel cyclic urea-type inhibitor, determined by nuclear magnetic resonance spectroscopy[END_REF][START_REF] Leonis | A contribution to the drug resistance mechanism of darunavir, amprenavir, indinavir, and saquinavir complexes with HIV-1 protease due to flap mutation I50V: A systematic MM-PBSA and thermodynamic integration study[END_REF][START_REF] Gupta | Structural studies on molecular mechanisms of nelfinavir resistance caused by non-active site mutation V77I in HIV-1 protease[END_REF]. In all common subtypes of WT HIV-1 PR (subtypes B, C, and CRF_01 A/E), the residence time of inhibitors in the active-site pocket, and thus the inhibitory strength, is correlated with the degree to which said inhibitor induces flap closure. Consequently, many drug resistance-inducing mutations perturb the dynamic behavior of the flaps relative to the WT enzyme [START_REF] Huang | Inhibitor-induced conformational shifts and ligand-exchange dynamics for HIV-1 protease measured by pulsed epr and NMR spectroscopy[END_REF]. Different inhibitors cause varying population shifts among the four possible flap conformations. According to DEER measurements, while inhibitors indinavir, nelfinavir, and atazanavir trigger very small differences in populations in all three subtypes-the semi-open conformation being the most populated-all other six FDA-approved inhibitors strongly shift the population equilibrium towards the closed configuration. Interestingly, the same six inhibitors also make the curled/tucked conformer much more likely in subtype CRF_01 A/E [START_REF] Huang | Inhibitor-induced conformational shifts and ligand-exchange dynamics for HIV-1 protease measured by pulsed epr and NMR spectroscopy[END_REF]. While smaller in scope than those of the flaps, conformational changes in the hydrophobic core predicted by MD simulations were also confirmed to impact protease activity [START_REF] Mittal | Hydrophobic core flexibility modulates enzyme activity in HIV-1 protease[END_REF]. In the WT HIV-1 PR, the two halves of the hydrophobic core slide over each other during these rearrangements, and cross-linking these two halves-thus impeding hydrophobic core rearrangements-causes a 150-fold reduction in catalytic efficiency [START_REF] Mittal | Hydrophobic core flexibility modulates enzyme activity in HIV-1 protease[END_REF][START_REF] Goldfarb | Defective hydrophobic sliding mechanism and active site expansion in HIV-1 protease drug resistant variant Gly48Thr/Leu89Met: Mechanisms for the loss of saquinavir binding potency[END_REF]. Also, mutations distal to the active site in the hydrophobic core are hypothesized to cause drug resistance by altering the flexibility of the protein [START_REF] Mittal | Hydrophobic core flexibility modulates enzyme activity in HIV-1 protease[END_REF].

Previous reports have highlighted that many distinct mutations can arise in patients developing drug resistance to HIV treatment [START_REF] Lee | Catalytic contributions from remote regions of enzyme structure[END_REF]. Some of these mutations are located in the active-site pocket and directly impair inhibitor binding (primary mutations), while others are more distal and indirectly induce resistance (secondary mutations) (Figure 3). None of these distal mutations cause significant changes in either k cat or K M in HIV-1 PR, suggesting that conformational events experienced by the enzyme play a significant role in resistance acquisition, which was confirmed by NMR, MD simulations, and DEER spectroscopy. Mutations in 45 of the 99 residues in HIV-1 PR have been linked to drug resistance in vitro [START_REF] Wu | Mutation patterns and structural correlates in human immunodeficiency virus type 1 protease following different protease inhibitor treatments[END_REF], and a quarter of the residues have been linked to drug resistance in patients, arising through evolutionary selective pressure driven by protease inhibitor therapy. However, mutations at only seven sites can be linked to direct active-site modifications (Figure 3a) [START_REF] Leonis | A contribution to the drug resistance mechanism of darunavir, amprenavir, indinavir, and saquinavir complexes with HIV-1 protease due to flap mutation I50V: A systematic MM-PBSA and thermodynamic integration study[END_REF][START_REF] Wu | Mutation patterns and structural correlates in human immunodeficiency virus type 1 protease following different protease inhibitor treatments[END_REF][START_REF] Ragland | Drug resistance conferred by mutations outside the active site through alterations in the dynamic and structural ensemble of HIV-1 protease[END_REF]. These primary mutations often decrease ligand binding affinity, both for inhibitors and natural substrates. Secondary mutations often arise to restore activity towards substrates, sometimes as a result of conformational behavior. For instance, incorporation of I15V, E35D, R41K, and R57K was demonstrated by DEER to increase backbone flexibility in WT HIV-1 PR [START_REF] Huang | The role of select subtype polymorphisms on HIV-1 protease conformational sampling and dynamics[END_REF]. These mutations, which are all believed to act as function-restoring secondary replacements, result in the increased population of the curled/tucked flap conformation, which does not normally occur in apo subtype B. Other secondary mutations can favor drug resistance. For example, 13 out of the 19 hydrophobic core residues are associated with drug resistance acquisition, despite the absence of direct contacts with inhibitors [START_REF] Mittal | Hydrophobic core flexibility modulates enzyme activity in HIV-1 protease[END_REF]. Importantly, the accumulation of mutations in HIV-1 PR does not necessarily lead to increased drug resistance [START_REF] Huang | The role of select subtype polymorphisms on HIV-1 protease conformational sampling and dynamics[END_REF][START_REF] Gupta | Structural studies on molecular mechanisms of nelfinavir resistance caused by non-active site mutation V77I in HIV-1 protease[END_REF][START_REF] Tzoupis | Systematic molecular dynamics, MM-PBSA, and ab initio approaches to the saquinavir resistance mechanism in HIV-1 PR due to 11 double and multiple mutations[END_REF]. Indeed, alterations in inhibitor-protein interactions caused by secondary mutations correlate only partly with K i variations. The explanation most often put forward is that these secondary mutations alter the dynamic ensemble of the protein, which propagates through the entire structure (Figure 3b), sometimes even affecting the dynamically constrained catalytic Asp25 residue [START_REF] Roche | Conformation of inhibitor-free HIV-1 protease derived from nmr spectroscopy in a weakly oriented solution[END_REF][START_REF] Leonis | A contribution to the drug resistance mechanism of darunavir, amprenavir, indinavir, and saquinavir complexes with HIV-1 protease due to flap mutation I50V: A systematic MM-PBSA and thermodynamic integration study[END_REF][START_REF] Gupta | Structural studies on molecular mechanisms of nelfinavir resistance caused by non-active site mutation V77I in HIV-1 protease[END_REF][START_REF] Ragland | Drug resistance conferred by mutations outside the active site through alterations in the dynamic and structural ensemble of HIV-1 protease[END_REF]. Thus, if a secondary mutation does not alter protein dynamics, it will have no effect on inhibitor binding. Nonetheless, increased drug resistance is the usual outcome of mutational accumulation. For instance, a DEER study comparing a pediatric AIDS patientderived HIV-1 PR before and after protease inhibitor treatment found that the accumulation of secondary mutations shifts the populations of the conformers to favor the open-like conformations, when it originally favored the closed conformation [100]. These authors emphasize the importance of the L63P replacement on the behavior of the protein in the pretreatment sample, arguing that it likely affects hydrophobic core packing, and thus favoring the closed conformation in the apo protein.

These results illustrate how knowledge of the conformational transitions in HIV-1 PR may help develop new drug strategies against AIDS, further exemplifying the importance of concerted and organized molecular flexibility for the proper catalytic function in this enzyme. Targeting conformational transitions has already provided promising treatment alternatives for new HIV-1 PR drug leads. The use of MD to screen for new competitive or allosteric inhibitors that target MD-identified, transiently open surface cavities was shown to be effective against mutants of HIV-1 PR [START_REF] Leonis | A contribution to the drug resistance mechanism of darunavir, amprenavir, indinavir, and saquinavir complexes with HIV-1 protease due to flap mutation I50V: A systematic MM-PBSA and thermodynamic integration study[END_REF]101,102]. In one case. a designed allosteric inhibitor was demonstrated to be effective in vitro using enzymatic essays, even against drug-resistant variants [102]. Interestingly, the allosteric inhibitor strategy used in these cases was designed to lock a conformation in place, thus preventing the catalytic cycle from talking place.

DNA Polymerase β

DNA polymerase β (Pol β) is a 39 kDa monomeric enzyme involved in the base excision repair of damaged DNA. Pol β prefers short-gapped or single-base-gapped DNA substrates [103] and is responsible for the repair of around 20,000 DNA lesions per cell per day [104]. Mutations of some residues in Pol β have been shown to increase the error rate of the enzyme; further, overexpression of Pol β has been implicated in nearly 30% of human cancers [105,106]. Incorporation of correct nucleotides is an essential step in the catalytic mechanism of the enzyme. The structure of Pol β consists of two domains: an 8 kDa lyase domain involved in the removal of deoxyribose 5′-phosphate, and a 31 kDa polymerase domain which performs template-directed DNA synthesis [107]. The polymerase domain is further composed of nucleoside 5′-triphosphate selection, nucleotide transferase, and DNAbinding subdomains, corresponding to the fingers, palm, and thumb regions, respectively. The catalytic mechanism of Pol β, which requires two Mg 2+ ions as cofactors, is shown in Figure 4.

Crystallographic studies have shown that Pol β undergoes large conformational changes upon substrate binding [110-115] and suggest an induced fit mechanism for the binding of correct nucleotides [111]. The apo form of the enzyme is characterized by an extended conformation of the lyase domain (Figure 5a), which adopts a compact open conformation upon DNA binding (Figure 5b). Binding of the nucleotide 5′-triphosphate (dNTP) promotes closing of the N-helix in the finger subdomain (Figure 5c). NMR studies were recently performed to characterize the millisecond timescale conformational exchange experienced by the apo and DNA-bound binary complex forms of Pol β [START_REF] Berlow | Substrate-dependent millisecond domain motions in DNA polymerase β[END_REF]. The authors of that study showed that conformational exchange in the apo form occurs primarily in the lyase domain and the base of the thumb in the polymerase domain, and corresponds to residues that are either in direct contact or in close proximity to the substrate in the binary complex. In contrast, the number of residues displaying conformational exchange in the binary complex with DNA was significantly reduced. These residues are primarily located in the polymerase domain, suggesting that DNA binding limits the motions in the lyase domain (Figure 6). Chemical shift variations were located in the lyase domain, associated with large conformational changes experienced by Pol β upon DNA binding (Figure 5a,b). Comparison of the conformational exchange relaxation rates with chemical shift variations between the apo and binary forms indicates that the enzyme in the apo form samples conformations adopted by the DNA-bound binary form, thus contradicting the induced fit mechanism proposed by the aforementioned crystallography experiments, at least for the formation of the binary complex (step 1 in Figure 4). Interestingly, residues displaying conformational exchange in the apo and binary complex forms correspond to mutational sites known to affect Pol β activity and identified in colon cancer tumors [116]. Taken together, these results hint at the potential role of conformational motions in the catalytic function of Pol β.

Conformational changes in both Pol β and substrates have been suggested to play a critical role in the selection and incorporation of correct nucleotides [117]. Experimental [109,[111][112][113]118] and computational [119,120] studies suggest an induced fit mechanism for the selection of correct nucleotides (step 2 in Figure 4), which form a Watson-Crick base pair with the DNA substrate. The mechanism involved in the selection of correct dNTP binding and discrimination is a hotly debated topic [111,114]. No residue displayed conformational exchange, suggesting a stable conformation of the ternary complex on the millisecond timescale, similar to that observed for the binary complex. In contrast, chemical shift changes upon addition of the incorrect dNTP were much smaller than those observed for the correct dNTP. Comparison of the direction of chemical shift variations during titrations, performed using chemical shift perturbation [123], revealed that the conformation of the mismatching complex is not on-pathway to the closed conformation observed upon binding of the correct nucleotide. Further, binding of the mismatched dNTP led to enhanced millisecond dynamics of 10 residues with conformational exchange rates varying between 500 and 2700 s -1 . These results support an induced fit mechanism for the ternary complex (step 2 in Figure 4), where binding of the correct nucleotide promotes a shift from the open binary complex form to a closed ternary complex conformation.

Concluding Remarks

The intrinsic dynamical properties of enzymes have been shown to impact the catalytic function in a variety of well-characterized systems. Advances in experimental and computational approaches continue to offer insights into the important role of conformational fluctuations in function on timescales relevant for enzyme catalysis and other biological events. Conformational exchange on the catalytically relevant timescale facilitates sampling of excited-state conformations or sub-states, aiding in various steps along the reaction pathway, such as substrate recognition, binding, and product release. In this review, we depicted the important effects of conformational dynamics on correct substrate selection and substrate/inhibitor recognition and binding in two enzyme systems. These observations also emphasize the role of dynamic fluctuations in tuning the conformational landscape for efficient catalysis, and present a potential approach for using dynamics to modulate function and for designing better inhibitors in drug design. Conformational exchange experienced by the Met20 loop in E. coli DHFR, as probed by 15 N-CPMG NMR relaxation dispersion experiments. The CPMG experiment is particularly well suited to extract exchange rates (k ex ) between two or more conformations in solution on the timescale of catalysis (ms) in many enzyme systems, offering a measure of comparison between conformational exchange experienced by the enzyme and its catalytic rate (k cat ). This method can also provide quantitative information on low-populated "invisible" excited sub-states and theft populations in solution (p A and p B ). The cartoon representation illustrates a simplified view of the closed (green) and occluded (magenta) conformations sampled by the Met20 loop in E. coli DHFR as it catalyzes hydride transfer, an atomic-scale movement essential to bacterial DHFR function and correlated with substrate/cofactor recognition and turnover in this enzyme (reviewed in [START_REF] Hanoian | Perspectives on electrostatics and conformational motions in enzyme catalysis[END_REF][START_REF] Bhabha | Keep on moving: Discovering and perturbing the conformational dynamics of enzymes[END_REF][START_REF] Luk | Protein motions and dynamic effects in enzyme catalysis[END_REF]). Depicted PDB structures are 1RX2 (green) and 1RX7 (magenta).

  [109]. Crystal structures with open (Figure 5d), closed (Figure 5e), and intermediate conformations have been obtained with mismatched or incorrect dNTPs, such as dA that cannot form a correct base pair with dG of the DNA substrate. Previous fluorescence studies [118,121,122] showed movements in Pol β upon incorporation of dNTPs and cofactors, while more recent studies [108] indicate the rapid movement (ms timescale) of the fingers domain upon binding to correct dNTPs. Incorporation of incorrect dNTPs did not induce these conformational changes. Moscato et al. characterized the effects of Pol β binding to correct and incorrect dNTPs on the millisecond timescale using NMR relaxation dispersion experiments [109]. Binding of correct dNTP to Pol β led to large chemical shift variations localized in the fingers subdomain, similar to previous observations

Figure 1 .

 1 Figure 1.

Figure 2 .

 2 Figure 2. Conformational states experienced by HIV-1 PR as it proceeds through its catalytic cycle. Crystal structures of HIV-1 PR in the semi-open (PDB 1HHR), closed (PDB 1HVR) and wide open (PDB 1TW7) conformations. The catalytic Asp25 residues from each protomer are depicted as cyan sticks and the "flaps" are depicted in red. The hinge loop, corresponding to residues 34-42, is shown in blue. The curled/tucked conformation of HIV-1 PR is not depicted, since it has never been crystallographically resolved.

Figure 3 .

 3 Figure 3. Location of drug resistance and conformational mutations in HIV-1 PR. Structural mapping of various mutations on the crystal structure of apo HIV-1 PR (1HHR). The catalytic Asp25 residue is depicted as cyan sticks on each enzyme protomer. All spheres represent a residue documented to undergo mutation in HIV-1 PR [97]. (a) Location of drug resistance-inducing mutations. Magenta spheres represent mutations that impair inhibitor binding through direct active-site contacts (primary mutations) and gray spheres represent distal indirect mutations (secondary mutations); (b) Location of mutations documented to affect the conformational sampling of HIV-1 PR, depicted as orange spheres. Gray spheres are other drug resistanceinducing mutations. All mutations were compiled from references listed in the text.

Figure 4 .

 4 Figure 4. Schematic representation of the catalytic mechanism of Pol β. E o and E c correspond to the open and closed conformations of the enzyme. Sequential binding of the DNA (DNA n ) and dNTP leads to the formation of the binary (step 1) and ternary complexes (step 2), respectively. Subsequent conformational rearrangement of the enzyme (E c ) aligns the active site for catalysis (step 3). The chemical step of catalysis is followed by the release of pyrophosphate (PP i ) and the DNA product (DNA n+1 ) (step 4).Figure adapted from [108,109].

  Figure adapted from [108,109].

Figure 5 .

 5 Figure 5. Structure of free and ligand-bound states of Pol β. The lyase domain is shown in red while the thumb, palm, and fingers subdomains of the polymerase domain are shown in blue, green and yellow, respectively. Pol β in the (a) apo (PDB 1BPD) form is characterized by an open conformation; (b) binary complex (PDB 1BPX) formation is accompanied by a 40 Ǻ conformational shift of the lyase domain to form a compact "open" conformation; (c) matched dNTP-bound ternary (PDB 1BPY) complex shows a 10 A shift of the N-helix of the fingers domain relative to the binary form (in orange), leading to the formation of a "closed" conformation. Binding of mismatched dNTPs has been shown to adopt (d) closed (PDB3C2M) and (e) open (PDB 4F5P) conformations of the N-helix. For oomparison, the N-helix from the binary form is shown in orange in the ternary complexes in c-e, illustrating subtle yet functionally significant conformational exchange in this enzyme. DNA substrates in all the binary and ternary forms are shown in grey (b-e), the dNTPs in c-e are shown as cyan sticks, and the Mg 2+ ions are represented as magenta spheres in c.

Figure 6 .

 6 Figure 6. Conformational exchange experienced by Pol β. Residues displaying elevated R 2 and 15 N-CPMG relaxation dispersion curves are shown as blue and red spheres, respectively, corresponding to Cα atoms. (a) The apo form shows millisecond dynamics predominantly in the lyase domain (residues 1-90). Residues with k ex ~1400 s -1 are shown as smaller spheres while residues with k ex ~4000 s -1 are shown as larger red spheres. Few residues in the other polymerase domains show elevated R2 values; (b) Onlyresidue Glu21 displays 15 N-CPMG relaxation dispersion in the binary form. Other residues displaying elevated R2 values are located in the polymerase domain of Pol β. DNA in the binary form is shown in black. The relaxation data used for preparing this figure were taken from reference [8].

Catalysts. Author manuscript; available in PMC 2017March 31.

Acknowledgments

This work was supported by the National Institute of General Medical Sciences (NIGMS) of the US National Institutes of Health (NIH) under award number R01GM105978 (to N.D.) and a Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant under award number RGPIN-2016-05557 (to N.D.). D.N.B. is the recipient of an NSERC Postgraduate Scholarships-Doctoral (PGS D) Award and N.D. holds a Fonds de Recherche Québec-Santé (FRQS) Research Scholar Junior 2 Career Award.


Teja Bhargav
email: bhargav.teja-nallapu@inria.fr

Nallapu

Raju Bapi

Surampudi

Nicolas P Rougier

The art of scaling up : a computational account on action selection in basal ganglia

) is exploited in this work. Crucial aspects in a computational model, like parameter tuning and detailed neural implementations, while moving from a simplistic to large-scale model, are studied.

I. INTRODUCTION

Action selection is one of the most vital processes for the survival of an organism.

The very importance of the process of action selection (and learning from it) in the survival of an organism, pushes for a need of understanding the process from an evolutionary, biological and physiological point of view.

Brain works by running complex dynamics forming complex connections within itself, specific to each behavioral repertoire. It is at the level of few (or many) neurons that the information transfer takes place, which drives all the processes like perception, memory, decision making, language or any other brain process. Ideally, it is that level of detail one wants to model any sub-system of brain or the whole model of brain itself. Because of the practical limitations like lack of detailed experimental data, computational challenges in studying the existing data forces major modeling efforts to employ simplistic neuron models and smaller population representations.

It is a positive sign that despite these limitations, large scale models still manage to emerge, if not at a full-human scale, but at least at a scale of a rodent's brain. A recent work in [START_REF] Moren | A full rat-scale model of the basal ganglia and thalamocortical network to reproduce parkinsonian tremor[END_REF] proposes a thalamo-cortical network model of BG to reproduce Parkinsonian tremor. Quite interestingly, the model consists of four neuron types in all the subareas, modeled at the full spatial size and neuron numbers of the rat. However, depending on the functional dynamics one tries to model like in [START_REF] Leblois | Competition between feedback loops underlies normal and pathological dynamics in the basal ganglia[END_REF], [START_REF] Guthrie | Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study[END_REF], often simpler neuron (rate) models are employed to keep the computations easier and emphasize more on the network dynamics than on the detail of neuronal implementations.

We considered one of our previous works on a computational study of decision making and its dynamics using a thalamo-cortical BG network [START_REF] Nallapu | Dynamics of reward based decision making a computational study[END_REF] inspired by [START_REF] Guthrie | Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study[END_REF] and replicated in [START_REF] Topalidou | re] interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study[END_REF]. It is a framework on the top of an optimal decision making model, accommodating representations of varying visual salience and delayed presentation of the stimuli, studying their effect on optimal decision making. The framework uses known network dynamics as in [START_REF] Guthrie | Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study[END_REF] and the model exhibits similar characteristics as compared to experimental work described in [START_REF] Pasquereau | Shaping of Motor Responses by Incentive Values through the Basal Ganglia[END_REF]. The framework is indeed robust to study the other factors mentioned, that affect optimal decision making -such as stimuli properties, presentation delays and different associations of reward in learning. Aforementioned model is still quite simplistic with as few as 72 neurons representing all of the cortico-thalamic basal ganglia network, involved in decision making. This is where we emphasize on modeling the same functional networks at a large scale.

The simple model has been successfully expanded to a large scale, both in terms of neuron populations and extending to a variety of tasks that the BG are known to be involved in. For example, we have attempted to build a similar large scale model of BG also demonstrating an arm reaching a target 1 (based on the model described in [START_REF] Vignesh Muralidharan | A scalable cortico-basal ganglia model to understand the neural dynamics of targeted reaching[END_REF]), besides the reward based decision making. Scaling up involved exploring various large scale neural modeling frameworks like DANA [START_REF] Rougier | Dana: Distributed numerical and adaptive modelling framework[END_REF] and NEF 2 (Nengo). The model is scaled using DANA to as many as 66,000 neurons. With the large scale model, the functional properties and neural activity patterns have been found to be consistent with the simplistic model. Using the large scale neural modeling platform, Nengo, also yielded the same dynamics of the simplistic model but now the model is expanded to a massive scale of 300K neurons. The power of converting existing models to efficiently weighed neural networks in Nengo is exploited in this work. Additionally, NEF accounts for various biologically realistic properties of neurons which form the internal neural networks.

II. METHODS

A. Task

The task used in this work is a probabilistic learning task that was described in [START_REF] Pasquereau | Shaping of Motor Responses by Incentive Values through the Basal Ganglia[END_REF] and used in [START_REF] Guthrie | Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study[END_REF], [START_REF] Topalidou | re] interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study[END_REF], [START_REF] Nallapu | Dynamics of reward based decision making a computational study[END_REF]. Four target shapes are associated with different reward probabilities (see figure 1). Each trial (an independent task turn), any two of the four possible shapes are presented at two random positions (out of the four cardinal positions -up, right, down and left). By the end of trial period, a choice is made and the reward is given according to the reward probability associated with the chosen shape.

In a single independent trial, the cognitive decision (shape of the cue) and motor decision (direction of position) are independent of each other. However, ideally, the motor decision is expected to be in the direction of the cue shape that is chosen. On subsequent reward association of the cue decision, one should eventually select the direction in which the most rewarding cue is present.

B. Model

In [START_REF] Leblois | Competition between feedback loops underlies normal and pathological dynamics in the basal ganglia[END_REF], authors demonstrated an action selection mechanism in the cortico-basal ganglia loops based on a competition between the positive feedback, direct pathway through the striatum and the negative feedback, hyper-direct pathway through the sub-thalamic nucleus. In [START_REF] Guthrie | Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study[END_REF], authors investigated further how multiple level action selection could be performed by the basal ganglia, and the model has been extended in a manner consistent with known anatomy and electro-physiology of the basal ganglia in the monkey (see figure 2). This model allows a bidirectional information flow between loops such that during early trials, a direction can be selected randomly, irrespective of the cue positions. However, after repeated trials, the model is able to consistently make the motor decision, only after cognitive decision (indeed biased by the cognitive decision) towards the position of the more rewarding cue shape.

To begin with, we reproduce the thalamo-cortical basal ganglia network with similar characteristics as in [START_REF] Guthrie | Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study[END_REF]. On a different note, this effort also reminds how a model needs to be reproducible and less constrained on its parametric values. In the new model, we could successfully demonstrate the decision in a single trial. With dopamine modulated learning in the cognitive channels between cortex and striatum, the model reached a state where it consistently chooses the higher rewarding stimulus. We also developed the model to serve as a framework to accommodate several other complex scenarios that are more likely to happen in real life decision making.

The model in its simplest form is based on few implicit assumptions, which facilitate the demonstration of optimal action selection. One of them being the cortico-striatal learning only in the cognitive channel. That is, attributing reward only to the shape of the cue chosen but not to the position it is shown. Secondly, in all the experiment scenarios, both the stimuli are presented simultaneously. Lastly, both the stimuli are represented by same visual salience in the experimental scenarios, which is an unlikely case with real-world choices. The model has been extended so as to account such assumptions, providing a framework to study factors affecting optimal decision making. However, to expand it to a larger scale, we chose to base on its basic form so that the larger model would provide better representations to challenge such assumptions.

One of the major limitations of the simple model, we propose, is the discrete representation of stimuli in the discussed brain areas. In every structure (thoughout this paper, we refer each brain region that is modelled as structure), a single neuron (in each of cognitive and motor channels) representing each cue shape stands too simplified a representation and limits us from analyzing cases with complex stimuli. In figure 3a, an ambiguous shape is shown, which resembles two of the known shapes but still uncertain to identify. Figure 3b shows a familiar shape but with added noise that makes it difficult to identify as a known shape. It is not just difficult, but impossible to represent such complex stimuli in the simple and basic model. Thus we begin to expand the model to a scale that is sufficient to provide more biologically plausible stimuli representations, thus demonstrating action selection. Subsequently, the large scale model poses new testable hypothesis in the context of stimuli representation.

III. LARGE SCALE MODEL

With the idea of scaling up the model and to have a larger population of neurons representing the stimuli, we decided to implement the model with discussed dynamics using DANA and Cython [START_REF] Behnel | Cython: The best of both worlds[END_REF] individually and then scaled up the number of neurons representing the cue shape inputs in important areas of cortex and striatum. DANA provides very intuitive way of modeling neural networks by using structures called Groupsto represent neuron groups and connections -various type of connectivities between the groups. It internally uses modified NumPy [START_REF] Van Der Walt | The numpy array: a structure for efficient numerical computation[END_REF] structured arrays made suitable for intensive computations in the group fields, by making them a contiguous block of memory.

In the simplistic model, in a regular structure (excluding the associative structure), there is one neuron representing each stimulus (stimulus -cue shape in a cognitive channel, cue position in a motor channel) and an associative structure has a 2D array of neurons where number of rows and columns both equal to the number of stimuli. Within each structure, each neuron (in each channel of any loop) represents the average activity of a small population and is modeled as a single rate neuron with the equation:

τ dm dt = -m + I s + I Ext -T (1)
decay time constant of the synaptic input τ and threshold of the neuron T are set to respective constant values. m is the output of the neuron. I s is the synaptic current. I Ext is an I In the expanded version, a population of neurons together represents all the stimuli. To begin with, we have a clear distinction in such population that each stimulus is represented by a subgroup of neurons and each subgroup with equal number of neurons. For example, in a cognitive channel of a structure, a population of 4N neurons (N>1) for 4 cue shapes, a subgroup of N neighboring neurons represent each cue shape (see figure 4). As for the synaptic input (I ext in equation 1), a Gaussian input at the center of the subgroup for a particular cue shape is given.

It is also interesting to explore how the changes in such stimuli affect the dynamics of decision making. This complex and varying representation of stimuli is what we want to emphasize while beginning to working on a large scale model. Few examples can be seen in figure 5, where different possibilities of stimuli are represented. This kind of encoding also could address the complex stimuli presented in figure 3.

Between such structures, the connectivity is based on the assumption that the neurons in both the structures correspond to particular stimuli (shape or position) and only those subgroups of neurons are connected in the respective structures. It might be biologically implausible to assume such a distinct connectivity between the structures but it provides a good understanding of expanding this kind of model from simplistic implementations to complex highly populated networks. Using the above described structures and connections, a large computational model is developed, which exhibits the intrinsic functional properties of the simpler model mentioned. The type of connections, few corresponding to those between cortex and striatum, are listed in table II.

The model is large in terms of significantly higher number of neurons in each structure. Cortex particularly has larger population than the other structures. allows complex stimulus representations. In the simpler model, the synaptic input corresponding to a stimulus is represented in the cortical structures as external current of fixed magnitude of 7 spike(sp)/s to the neuron representing that stimulus. In case of larger populations, for each stimulus, a Gaussian input (of mean magnitude of 14 sp/s and variance spreading over the subgroup of the cortical structure) is provided. For instance, if the cortex has 1024 neurons, 256 neurons for each stimulus, and to a particular stimulus, a Gaussian input spreading over these 256 neurons, is given at the beginning of the trial. The mean cortical activity (over each sub-population), when the model is run for an independent trial, is found to be similar to that of simple model before expanding. This is one of the key characteristics a model should have, when scaled up.

The model is also run for a session of 120 trials, with cortico-striatal learning. Although the time taken for the completion of session is long (around 40 minutes, whereas the simpler models run in few seconds), model reached a performance over 90%. Owing to this computational expense, it was not easy to run the model for 250 simulations to generate a mean performance over each of the 120 trials of all 250 sessions. The key challenge in computation lies with the structures like associative cortex, which is a 2D representation. Since the connections involving associative cortex are like the ones described in figure 6, the weight matrices between the structures span up to a scale of over thousands of rows and columns, which inevitably makes the computation expensive. Fig. 6. Neurons corresponding to each stimulus in both the structures are connected. For a given stimulus (C 0 , M 0 ), each neuron in target is connected to a block of neurons in source. For eg. neuron (0,0) in target is connected to the block [(0,0,(0,1),(1,0),(1,1)] in source. Similarly all the neurons that are connected are highlighted in colors.

This leaves an interesting question about the connection weights between two highly but unequally populated (like cortex and striatum). And to point out, even in this larger model, each neuron remains to be a mere computational unit, without accounting for any other of the biological neuron properties (spiking or population coded etc). Hence, we move further exploring another large scale neural modeling framework, Nengo which appears to solve this problem by effectively computing some local weights within a sub-population of neurons while providing ready-to-use spiking neurons to build the structures.

IV. THE NEF, NENGO AND RESULTS

The underlying principles of NEF are: representation, transformation and dynamics. The activity of a group of neurons is considered to be best represented by encoding of some underlying variable, e.g., any vector generally of smaller dimensionality. The transformation of information from one neural group to the other neural group. i.e., the neural group representing x can be connected to other group of neurons representing a variable y, such that y = f (x).

Any desired function can be approximated with improved accuracy by increasing the number of neurons. The dynamics of NEF allows us to build recurrent neural networks. These networks can compute the time evolution of a given variable x of the form dx dt = f (x, u) for any control variable u. In a connection between two neurons, as the activity in the presynaptic neuron changes over time, x(t), the value actually represented in the post-synaptic neuron over time will be f(x(t)) × h(t), where h(t) represents the post-synaptic activity (e.g, h(t)=e -t/τ , for t>0 with a time constant τ ).

The structures involved are modeled as neuron ensembles, collection of several thousands of neurons. The basic pattern of connectivities are retained as listed in table II. When neuron ensembles in two different structures are connected, the individual spiking activity of each neuron is taken into account and a resultant activity of the ensemble is calculated. It is this resultant activity which is provided as the synaptic input to the target structure. This is a striking advantage in using Nengo. The individual neuron works on the basis of chosen neural dynamics, while the network dynamics can be modeled as required. The following simulations assume cue shapes are represented by [0, 1, 2, 3] in the cognitive channel of the cortex and the cardinal directions, where the shapes could be presented, are also represented by [0, 1, 2, 3] in the motor channel of the cortex. The cue shapes have a rewarding probability in the order 1>0>2>3. The synaptic weights that were learned in the previous model are applied. To ensure the optimal action is selected in a generic manner, we exchanged the weights of the first (0) and the second (1) stimuli. These weights ensure that the highest rewarding cue is always selected. It is also implicit in the model dynamics that the position corresponding to the chosen shape only will be selected.

There is an array of neuron ensembles representing each stimulus and corresponding arrays are connected between structures. The effective activity of each of these arrays is taken as the resultant activity for the stimulus the ensemble array represents. Figure 8 displays a case where shape 0 is shown at position 3 and shape 1 is shown at position 2. Since shape 1 has higher synaptic weight, it should be chosen over shape 0. It can be seen that shape 1 is selected (label 1 -green in CTX COG). Subsequently, for the motor decision, position 2 is selected (label 2 -orange in CTX MOT).

For the models claiming to have network-induced dynamics, the choice of the neuronal dynamics should not make a significant difference. This has been demonstrated clearly in this work. The model developed is tested under similar conditions of action selection task, switching between direct computations and neural implementations like Sigmoidal and Integrate & Fire (IF). To start with, we used simple neurons that directly compute a function. Then the neurons are changed to detailed implementations, but to a lower scale. However, the switch between neuron implementations involved minimal tuning of parameters of the network as well as optimization of few framework parameters, provided there is computational power to handle the large scale of spiking or IF neurons. Fig. 8. For the combinations of cue shape and position, the best rewarding cue shape is selected (see CTX COG) and the corresponding position (as per stim cog and stim mot represent shape 0 is shown at position 3 and shape 1 is shown at position 2). It can be seen that, at a point later than choosing shape, corresponding position is chosen (see CTX MOT). Inset The decision in the original simplistic model V. DISCUSSION Although the population numbers in this work are comparable to that of experimental numbers of a rat's BG and some recent modeling work by Moren et al. 2015 [START_REF] Moren | A full rat-scale model of the basal ganglia and thalamocortical network to reproduce parkinsonian tremor[END_REF] (see table IV), we are still far away from the scale of that of humans' (150K in rats where as 2.2 million neurons, only the globus pallidus and sub-thalamic nucleus combined [START_REF] Hardman | Comparison of the basal ganglia in rats, marmosets, macaques, baboons, and humans: volume and neuronal number for the output, internal relay, and striatal modulating nuclei[END_REF]). As for results, the neural activity corresponding to synaptic weight-driven decision making and the ideal choice as per high reward, were consistent. We also believe, at a full range simulations on the model, performance would match that from the previous simplistic model. In future, such a functional large scale network could become readily usable in other functional processes involving motor reaching or motor learning [START_REF] Magdoom | Modeling basal ganglia for understanding parkinsonian reaching movements[END_REF], modeling deficits in neuro-degenerative disorders such as Parkinsonianism [START_REF] Muralidharan | A computational model of altered gait patterns in parkinsons disease patients negotiating narrow doorways[END_REF] etc.

One pertinent question that arises is about the usefulness of a larger network, if the same qualitative behavior could be realized in a simpler, smaller network. In addition to being able to replicate the basic features of experimental observations as in the case of simple model, a large scale model is expected to provide additional advantages. Some of the advantages are: the flexibility to cope with sophisticated stimulus representations as opposed to simplistic ones; the degree of match with the details of known anatomical features, that is, the feasibility of a more biologically plausible architecture; and more importantly the ability to cope with insults / injuries to the system without degradation in performance, that is, graceful degradation. These additional benefits make it more appealing to try and scale up a simpler architecture. But when is a model sufficiently large? A model does not need to be as large as it can get but could be just large enough to answer questions like how do we conclude from the activity of a group of neurons that a stimulus is chosen? and how do we compare a weakly activated stimulus to a strongly activated one?

Model

Cortical level Total Simplistic model [

VI. CONCLUSION

In a model, when it comes to designing structures with bigger populations, it is quite challenging to come-up with appropriate connectivities among these structures. Assuming that an ideal large scale model takes the realistic neuronal population ratios between structures (like a rat cortex having considerably large populations than that in its striatum, as suggested by experimental data [START_REF] Paxinos | The Rat Nervous System[END_REF]), the connectivity patterns are difficult to assume and hence compute. In this research, an attempt was made by using some simple many-to-one connections in different sparsities. Using a sophisticated framework like Nengo provides benefit of taking care of internal computations among such sub-populations to propagate the resultant population activity through the inter-structural connections.

A general observation is that in neuroscience, there have been very few general principles governing systems. Also quite often, computational models are very rigid in terms of their specifications like having to specify range of values for various parameters, etc. Hence one should try to identify some generic characteristics of the structures involved in a foolproof manner, making them independent of the model details or the range of parameters used. This means model should be able to represent multiple scenarios over a range of parameters used. In this work using Nengo, such configurations have been explored. Figure 7 displays how certain parameters can be isolated from the model to better tune them. Many key parameters involving reward-prediction errors, dopamine modulation etc, could be observed independently.

The framework provides various types of neuronal implementations in-built, allowing configuration of different neuron models in different structures easily. Because of the optimal weights computation within the neuron ensembles in Nengo, inter-structural connections carry network level dynamics in a very reasonable way.

The large scale models described here, could be extended to model the formation habits, expanding structures like cortex. It has been suggested that associative striatum plays a crucial role in goal-directed actions, contrary to sensori-motor that is part of the habitual system. New evidence suggests that automatic behaviors, such as habits, are stored in cortex and that the role of BG is to train cortex. In other words, BG learns quickly via reinforcement learning to activate the correct postsynaptic target in cortex, which leads to appropriate cortical Hebbian learning ( [START_REF] Ashby | The role of the basal ganglia in category learning[END_REF], [START_REF] Ashby | Cortical and basal ganglia contributions to habit learning and automaticity[END_REF]). More evidence supporting this idea is provided by Piron et al. 2016 [19]. The scaled up cortex could provide different structured representations individually for stimulus representations, habitual learning etc, so that it could make decisions independent of BG after the acquisition of habits.

In most of the common tasks of action selection, identifying whether a stimulus is the same or different from a previous one involves coordination between the incoming sensory information and working memory [START_REF] Pooresmaeili | The effect of visual salience on memory-based choices[END_REF]. Also, in one of the cases discussed about complex stimuli, when all the choices are not presented at once and new choices appear in the middle of decision making, one needs to model a strategy with which whether value computation of each choice is re-computed. Thus the role of working memory in decision making paradigms where the basal ganglia also play a crucial role, should be studied.

Thus, a comprehensive network with all the structures modeled, demonstrating optimal action selection, motor reaching (in the lines of demonstration in [START_REF] Vignesh Muralidharan | A scalable cortico-basal ganglia model to understand the neural dynamics of targeted reaching[END_REF]), habitual learning, could be referred as true large scale model. Besides the mere high population of neurons, 'large-scale' should be with respect to the number of structures involved, different responsibilities each structure switches between and number of cognitive or biological processes simulated. Also, from a unified research perspective, when models of certain important pathways are developed computationally and made available on frameworks like Nengo, they are often reusable in the development of higher functional models using those pathways. Our two cents to reproducible science, brief pieces of code demonstrating simple as well as discussed large scale models are made public 3 . After all, science ought to be reproducible, replicable 4and extendable.

Fig. 1 .Fig. 2 .Fig. 3 .

 123 Fig.1. The two armed bandit task as described in[START_REF] Pasquereau | Shaping of Motor Responses by Incentive Values through the Basal Ganglia[END_REF],[START_REF] Guthrie | Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study[END_REF].

Fig. 4 .

 4 Fig. 4. (i) A motor structure with 4N neurons, with each N neurons representing a motor stimulus (cue position). A cognitive channel also has similar structure. (ii) An associative structure with 4Nx4N (16x16 in this case) neurons. All the columns of neurons represent 4 motor stimuli equally and all rows represent the 4 cognitive stimuli.

Fig. 5 .

 5 Fig. 5. Stimulus in the form of a Gaussian input over a subgroup of neurons and various scenarios of input representations. The inputs shown here are only to demonstrate the differences in their mean or variance, or in the timing (see Difference in timing), irrespective of any scale.

Fig. 7 .

 7 Fig. 7. BG network model implemented in nengo

  The population numbers of structures are listed in table III. Higher population in cortex

	Structure	Threshold(T) Decay (τ )
	Cortex		-3	10
	Striatum		0	10
	Globus Pallidus interna (GPi)	10	10
	Subthalamic Nucleus (STN)	-10	10
	Thalamus		-40	10
		TABLE I
	PARAMETERS IN EACH STRUCTURE
	Connection	Source		Target
	One to One	cognitive cortex	cognitive striatum
	One to One	motor cortex	motor striatum
	Associative to Associative associative cortex associative striatum
	Cognitive to Associative	cognitive cortex	associative striatum
	Motor to Associative	motor cortex	associative striatum
		TABLE II
	CONNECTIONS BETWEEN STRUCTURES

https://www.youtube.com/watch?v=oIHfDabVqcg

Neural Engineering Framework -http://nengo.ca/

https://github.com/cervere/basal-ganglia.git

http://www.labri.fr/perso/nrougier/downloads/ReproducibleScience.pdf

VII. ACKNOWLEDGEMENTS

The authors would like to acknowledge the following grants which have been a major support to this research. • Indo-French CEFIPRA Grant for the project Basal Ganglia at Large (No. DST-INRIA 2013-02/Basal Ganglia dated 13-09-2014) • Internships programme at INRIA, 6 month Internship with Team Mnemosyne at INRIA Bordeaux -Sud-Ouest • Travel Support by Nengo 3 rd annual Summer School on large-scale brain modeling, University of Waterloo, ON, Canada, 5 th June, 2016 -17 th June, 2016