Yachong Guo

Vladimir A Baulin

Fabrice Thalmann

Peroxidised phospholipid bilayers: insight from Coarse-Grained Molecular Dynamics simulations

An original coarse-grained model for peroxidised phospholipids is presented, based on the MAR-TINI lipid force field. This model results from a combination of thermodynamic modelling and structural information on the area per lipid, which have been made available recently. The resulting coarse-grained lipid molecules form stable bilayers, and a set of elastic coefficients (compressibility and bending moduli) is obtained. We compare the compressibility coefficient to the experimental values [Weber et al., Soft Matter, 2014, 10, 4241]. Predictions for mechanical properties, membrane thickness and lateral distribution of hydroperoxide groups in the phospholipid bilayer are presented.

Introduction

Oxidative stress 1 plays an important role in regulating vital processes in living organisms: defense against bacteria, cell apoptosis, immune response, cell maturation and differentiation. Free radicals or reactive oxygen species (ROS) 1 may react chemically with cell constituents, leading to irreversible damage. First line of defense of cells is the cell membrane [START_REF] Alberts | Molecular Biology of the Cell[END_REF] where lipids, proteins, cholesterol, vitamins and fatty acids first interact with the external environment of the cell.

It is generally accepted [START_REF] Repetto | Lipid Peroxidation[END_REF][START_REF] Frei | [END_REF] that the protection of cell membranes from oxidation stress is due to incorporation of molecules containing weak chemical bonds (anti-oxidants): vitamins, unsaturated fatty acids, unsaturated lipids, that can break and deactivate free radicals. In particular, oxygen radicals damage weak double bond in the tails of unsaturated lipids and fatty acids, provoking changes in their conformations and disturbing the bilayer. Balanced amount of free radicals and anti-oxidants is essential for normal function of living organisms. 5 However, when this balance is altered, it can lead to a series of diseases ranging from cancer and diabetes to several ageing diseases 6 such as Parkinson and Alzheimer diseases. Lipid peroxidation is a major consequence of oxidative damage to cell membranes due to various external factors 1,[START_REF] Repetto | Lipid Peroxidation[END_REF] such as reactive oxygen species (ROS), inflammation, catalysis by peroxisomal oxidases, virus phagocytosis, ultraviolet and ionic irradiation. Photooxidation of giant lipid vesicles allows to quantify the effect of oxidation on lipid bilayers. [7][8][9][10][11] In particular, with a careful choice of photosensitizer, it becomes possible to generate specifically lipid peroxides. 9 This approach allows to control the rate of oxidation and the total area of the vesicle, providing information on the rigidity of the lipid bilayer and the variation of the area per lipid with the degree of oxidation.

In turn, lipid bilayer structure modifications due to oxidised lipids can be directly assessed with all-atoms Molecular Dynamics (MD) simulations, 12 including peroxidised polyunsaturated lipids. [13][14][15] Only recently have been introduced coarse-grained (CG) model of oxidised species 16 and Single Chain Mean Field (SCMF) approaches. 9,17 According to these simulations, the increase of area per lipid, decrease of bilayer thickness and shift of peroxidised group towards the surface of the bilayer are general manifestations of oxidation.

In this paper we introduce a coarse-grained molecular model of hydroperoxidised lipids based on recent experimental data. 9,11 Using this model we evaluate the effect of peroxidation on the thermodynamic and mechanical properties of fully peroxidised lipid bilayers.

Coarse-grained models have been widely used to describe the structure and dynamic properties of lipid bilayers providing good pictures of self-assembly of lipid bilayers and collective phenomena. 18 One of the most widely employed coarse-grained model of lipid bilayers for molecular dynamics simulations is the MARTINI force field, 19 which gives realistic structure and elastic properties of lipid bilayers. Thus, we use this model as a starting point for peroxidised phospholipid molecules.

Coarse-grained models of hydroperoxized lipids

We construct oxidised lipid models based on the original MARTINI models 19 for 1-palmitoyl-2-oleoyl-sn-glycero-3-phosphocholine (POPC) and 1,2-dioleoyl-sn-glycero-3phosphocholine (DOPC) phospholipid molecules. We assume that hydroperoxidation acts on the double bond in the tail, C--C, resulting in a shift of its position and the addition of a side -OOH group. Further oxidation damage to lipids may cause chain cleavage and severe lipid degradation, which we do not consider in the present study.

MARTINI coarse-graining strategy is based on a 4 atoms to 1 bead correspondence, 19 with effective potentials between beads accounting for the thermodynamic mixing properties of the underlying molecular sub-groups. Hydroperoxidation has the effect of adding two oxygens atoms to an existing group of 4 carbons (Fig. 1) as a result of so-called Type II photooxidation reactions. 22,23 The CG model must therefore account for 6 atoms with amphipathic properties.

We consider that the two peroxidised products are indistinguishable at the coarse-grained level adopted here, and assume that both can be suitably represented by a single assembly of beads with average properties. The hydrophobic part retains an unsaturated C--C bond, as in the original target. The hydrophilic part OOH is without doubt a polar group with donor and acceptor hydrogen bonding character. The 6 atoms group C 4 OOH is bulkier than the original unoxidised group C 4 . One could imagine introducing for that a larger bead, but it would then be necessary to reimplement all the pairwise Lennard-Jones interactions with the other existing coarse-grained beads. Then, a bead of intermediate polarity can only poorly reproduce the two-sided (hydrophobic-polar) character of C 4 OOH. At the opposite, a set of two beads (dimer) can optimally traduce the intrinsic amphipathy of the group. By adjusting the separation between beads, as in the case of the glycerol backbone, the bulkiness of the pair can be set to a reasonable value, 50% higher than the original C 4 group (6 atoms instead of 4).

Our systematic investigations lead us to the conclusion that an optimal choice is the adjunction of MARTINI C3 (hydrophobic) bead to a MARTINI P2 (polar, h-bonding donor and acceptor) bead, separated by 0.33 nm, smaller than the standard lipid bonding distance (0.47 nm). Note that the structural properties of the resulting bilayer depend quite sensitively on this separation. This choice, as explained in Section 4, comes from a comparison with the experimental change in area per lipid resulting from the complete hydroperoxidation of the unsaturated pure POPC and DOPC lipid bilayers. The average increase in area per lipid following complete peroxidation is reported to be +15.6% for POPC bilay-ers, and +19.1% for DOPC bilayers respectively. 9 Similar area changes were also obtained by monitoring giant vesicles adhering onto specially prepared adhesive surfaces, 11 then leading to relative area changes of +14.3% and +18.4% for oxidised POPC and DOPC vesicles respectively. These values are fairly consistent, given the practical difficulties inherent to these experimental techniques. This combination of results puts stringent constraint on the CG candidate models.

Mapping to this structural quantity, however, is no substitute for performing a thermodynamic mixing assay of the C 4 OOH target group. For this purpose, knowing the chemical structure of the oxidised lipid, we selected tert-butyl hydroperoxide (TBHP, CAS 75-91-2, a commercial bleaching agent) [START_REF] Sanchez | Kirk-Othmer Encyclopedia of Chemical Technology[END_REF] as the best analogue to the peroxidised functional group C 4 O 2 . The values of water solubility and octanol/water partitioning log P ow data of TBHP we could find are scattered between 0.6 and 1.23, while simulations of the C3-P2 dimer alone suggest a numerical, coarse-grained log P ow = 1.4. Finally, the acidic constant of TBHP, pKa = 12.8, suggests a predominantly protonated, neutral hydroperoxide group, for neutral or slightly acidic pH solutions.

In what follows, we designate by HP-POPC the hydroperoxidised POPC molecule, and by DHP-DOPC the doubly hydroperoxidised DOPC (Fig. 1). We find that bilayers formed with pure HP-POPC and DHP-DOPC are stable, and provide a set of numerically estimated structural and mechanical parameters, associated with these two lipid bilayers. Snapshots of bilayers formed with POPC, DOPC, HP-POPC and DHP-POPC are provided in Fig. 2.

Results and discussion

Structural changes upon peroxidation

Pure HP-POPC and DHP-DOPC bilayers appear numerically stable and lend themselves to the determination of fairly precise structural properties. The main structural data are summarized in Table 1. Area per lipid was used to set up CG models. Assuming that these values are correct, resulting CG models provide a quantitative estimate of membrane thickness and membrane specific volume changes due to peroxidation. If one assumes that membranes are quasi-incompressible media, the volume change is entirely due to the added hydroperoxide groups, and it becomes possible to split the increase in area per lipid into two contributions: homogeneous increase and structural reorganisation. The homogeneous increase of area is directly proportional to the relative volume change (∆A/A) hom = 2/3(∆V /V b ), as if the bilayer was an homogeneous fluid with volume V b . The second contribution is due to the membrane reorganisation due to changes in chemical affinity and polarity following the addition of the hydroperoxide groups. The corresponding "structural" relative increase of area is (∆A/A) str = ∆A/A -(∆A/A) hom .

Lipids were simulated in a box with lateral x, y directions and transverse z direction, at a temperature of 300 K. An anisotropic Parinello-Rahman barostat was applied to simulate a tensionless bilayer (all three directions beeing subject to a 1 bar compressive stress), and the average lateral size L x = L y of the system provides the desired bilayer area A = L x L y . Changes in membrane volume and membrane thickness upon peroxidation are obtained by monitoring the changes in the average simulation box volume L 2

x L z and lateral extension L z . The area of the simulation box and the area of the bilayer can be considered equal, as the excess area caused by undulations is small, especially for systems comprising only 512 lipids. There is however a significant difference between the volume and thickness of the box compared with the volume and thickness of the bilayer only. Let us denote by V b and L b the volume and lateral thickness of the bilayer, and by V and Lz the volume and thickness of the simulation box mentioned in Table 1. The simplest way to relate V and V b or L z and L b consists in removing the volume of the homogeneous solvent taken at same pressure and temperature. The volume of 3072 CG fluid water system is V s = 366.04 ± 0.07 nm [START_REF] Repetto | Lipid Peroxidation[END_REF] . The bilayer volume and thickness follows from the expressions

V b = V -V s ; L b = L z - V b A . (1)
Such an approximation means that change in water structure at the vicinity of bilayers are disregarded, and is known as the Luzzati approximation. [START_REF] Cevc | Phospholipid Bilayers: Physical Principles and Models[END_REF] The resulting bilayer volumes and thicknesses are summarised in Table 2 below. Upon peroxidation, the volume change of the bilayer coincides with the volume change of the simulation box. Relative volume, thickness and area changes are summarised in Table 3. We notice in particular that the structural area variations (∆A/A) str are almost identical for both oxidised lipids. The higher density of hydroperoxide groups in DHP-DOPC compared with HP-POPC does not lead to an increase in area per lipid beyond a trivial change in bilayer volume V b .

Hydroperoxide group distribution

The CG models give also a prediction for lateral density profiles, and in particular for the hydroperoxide group distribution. The partitioning of C 4 OOH between the water interface and the inner hydrophobic region obey a subtle statistical balance. On the one hand the group must show affinity for the interfacial water, on the other hand, the h-bonding donor and acceptor character promotes self-interaction.

The density distribution of an equilibrated coarse grained DHP-DOPC bilayer and HP-POPC bilayer is shown in Fig. 3. Both density distributions were obtained from a simulation of a bilayer patch containing 512 DHP-DOPC/HP-POPC lipids at full hydration (6 CG water beads equivalent to 24 water molecules per lipid) at room temperature (T = 300 K) with the lateral and normal pressures independently coupled to a pressure of 1 bar. The local center of the bilayer was calculated and used in the present work to determine the density as follows: the xy plan of the simulation box was divided into 8 × 8 regions, and the local positions of the bilayer center were calculated as z = z , where the averaging was performed over all lipid monomers in the given region. This position was further used in binning the density profile to shift the bilayer center. and one hydrophilic (P2) referred as polar oxidised, kept at a relative distance of 0.33 nm. The distribution of the polar oxidised beads, representing the OOH side group, is naturally wider than its hydrophobic counterpart which belongs to the lipid tail backbone. A bimodal distribution is seen in both cases. Roughly half of the polar oxidised beads overlap with the glycerol group in the case of HP-POPC, with the other half remaining "buried" at the same place as the corresponding beads of the opposite saturated lipid tails (Fig. 3 above). By comparison, only a quarter of the polar oxidised beads in DHP-DOPC occupy a region close to the glycerol part, the other three quarters staying deep into the hydrophobic region (Fig. 3 below). As DHP-DOPC has twice as many oxidised groups as HP-POPC, it seems that all the extra oxidised beads go into the hydrophobic core of the bilayer, and that the interfacial glycerol region cannot take more oxidised groups than it has in the HP-POPC case. This is consistent with the data in Table 3 suggesting that the structural increase in area per lipid is similar for both species.

Fig. 4 depicts the orientational distribution of the packing angles: the angle between oxidised tail and z-axis α and the angle between two tails of the lipid β . Both DHP-DOPC and HP-POPC have a similar distribution of α, though, judging from the wider distribution, HP-POPC has a larger averaged angle than DHP-DOPC. Peroxidised lipids are therefore significantly tilted with respect to the bilayer normal.

The angle β is defined as the angle between oxidised bead, glycerol bead and mirror alkyl bead in the opposite tail. We find a wide distribution of β for DHP-DOPC with a peak value around 68 • . In turn, the angle distribution of β for HP-POPC is narrower with its peak value around 51 • . The packing angle β is closely related to the effective shape of a lipid molecule and thus it determines the ability of lipids to form a stable bilayer. Wide angles correspond to larger deviations from cylindrical shape, possibly resulting into less stable bilayers.

Elasticity of hydroperoxidised bilayers

The membrane compressibility (or stretching modulus) K A gives the relative area (A) variation consecutive to applied tension (σ ), according to the definition [START_REF] Cevc | Phospholipid Bilayers: Physical Principles and Models[END_REF] :

K A = 1 A dA dσ T,P . (2)
The stretching modulus can be experimentally determined by micropipette aspiration. [START_REF] Rawicz | [END_REF] Values of K A for DHP-DOPC and HP-POPC were published first by Weber et al. 9 and are summarised in the Table 4.

The elastic coefficients K A associated to our coarse-grained numerical models come from an area fluctuation analysis, according to the equilibrium statistical relation

K A = k B T A A 2 -A 2 = k B T 4( L 2 x -L x 2 ) , (3)
with A and A 2 the mean value and the quadratic fluctuation of the sample monolayer area A = L x L y , and L x , L 2 x the mean value and quadratic fluctuation of the lateral size of the anisotropic simulation box. The fluctuations of area result from coupling the system to a semi-isotropic Parrinello-Rahman barostat [START_REF] Parrinello | [END_REF] , known to generate the adequate constant pressure box size fluctuations. It can be shown that the elastic coefficient determined in this way is consistent with the one obtained by exerting an change in lateral stress (tension) on the membrane. The uncertainties quoted in Table 4, are associated with the precision to which L 2

x -L x 2 is determined.

Numerical estimates of K A notoriously depend on sample size. 20 According to den Otter, larger samples appear softer due to long wavelength undulation modes, whose contributions can be accounted for analytically in the relevant limit. 21 Using this analytical relation (cf section 4) a better, undulation-free estimate of the membrane compressibility can be inferred, based on the apparent K A corresponding to two different sample sizes.

We report in Table 4 our values obtained for 512 (small) and 8192 (large) lipid samples, along with the undulation free extrapolation. For DOPC and POPC bilayers, the MARTINI CG model overestimates the accepted experimental values, while large samples (e.g. 8192 lipids in our case) show a (coincidental) better agreement with them. We assume the same trend holds for the hydroperoxidised bilayers. Our investigations suggests that hydroperoxidation leads to a significant decrease of K A in both POPC and DOPC cases, whether the simulated system is small or large. This decrease of K A upon peroxidation appears therefore like a robust and significant trend.

The membrane bending modulus coefficient κ b controls the out-of-plane bending deformation of a membrane in the Canham-Helfrich elastic model. 29 It can be determined experimentally by a number of techniques, including micropipette vesicle aspiration, vesicle flicker motion and nanotube pulling, but unfortunately we are not aware of any published values of κ b to date. We therefore take our values as predictions.

The numerical determination of the bending modulus requires to deal with simulated systems of sufficient size corresponding to a small ratio L b /L x between the membrane thickness and the lateral extension. Only under these conditions does the continuous Helfrich elastic model bring a reasonable description of the out-of-plane membrane fluctuations. Our determination of κ b is based on the equilibrium fluctuation analysis of samples comprising 8192 lipids. Larger systems would approach better the Helfrich continuous limit, but equilibration times tend to increase sharply due to a combination of weak restoring forces and hydrodynamic effects, in spite of the parallel computation capabilities of the simulation tools.

In the continuum model, the bending modulus appears in the quadratic out-of-plane fluctuations spectrum u(q)u(-q) , where q is the wavevector parallel to the membrane plane, and u(x, y) the out-of-plane displacement in the Monge representation (elevation of the continuous surface):

u(q)u(-q) = k B T κ b 1 A q 4 . (4)
Numerical simulations do not provide u(x, y) directly, and there is a certain arbitrariness when it comes to determining u(x, y) from the discrete beads representation of the lipid molecules. At the molecular scale, each lipid fluctuates around the average bilayer position, due to tilt, protrusion and other possible movements. This gives extra-contributions to the structure factor of the bilayer, and eqn (4) is not expected to hold exactly. 30 At smaller q, however, eqn (4) should hold for tensionless bilayers.

We derive our numerical estimate of the bending moduli by fitting U(q)U(-q) to a q -4 behaviour, where U(q) is a discrete beads estimate of the elevation field u(q) 31 (see Fig. 6 and Section 4 for details). Our predictions for κ b are summarised in Table 5. As for the area compressibility (or stretching) coefficient, hydroperoxidation causes a sharp decrease in the bending modulus value. As a result, the elastic bending forces are weaker, and the bilayer undulation dynamics slower.

The POPC, HP-POPC and DOPC undulation spectra all follow the Helfrich q -4 behaviour, while DHP-DOPC undulations does not comply with it so well, showing a weaker appearent exponent. This might be due to DHP-DOPC molecules not fitting perfectly [START_REF] Marsh | Handbook of Lipid Bilayers, 2nd Edition[END_REF] Coarse-graining molecular dynamics is always faster than reality by a significant factor. Only relative trends may be of significance.

J o u r n l N a me , [ y e a r ] , [ v o l . ] , 1-10 | 5 K A (mN/m) K A (mN/m) K A (mN/m) K A (mN/
into the bilayer structure.

Relative changes in diffusion coefficients

We have estimated the diffusion coefficient of each lipid, at 300 K, using the MARTINI coarse-grained dynamics. It is known that coarse-graining predicts faster diffusion than experimentally observed. The extent of the acceleration factor, depending on temperature, may lie between 4 and 10. We observe that the hydroperoxidised lipids diffuse slower than the non-oxidised molecules by roughly 25%. This suggests an increased cohesion between peroxidised molecules, but only by a modest amount. The lateral diffusion coefficient was calculated from the meansquare displacement (MSD) of lipid centres of masses with time. The resulting MSD curves are linear, to a good approximation, over a time interval between 20 and 50 ns (simulation time). Lateral diffusion coefficients were obtained from linear fits of these MSD curves between 20 and 50 ns.

According to Table 6 and Fig. 5, the lateral diffusion coefficient decreases due to lipid oxidation, and we observe that the diffusion coefficient ratio between the oxidised and the non-oxidised species is similar in both cases: 0.77 for HP-POPC/POPC and 0.76 for DHP-DOPC/DOPC. The decrease in diffusion coefficient could arise from a larger friction at the water-bilayer interface, or from stronger lipid-lipid interaction, due to attractive patches in the tails at the position of the peroxide groups. The former interpretation is more consistent with the common 25% drop in value.

Two experimental values, based on fluorescent probes diffusion, are provided to show the expected order of magnitude of the ratio between CG simulated and experimental time scales [START_REF] Marsh | Handbook of Lipid Bilayers, 2nd Edition[END_REF] . This choice is somewhat arbitrary, as a large number of experimental diffusion coefficients values are available, which differ greatly depending on the technique and system preparation. Coarse-Grained models do not perform very well as far as reproducing dynamics and transport properties 32 . We see a reduction of lipid mobility upon peroxidation that seems due to increased friction between peroxidized groups and interfacial water. It might be that important dynamical changes, which are not be accounted for in our approach, alter the overall picture.

Simulation and analysis details

Parameterisation of the CG hydroperoxidised lipids

A first attempt was made by substituting one hydrophobic bead with an hydrophilic bead. Based on log P ow , we find that a moderately hydrophilic bead does not account for the observed area change. Conversely, significant area increase can only be obtained for strongly hydrophilic beads which violate the log P ow requirement. The partition coefficient and the increase in area per lipid can be reconciled by modelling the hydroperoxide group as two partially overlapping beads. A systematic screening was performed with pairs of beads with different polarities (C1 till C3, and P1 till P4) and separations (0.3 till 0.4 nm) and compared to the experimental area per lipids of both HP-POPC and DHP-DOPC. Two dimers (C2-P2) and (C3-P2) emerged as the best contenders based on both partitioning and area per lipid criteria. Both could be used with little changes in the results and we picked-up C3-P2 as reference dimer. To give an idea on how much bead separation affects the area per lipid, we note that a distance 0.32 nm would give area changes of +15.8% (HP-POPC) and +17.7% (DHP-DOPC), as compared with values in Table 3, obtained for 0.33 nm.

Most MARTINI CG beads have a Van der Waals radius σ =0.47 nm, so that a dimer made of two beads at distance r = 0.33 nm gives an excluded volume 50% larger than a single bead. More precisely, if the beads were hard spheres, the combination of two hard spheres of radius 0.47 separated by 0.33 gives a total excluded volume equal to 1.505 times the excluded volume of a single sphere. A direct consequence is that the volume per lipid increases by respectively 5.5% and 10% for POPC and DOPC. No bending potential was added between the C3-P2 bond and the remaining of the lipid chain. Steric interactions ensures that the polar moiety P2 remains at distance from the chain, and additional structural information is lacking to improve over the current parameterisation. One must remember also that our CG model stands for a chemical mixture of at least two chemical compounds, for which the cis or trans nature of the unsaturated double bond is not established even though a majority of trans bonds is expected.

Partitioning of the corresponding C3-P2 dimers was carried out by studying a slab of 1920 octanol molecules (MARTINI "OCO" dimers) surrounded by 2000 "water molecules" (one CG water representing 4 real water molecules), in which 40 dimers were inserted. The concentration ratio was found to be P ow = [C3P2] oct /[C3P2] w = 26, or equivalently logP ow = 1.4. This can, for instance be compared with the estimated logP ow = 0.9 of tertbutyl hydroperoxide ((CH 3 ) 3 COOH, CAS-75-91-2), obtained using with a logPow predicting software (VCCLab 33 ). This indicates that our residue C3-P2 has a realistic hydrophilic-hydrophobic balance.

Stability of peroxidised bilayers

Bilayers composed of single components HP-POPC and DHP-DOPC are stable, to the extent that systems composed of pure HP-POPC and DHP-POPC exhibit stationary properties and show no significant evolution over the simulated times. In practice, initial configurations are obtained by starting from thermalized POPC and DOPC bilayers, inserting the extra beads and reequilibrating again the systems.

Simulation details

This CG model of peroxidised lipids is built on the version 2.0 of MARTINI lipid molecules topologies, and the version 2.1 of the bead interaction parameters. Trajectories were produced using Gromacs v-4.6.

Simulations were carried out at constant temperature (300 K) using the Gromacs v-rescale weak coupling scheme, which is an alternative to Nose-Hoover thermostating (time constant 1 ps). 34 Anisotropic Parrinello-Rahman barostat was used to model a tensionless bilayer (time constant 12 ps, compressibility 3 × 10 -4 bar -1 ). Simulation box size fluctuations are assumed to be consistent with a constant Temperature-Pressure ensemble, and were used as such for the determination of the membrane area compressibility coefficient. Short range electrostatics is assumed, with truncation at a radius of 1.2 nm, while the Verlet neighbour list radius was set to 1.4 nm. Time steps of 30 fs and 40 fs were used throughout, in the absence of rigid constraints, without noticeable differences in the output. Simulation times are summarized in Table 7. Table 7 Trajectories used for the determination of structural parameters.

Size dependence of the elastic compressibility coefficient

Eq. ( 14) in ref 21 relates the apparent compressibility K A to the undulation free compressibility K A , given the system projected area A and the membrane bending modulus κ. It reads

K -1 A = K -1 A + k B T 32π 3 κ 2 A . (5)
This expression is assumes a Canham-Helfrich elastic behaviour, and account for modes of wavelength 2π/ A < q < 2π/λ , with λ a characteristic length of the size of the membrane thickness. If K A ,1 and K A ,2 are known for two different projected areas A ,1 , A ,2 , one can extract an effective bending modulus κ and derive K A from eq. ( 5)

κ k B T 2 = 1 32π 3 k B T A ,1 -A ,2 K -1 A ,1 -K -1 A ,2 (6)
This is a less accurate estimate of κ than fitting the undulation spectrum to a Helfrich q -4 law, as (5) considers modes q ∼ 2π/λ which may deviate from the ideal Helfrich behaviour. The values of κ found in this way are nevertheless reasonably similar to those determined directly, namely 30 kT (POPC), 28.3 kT (POPC), 15.1 kT (HP-POPC) and 12.8 kT (DHP-DOPC). This is 50% larger than values in Table 5, except for DHP-DOPC for which the agreement is better. The related undulation free K A coefficients are Fig. 6 Log-scale representation of the out-of-plane fluctuations of the bilayers. Only q ≤ 0.35 [ln(q) ≤ -1.05] values were used for fitting the q -4 behaviour (q in nm -1 unit). Deviation from a q -4 behaviour can be seen in the DHP-DOPC case, which is the softer among all bilayers.

shown in the third column of Table 4.

Fitting and estimation of errors

Averages of the area, simulation box volume and thickness comes from the thermodynamic data recorded during the simulations, and Gaussian confidence intervals on the sampled means were obtained by using a bunching/boxing algorithm for timecorrelated time sequences. [START_REF] Allen | Computer simulation of liquids[END_REF] The statistical precision on these quantities is very satisfactory.

The bunching algorithm was also used to estimate the accuracy of the sample variance of the statistical L x time sequence. This lead to larger uncertainties for the compressibility coefficients K A . In the case of the undulation free K A , uncertainties on K A dominates the final error.

Modes U(q,t) reflects the Fourier transform of the centre of masses density field. Let us denote (x i , y i , z i ) the space coordinate of the centre of mass of each lipid molecule in one of the bilayer leaflet, and N l the number of lipids in this leaflet. The density field in reciprocal space is given by = ρ (t) + iq z U(q,t)

ρ q = 1 N l N l ∑ i=1 exp(
We therefore simply assume that u(q x , q y ) U(q x , q y ) = N -1 l ∑ i z i exp(iq x x i + iq y y i ).

The undulation modes spectra ln( U(q x , q y )U(-q x , -q y ) ) were fitted to straight lines C -4 ln(q); q = q 2

x + q 2 y , with uncertainties resulting from a standard χ 2 least-squares procedure. [START_REF] Press | Numerical Recipes in C (2Nd Ed.)[END_REF] A similar estimate of the values and confidence intervals for the diffusion coefficients was used, based on the average and variance of the quadratic displacement time signal S(t, τ) of the lipid centers of masses, with respect to t, for fixed τ.

S(t, τ) =

1

N ∑ i (x i (t + τ) -x i (t)) 2 + (y i (t + τ) -y i (t)) 2 . ( 8
)

Conclusion

We presented a coarse-grained model for two hydroperoxidised phospholipids based on the MARTINI force field for POPC and DOPC. The parameters of our coarse-grained molecules were set by combining thermodynamic data (the TBHP partitioning coefficient), molecular packing arguments (the addition of OOH group) and area per lipid changes.

The resulting molecules, HP-POPC and DHP-DOPC, form stable bilayers on accessible simulation times. Our results suggests that the excess area per lipid caused by peroxidation is common to both species, about 13% in relative value. The hydroperoxidised bilayers are thinner and softer than their non oxidised counterparts. A significant drop in compressibility (stretching) coefficient K A is observed, qualitatively similar to experimental values. A similar drop of the bending coefficient κ b is predicted. The diffusion coefficient of the coarse-grained hydroperoxized lipids shows a 25% decrease as compared with the non-oxidised ones.

Using our model, we derive the lateral distribution of peroxide groups in the bilayers, and gain access to various structural data, such as variation of area per lipid, density profile, compressibility and bending moduli, diffusion coefficient of lipids. We hope that new experimental results will appear in a close future that can be compared to the picture presented above.

A question of interest concerns mixtures of regular and peroxidised lipid species, and in particular the possible non-ideal features of such systems. The presence of heterogeneities, if established, would be an element to consider in the understanding of the physiological consequences of lipid peroxidation. We are currently working into this direction, with the help of these newly introduced lipid models.

Acknowledgment

Authors acknowledge funding from Marie Curie Actions under EU FP7 Initial Training Network SNAL 608184. F.T. and Y.G. thanks the Strasbourg High Performance Cluster Equip@Meso for providing computing resources. Y.G would like to express his special thanks to Dr. Olivier Benzerara in Institut Charles Sadron, Strasbourg, France for all the discussion and help. F.T. acknowledges discussions with V. Knecht, M. Batista, A.P. Schroder and C.M. Marques.

a

  Departament d'Enginyeria Quimica, Universitat Rovira i Virgili 26 Av. dels Paisos Catalans, 43007 Tarragona, Spain b Institut Charles Sadron, CNRS and University of Strasbourg, 23 rue du Loess, F-67034 Strasbourg, France. Fax: 33 388414099; Tel: 33 388414097; E-mail: thalmann@ics.u-strasbg.fr

Fig. 1

 1 Fig. 1 Left: addition of reactive singlet oxygen to a chain unsaturation. Right: oxidised lipid molecules based on MARTINI coarse-grained models for POPC and DOPC phospholipids: HP-POPC and DHP-DOPC respectively. Bead types without exception are already present in the MARTINI model, 19 without change of the mutual interaction parameters.

Fig. 2

 2 Fig. 2 Snapshots of the 512 lipid samples: top and middle rows. Distribution of the polar oxidised beads and water regions in HP-POPC bilayers (bottom left) and DHP-DOPC (bottom right).

Fig. 3

 3 Fig. 3 Partial mass densities for HP-POPC (top) and DHP-DOPC (bottom) phosphate group (Green/full line), glycerol (Red/dashed line), Hydrophobic Oxidised (Blue/dotted line), Polar Oxidised (Cyan/dash-dotted line), 3rd hydrophobic group (C3A in the Martini model of POPC) of the opposite saturated tail of POPC (Black/dash-dot-dotted line). The bilayer center lies at z = 0 nm.

Fig. 4

 4 Fig. 4 Left) Definition of the tilting and opening angles α and β . Right) Distribution of orientations of α and β for HP-POPC and DHP-DOPC.

Fig. 5

 5 Fig. 5 Mean lateral displacements ∆x 2 + ∆y 2 versus time, for the four kinds of lipids.

Table 1

 1 Simulation box properties for symmetric bilayers with 512 lipids and 3072 CG water beads, at 1 bar and 300 K.

Table 2

 2 Area/lipid (nm 2 ) 0.6402 ± 0.0008 0.7453 ± 0.0012 0.6698 ± 0.0006 0.8005 ± 0.0016 Volume/lipid (nm 3 ) 1.3789 ± 0.0002 1.4542 ± 0.0002 1.4875 ± 0.0002 1.6357 ± 0.0002 Equilibrium bilayer thickness, area, volume, area per lipid and volume per lipid of POPC and DOPC phospholipids, and corresponding oxidised analogs HP-POPC and DHP-DOPC (512 lipids systems).

					POPC	HP-POPC	DOPC	DHP-DOPC
			Thickness (nm)	4.318 ± 0.006	3.90 ± 0.01	4.442 ± 0.007	4.08 ± 0.03
			Area (nm 2 )	163.9 ± 0.02	190.8 ± 0.3	171.48 ± 0.13	204.9 ± 0.4
			Volume (nm 3 )	706.02 ± 0.08	744.61 ± 0.08	761.65 ± 0.08	837.53 ± 0.09
	Lipid	Volume	Area	Thickness (∆A/A) str Weber et al.
	POPC	5.46%	16.4%	-9.6%	12.8%	15.6%
	DOPC	9.96%	19.5%	-8.1%	12.9%	19.1%

The hydroperoxide group is represented by two connected beads, one hydrophobic (C3) referred as hydrophobic oxidised

Table 3

 3 Relative structural changes of the bilayer following peroxidation of POPC and DOPC molecules. The last column represents the experimental changes in area per lipid reported by Weber et al.9

Table 4

 4 Stretching moduli of POPC, HP-POPC, DOPC, DHP-DOPC. Experimental results for POPC and DOPC bilayers are taken from Rawicz et al,[START_REF] Rawicz | [END_REF] and for HP-POPC and DHP-DOPC from Weber et al.9

		κ b (k B T )	κ b (10 -20 J)	κ b (10 -20 J)
		CG model	CG model	Experimental
	POPC	30.0 ± 1.5	12.4 ± 0.7	9.0 ± 0.6
	HP-POPC	15.1 ± 0.9	6.3 ± 0.4	-
	DOPC	28.3 ± 1.5	11.7 ± 0.7	± 1.0
	DHP-DOPC 12.8 ± 1.2	5.3 ± 0.5	-

Table 5

 5 Bending moduli of POPC, DOPC, hydroperoxidised POPC and DOPC molecules. Experimental data are taken from Rawicz et al.[START_REF] Rawicz | [END_REF]

		D (µm 2 /s)	D (µm 2 /s)
		CG model	Experimental
	POPC	40.2 ± 2.6	4.2 ± 0.4
	HP-POPC	32.2 ± 2.2	-
	DOPC	34.5 ± 1.5	6.3 ± 0.2
	DHP-DOPC 24.4 ± 2.2	-

Table 6

 6 Diffusion coefficients of the numerical CG lipids. Experimental values comes from ref.

  iq x x i (t) + iq y y i (t) + iq z z i (t)) x x i (t) + iq y y i (t))[1 + iq z z i (t)]

	=	1 N l	N l i=1 ∑	exp(iq

[1][START_REF] Alberts | Molecular Biology of the Cell[END_REF][START_REF] Repetto | Lipid Peroxidation[END_REF][START_REF] Frei | [END_REF][5][6][7][8][9][10]

J o u r n a l N a me , [ y e a r ] , [ v o l . ] ,

1-10 J o u r n a l N a me , [ y e a r ] , [ v o l . ] ,


Deniz Engin

Franc ¸ois Schnitzler

Ngoc Q K Duong

Yannis Avrithis

D ) Crippler

On the hidden treasure of dialog in video question answering

High-level understanding of stories in video such as movies and TV shows from raw data is extremely challenging. Modern video question answering (VideoQA) systems often use additional human-made sources like plot synopses, scripts, video descriptions or knowledge bases. In this work, we present a new approach to understand the whole story without such external sources. The secret lies in the dialog: unlike any prior work, we treat dialog as a noisy source to be converted into text description via dialog summarization, much like recent methods treat video. The input of each modality is encoded by transformers independently, and a simple fusion method combines all modalities, using soft temporal attention for localization over long inputs. Our model outperforms the state of the art on the KnowIT VQA dataset by a large margin, without using question-specific human annotation or humanmade plot summaries. It even outperforms human evaluators who have never watched any whole episode before.

Code is available at https://engindeniz.github. io/dialogsummary-videoqa What did the guys name their robot?

Inputs

Kripke is going to name his robot Scrap Metal. Sheldon and Leonard are going to defeat Kripke's robot because theirs is better in design and execution.

Introduction

Deep learning has accelerated progress in vision and language tasks. Visual-semantic embeddings [START_REF] Kiros | Unifying visual-semantic embeddings with multimodal neural language models[END_REF]9] have allowed zero-shot learning, cross-modal retrieval and generating new descriptions from embeddings. Image captioning [START_REF] Vinyals | Show and tell: A neural image caption generator[END_REF] and visual question answering (VQA) [2] have demonstrated generation of realistic natural language description of images and a great extent of multimodal semantic understanding. The extension to video captioning [19,[START_REF] Venugopalan | Sequence to sequence-video to text[END_REF] and video question answering (VideoQA) [29,[START_REF] Lei | TVQA: Localized, compositional video question answering[END_REF] has enabled further progress because video requires a higher level of reasoning to understand complex events [START_REF] Zellers | From recognition to cognition: Visual commonsense reasoning[END_REF].

VideoQA systems typically have similar architecture focusing on multimodal embeddings/description, temporal attention and localization, multimodal fusion and reasoning. While it is often hard to isolate progress in individual components, there are some clear trends. For instance, custom self-attention and memory mechanisms for fusion and rea-Figure 1: In VideoQA, a question is associated with Scene B, but it can only be answered by information from Scene A. We generate episode dialog summaries from subtitles and give them as input to our VideoQA system, dispensing with the need for external knowledge.

soning [START_REF] Na | A readwrite memory network for movie story understanding[END_REF][START_REF] Kim | Multimodal dual attention memory for video story question answering[END_REF][START_REF] Fan | Heterogeneous memory enhanced multimodal attention model for video question answering[END_REF] are gradually being streamlined by using transformer architectures [START_REF] Urooj | MMFT-BERT: Multimodal fusion transformer with bert encodings for visual question answering[END_REF][START_REF] Kim | Modality shifting attention network for multimodal video question answering[END_REF]36]; while visual embeddings [29] are being replaced by semantic embeddings [START_REF] Lei | TVQA: Localized, compositional video question answering[END_REF] and text descriptions by captioning [14,[START_REF] Chadha | iPerceive: Applying common-sense reasoning to multi-modal dense video captioning and video question answering[END_REF].

Datasets are essential for progress in the field, but often introduce bias. For instance, questions from text summaries are less relevant to visual information [29]; super-vised temporal localization [START_REF] Lei | TVQA: Localized, compositional video question answering[END_REF] biases system design towards two-stage localization→answering [START_REF] Lei | TVQA+: Spatio-temporal grounding for video question answering[END_REF][START_REF] Kim | Modality shifting attention network for multimodal video question answering[END_REF]; fixed question structure focusing on temporal localization [START_REF] Lei | TVQA: Localized, compositional video question answering[END_REF] often results in mere alignment of questions with subtitles and matching answers with the discovered context [14], providing little progress on the main objective, which is to study the level of understanding.

Bias can be removed by removing localization supervision and balancing questions over different aspects of comprehension, for instance visual, textual, or semantic [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF]. However, the requirement of external knowledge, which can be in the form of hints or even ground truth, does not leave much progress in inferring such knowledge from raw data [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF]. Even weakening this requirement to plain text human-generated summaries [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF], still leaves a system unusable in the absence of such data.

In many cases, as illustrated in Figure 1, a question on some part of a story may require knowledge that can be recovered from dialog in other parts of the story. However, despite being textual, raw dialog is often informal and repetitive; searching over all available duration of such noisy source is error-prone and impractical. Inspired by the trend of video captioning, we go a step further and apply the same idea to dialog: We summarize raw dialog, converting it into text description for question answering.

Our finding is astounding: our dialog summary is not only a valid replacement for human-generated summary in handling questions that require knowledge on a whole story, but it outperforms them by a large margin.

Our contributions can be summarized as follows:

1. We apply dialog summarization to video question answering for the first time (Subsection 5.1). 2. Building on a modern VideoQA system, we convert all input sources into plain text description. 3. We introduce a weakly-supervised soft temporal attention mechanism for localization (Subsection 6.2). 4. We devise a very simple multimodal fusion mechanism that has no hyperparameters (Section 7). 5. We set a new state of the art on KnowIT VQA dataset [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF] and we beat non-expert humans for the first time, working only with raw data (Section 8).

Related Work

Video Question Answering Progress on video question answering has been facilitated and driven by several datasets and benchmarks. VideoQA by Tapaswi et al. [29] addresses answering questions created from plot synopses using a variety of input sources, including video, subtitles, scene descriptions, scripts and the plot synopses themselves. Methods experimenting on MovieQA focus on memory networks capturing information from the whole movie by videos and subtitles [START_REF] Na | A readwrite memory network for movie story understanding[END_REF][START_REF] Kim | Progressive attention memory network for movie story question answering[END_REF], scene-based memory attention networks to learn joint representations of frames and captions [START_REF] Kim | Multimodal dual attention memory for video story question answering[END_REF], and LSTM-based sequence encoders to learn visual-text embeddings [START_REF] Liang | Focal visual-text attention for visual question answering[END_REF]. TVQA [START_REF] Lei | TVQA: Localized, compositional video question answering[END_REF] and TVQA+ [START_REF] Lei | TVQA+: Spatio-temporal grounding for video question answering[END_REF] address scene-based questions containing temporal localization of the answer in TV shows, using video and subtitles. The questions are structured in two parts: one specifying a temporal location in the scene and the other requesting some information from that location. This encourages working with more than one modalities. Methods experimenting on these datasets focus on temporal localization and attention [START_REF] Lei | TVQA+: Spatio-temporal grounding for video question answering[END_REF][START_REF] Kim | Modality shifting attention network for multimodal video question answering[END_REF], captioning [14, 3] and transformer-based pipelines capturing visual-semantic and language information [36,[START_REF] Urooj | MMFT-BERT: Multimodal fusion transformer with bert encodings for visual question answering[END_REF].

KnowIT VQA [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF] is a knowledge-based dataset, including questions related to the scene, the episode or the entire story of a TV show, as well as knowledge annotation required to address certain questions, in the form of hints. Transformer-based methods are proposed to address this task by employing knowledge annotation [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF] or external human-generated plot summaries [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF]. Our method differs in substituting human-generated knowledge by summaries automatically generated from raw dialog.

Dialog Summarization Dial2Desc dataset [START_REF] Pan | Dial2desc: end-to-end dialogue description generation[END_REF] addresses generating high-level short descriptions from dialog using a transformer-based text generator. SAMSum corpus [START_REF] Gliwa | SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization[END_REF] is a human-annotated dialog summarization dataset providing speaker information. Methods experimenting on this dataset include existing document summarization methods [START_REF] Gliwa | SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization[END_REF], graph neural networks integrating cross-sentence information flow [39] and graph construction from utterance and commonsense knowledge [START_REF] Feng | Incorporating commonsense knowledge into abstractive dialogue summarization via heterogeneous graph networks[END_REF]. Since dialog differs from structured text and requires extraction of the conversation structure, recent work focuses on representing the dialog from different views by sequence to sequence models [START_REF] Chen | Multi-view sequence-tosequence models with conversational structure for abstractive dialogue summarization[END_REF]. We follow this approach.

Overview

We address knowledge-based video question answering on TV shows. Each episode is split in scenes. For each scene, we are given the video (frames) and dialog (speaker names followed by subtitle text) and a number of multiplechoice questions. Certain questions require high-level understanding of the whole episode or show. Garcia et al. [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] rely on human-generated plot summaries (or plot for short), which we use only for comparison. Our objective is to extract the required knowledge from raw data.

As shown in Figure 2, we first convert inputs into plain text description, including both video (by visual recognition) and dialog (by summarization) (Section 5). A number of separate streams then map text to embeddings, at the level of both scene (video and scene dialog summary) and episode (episode dialog summary and plot). The ques- tion and answers are embedded together with the input text of each stream. A temporal attention mechanism localizes relevant intervals from episode inputs. Finally, question answering is addressed both in a single-stream (Section 6) and a multi-stream (Section 7) scenario. The latter amounts to multi-modal fusion. We begin our discussion with transformer networks (Section 4), which we use both for dialog summarization and text embeddings in general.

Transformers

The transformer [START_REF] Vaswani | Attention is all you need[END_REF] is a network architecture that allows for efficient pairwise interaction between input elements. Its main component is an attention function, which acts as a form of associative memory. Multi-head attention is a fusion of several attention functions. The architecture is a stack of multi-head attention, element-wise fullyconnected and normalization layers with residual connections. Originally developed for machine translation, it includes an encoder and a decoder stack. The decoder additionally attends over the output of the encoder stack and is auto-regressive, consuming previously generated symbols when generating the next.

BERT [START_REF] Devlin | BERT: Pre-training of deep bidirectional transformers for language understanding[END_REF] is a transformer bidirectional encoder only, mapping a sequence of tokens to a sequence of ddimensional vectors. It is pre-trained on unsupervised tasks including prediction of masked tokens and next sentence, and can be also fine-tuned on supervised downstream tasks. It can take a number of sentences as in input, where a sentence is an arbitrary span of contiguous text.

We use BERT as the backbone of our model architecture to represent text, using two sentences at a time. Given strings A and B, the input is given as

tok k ([CLS] + A + [SEP] + B + [SEP]), (1)
where + is string concatenation and tok k is tokenization into k tokens, with zero padding if the input length is less than k and truncation if it is greater. Tokens are represented by WordPiece embeddings [START_REF] Schuster | Japanese and korean voice search[END_REF][START_REF] Wu | Google's neural machine translation system: Bridging the gap between human and machine translation[END_REF], concatenated with position embeddings representing their position in the input sequence and segment embeddings, where segments correspond to sentences and are defined according to occurrences of the separator token [SEP]. The output vector in R d corresponding to token [CLS] is an aggregated representation of the entire input sequence and we denote it as

f (A, B). (2)
Sentence-BERT [26] takes a single sentence as input and is trained by metric learning objectives, e.g. in a siamese or triplet structure, facilitating efficient sentence similarity search. It is learned by fine-tuning a pre-trained BERT model on supervised semantic textual similarity.

BART [START_REF] Lewis | BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[END_REF] combines a bidirectional encoder and an auto-regressive decoder. It is pre-trained as an unsupervised denoising autoencoder, i.e., corrupting input text and learning to reconstruct the original, and fine-tuned on supervised classification, generation or translation tasks. It is particularly effective on text generation, including abstractive dialog, question answering and summarization tasks.

Following [START_REF] Chen | Multi-view sequence-tosequence models with conversational structure for abstractive dialogue summarization[END_REF], we use sentence-BERT and BART to segment and summarize dialog respectively.

Input description

All input sources, i.e., video, dialog and plot, are converted into plain text description before being used for question answering. Video is first converted into a scene graph by a visual recognition pipeline and then to text description by a set of rules. Importantly, although already in textual form, dialog is also converted into text description by dialog summarization. The plot, already in text description form, is used as is, but for comparison only: Our main contribution is to replace human-generated plots by automatically generated descriptions.

Dialog

As the main form of human communication, dialog is an essential input source for video understanding and question answering. We use dialog in three ways: raw dialog per scene, dialog summary per scene and the collection of dialog summary over a whole episode.

Raw scene dialog

As in all prior work, we use the raw dialog associated to the scene of the question, as is. Although in textual form, it is not a text description. It may still contain more information than dialog summary, which is important to investigate.

Scene dialog summary Given the dialog associated to the scene of the question, we convert this input source into text description by dialog summarization. Despite being of textual form, dialog is very different from text description: conversations are often informal, verbose and repetitive, with few utterances being informative; while a description is a narrative in third-person point of view with clear information flow structured in paragraphs [START_REF] Chen | Multi-view sequence-tosequence models with conversational structure for abstractive dialogue summarization[END_REF]. Identifying the speaking person is also substantial, especially with multiple people in a conversation. Rather than generic document summarization [START_REF] Gliwa | SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization[END_REF], we follow a dedicated dialog summarization method [START_REF] Chen | Multi-view sequence-tosequence models with conversational structure for abstractive dialogue summarization[END_REF], which blends character names with events in the generated summaries.

A dialog is a sequence of utterances, each including a speaker (character) name and a sentence (sequence of tokens). Each utterance is mapped to a vector embedding by Sentence-BERT [26]. The sequence of embeddings over the entire dialog is segmented according to topic, e.g. greetings, today's plan, etc. by C99 [START_REF] Freddy | Advances in domain independent linear text segmentation[END_REF], as well as stage, e.g. opening, intention, discussion, conclusion by a hidden Markov model (HMM) [START_REF] Althoff | Large-scale analysis of counseling conversations: An application of natural language processing to mental health[END_REF]. As a result, for each view (topic or stage), the dialog is represented by a sequence of blocks, each containing several utterances.

Given the above structure, the input is re-embedded and the summary is generated using an extension of BART [START_REF] Lewis | BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[END_REF]. In particular, there is one encoder per view, mapping each block to an embedding. An LSTM [START_REF] Hochreiter | Long short-term memory[END_REF] follows, aggregating the entire view into one embedding, obtained as its last hidden state. The decoder attends over the output of each encoder using a multi-view attention layer to weight the contribution of each view. It is auto-regressive, using previous tokens from ground truth at training and previously predicted tokens by the encoder at inference.

We train the HMM on the dialog sources of our video QA training set; otherwise, we use Sentence-BERT and BART as used/trained by [START_REF] Chen | Multi-view sequence-tosequence models with conversational structure for abstractive dialogue summarization[END_REF]. Once a scene dialog summary is generated, it is re-embedded by BERT [START_REF] Devlin | BERT: Pre-training of deep bidirectional transformers for language understanding[END_REF] like all other input sources, as discussed in Section 6.

Episode dialog summary We collect the scene dialog summaries for all scenes of an episode and we concatenate them into an episode dialog summary. Assuming that the episode of the scene of the question is known, we make available the associated episode dialog summary for question answering. This is a long input source and requires temporal attention, as discussed in Subsection 6.2. Importantly, episode dialog summary is our most important contribution in substituting plot summary by an automatically generated description.

Plot summary

As part of our comparison to [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF], we use publicly available plot summaries 1 , already in text description form. Assuming that the episode of the scene of the question is known, we make available the associated plot as is, to help answering knowledge-based questions. A plot is shorter and higher-level than our episode dialog summary, but it is still long enough to require temporal attention. It is important to investigate whether we can dispense of such a humangenerated input and how much more information it contains relative to what we can extract automatically.

Video

We use a visual recognition pipeline to convert raw input video into text description. Following [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF], this pipeline comprises four components: character recognition [START_REF] Schroff | FaceNet: A unified embedding for face recognition and clustering[END_REF], place recognition [START_REF] Zhou | Places: A 10 million image database for scene recognition[END_REF], object relation detection [START_REF] Zhang | Largescale visual relationship understanding[END_REF], and action recognition [START_REF] Chao-Yuan | Long-term feature banks for detailed video understanding[END_REF]. The outputs of these components are character, place, object, relation and action nodes. A directed video scene graph is generated by collecting all nodes along with edges and then a textual scene description is obtained according to a set of predefined rules.

Single-stream QA

As shown in Figure 2, there is one stream per input source, using a transformer to map inputs to embeddings. Following [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF], we first attempt question answering on each stream alone. In doing so, we learn a linear classifier while fine-tuning the entire transformer representation per stream. Unlike most existing works, this allows adapting to the data at hand, for instance a particular TV show.

We differentiate scene from episode inputs, as discussed below. In both cases, the given question and candidate answer strings are denoted as q and a c for c = 1, . . . , n c respectively, where n c is the number of candidate answers.

Scene input sources

Scene input sources refer to the scene of the question, i.e., raw scene dialog, scene dialog summary or video. The input string is denoted by x. For each c = 1, . . . , n c , we embed x, q and a c jointly to d-dimensional vector

y c := f (x + q, a c ), (3)
where + is string concatenation and f is BERT (2). A linear classifier with parameters w ∈ R d , b ∈ R yields a score per candidate answer

z c := w ⊤ • y c + b. (4)
The score vector z := (z 1 , ..., z nc ) is followed by softmax and cross-entropy loss. At training, we use f as pre-trained and we fine-tune it while optimizing W, b on the correct answers of the QA training set. At inference, we predict arg max c z c .

Episode input sources

Episode input sources refer to the entire episode of the scene of the question, i.e., episode dialog summary and plot. Because such input is typically longer than the transformer's maximum sequence length k (1), we split it into overlapping parts in a sliding window fashion. Each part contains the question and one answer, so the window length is w = k -|q| -|a c |. Given an input of length ℓ tokens, the number of parts is n := ℓ-w s + 1, where s is the stride. Because all inputs in a mini-batch must have the same number of parts n p to be stacked in a tensor, certain parts are zero-padded if n < n p and discarded if n > n p .

Embedding The input strings of the parts are denoted by p j for j = 1, . . . , n p . Each part p j is combined with each candidate answer a c separately, yielding the d-dimensional vectors y c j := f (p j + q, a c )

for c = 1, . . . , n c and j = 1, . . . , n p . A classifier with parameters w ∈ R d , b ∈ R yields a score per candidate answer c and part j:

z c j := w ⊤ • y c j + b. (6)
Temporal attention At this point, unlike scene inputs (4), predictions from [START_REF] Devlin | BERT: Pre-training of deep bidirectional transformers for language understanding[END_REF] are not meaningful unless a part j is known, which amounts to temporal localization of the part of the input sequence that contains the information needed to answer a question. In TVQA [START_REF] Lei | TVQA: Localized, compositional video question answering[END_REF] and related work [START_REF] Lei | TVQA+: Spatio-temporal grounding for video question answering[END_REF]14,[START_REF] Kim | Modality shifting attention network for multimodal video question answering[END_REF], localization ground truth is available, allowing a two-stage localize-then-answer approach. Without such information, the problem is weakly supervised.

Previous work [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] simply chooses the part j corresponding to the maximum score z c j over all answers c and all parts j in (6), which is called hard temporal attention in the following. Such hard decision may be harmful when the chosen j is incorrect, especially when the predicted answer happens to be correct, because then the model may receive arbitrary gradient signals at training. To alleviate this, we follow a soft temporal attention approach.

In particular, let S be the n p × n c matrix with elements z c j over all answers c and all parts j (6). For each part j, we take the maximum score over answers

s j := max c z c j , (7)
giving rise to a vector s := (s 1 , . . . , s np ), containing a single best score per part. Then, by soft assignment over the rows of S-corresponding to parts-we obtain a score for each answer c, represented by score vector z ∈ R c :

z := softmax(s/T ) ⊤ • S, (8)
where T is a temperature parameter. With this definition of z, we have a single score vector and we proceed as in (4).

Multi-stream QA

Once a separate transformer has been fine-tuned separately for each stream, we combine all streams into a single question answering classifier, which amounts to multimodal fusion. Here, we introduce two new simple solutions.

In both cases, we freeze all transformers and obtain ddimensional embeddings y c for each candidate answer c and for each stream. For scene inputs, y c is obtained directly from (3). Episode input streams produce n p embeddings per answer. Temporal localization is thus required for part selection, similar to single stream training. Again, hard temporal attention amounts to choosing the part with the highest score according to (6): y c := y c j * where j * := arg max j (z c j ) and y c j is given by [START_REF] Freddy | Advances in domain independent linear text segmentation[END_REF]. Instead, similar to (8), we follow soft temporal attention:

y c := softmax(s/T ) ⊤ • Y emb c , (9)
where Y emb c is a n p × d matrix collecting the embeddings y c j (5) of all parts j. Finally, for each answer c, the embeddings y c of all streams are stacked into a n s × d embedding matrix Y c , where n s is the number of streams.

Multi-stream attention

The columns of Y c are embeddings of different streams. We weight them according to weights w c ∈ R ns obtained from Y c itself, using a multistream attention block, consisting of two fully connected layers followed by softmax:

Y att c = diag(w c ) • Y c . ( 10
)
For each answer c, a fully connected layer maps the d × n s matrix Y att c to a scalar score. All n c scores are followed by softmax and cross-entropy loss, whereby the parameters of all layers are jointly optimized.

Self-attention Alternatively, Y c is mapped to Y att c ∈ R d×ns by a single multi-head self-attention block, as in transformers [START_REF] Vaswani | Attention is all you need[END_REF]:

Y att c = MultiHeadAttention(Y c , Y c , Y c ). ( 11
)
The remaining pipeline is the same as in the previous case. Implementation details For scene dialog summary generation, we set the minimum sequence length to 30 tokens and the maximum to 100 in the BART [START_REF] Lewis | BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[END_REF] model. With this setting, episode dialog summaries are 2078 tokens long on average, while plot summaries are 659 tokens long. We fine-tune the BERT BASE [START_REF] Devlin | BERT: Pre-training of deep bidirectional transformers for language understanding[END_REF] uncased model with N = 12 transformer blocks, h = 12 self-attention heads and embedding dimension d = 768 for single-stream models. The maximum token length k is 512 for scene, 200 for plot and 300 for episode dialog summary inputs. The stride s is 100 for plot and 200 for episode dialog summary. The maximum number of parts n p is 10 for both. The batch size is 8 for all single-stream models and 32 for multistream. We use SGD with momentum 0.9 scheduled with initial learning rate 10 -4 for multi-stream fusions. We use h = 1 attention head, and N = 2 stacks for self-attention and multi-stream self-attention methods. The number of streams n s varies per experiment.

Quantitative results

Table 1 compares of our method with the state of the art. Rookies and Masters are human evaluators: Masters have watched most of the show, whereas Rookies have never watched an episode before [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF]. TVQA [START_REF] Lei | TVQA: Localized, compositional video question answering[END_REF] encodes visual features and subtitles without considering knowledge information; its results are as reported in [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF]. ROCK [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF] uses four visual representations (image, concepts, facial, caption); ROCK facial is one of its best results. ROCK GT [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF] and ROLL human [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] use the human knowledge annotation provided by the dataset [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF], while ROLL [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] uses humanwritten plot summaries instead. Our method uses scene video and scene dialog summary as well as the episode dialog summary that it automatically generates, without any human annotation. Ours plot additionally uses the same plot as [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF]. TVQA uses LSTM; all other methods are based on BERT.

Our method outperforms the best state of the art method (ROLL [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF]) by 6.6%, without any human annotation. By using additional human-generated plots, the gain decreases to 5.8%. This indicates that our episode dialog summary captures the required knowledge and removes the requirement of human-generated input; in fact, human-generated input is harmful. On temporal and knowledge questions in particular, we gain 13.9% and 7.6%, respectively, without any human annotation. This implies that our automatically generated episode dialog summary increases the understanding of the episode and helps answering all types of questions. Despite ROLL human [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] and ROCK GT [START_REF] Garcia | KnowIT VQA: Answering knowledge-based questions about videos[END_REF] using ground-truth knowledge, we outperform them by 16.1% and 5.0%, respectively, without any human annotation. We also outperform Rookies, presumably by having access to the dialog of the entire episode. Comparing to Masters, there is still room for improvement.

Qualitative analysis

Figure 3 visualizes the correct predictions of our method with stream attention scores for different question types. In all examples, the model receives three input sources, question/answers and attention scores over inputs. Figure 3(a) shows a knowledge question, answered based on episode dialog summary, which has the highest attention score. As shown in Figure 3(b), a textual question can be answered by using scene dialog summary, but also by episode dialog summary, since the latter includes the former. Temporal questions can be answered from scene inputs such as scene dialog summary or video description. According to attention scores, the question in Figure 3(c) is answered by episode dialog summary, which includes the correct answer. Finally, Figure 3(d) shows a visual question answered by video description.

Ablation studies

Single-stream results Table 2 shows our single-stream QA results. We reproduce [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] for dialog, video, and plot inputs. We replace the plot stream by one using our new temporal attention (Subsection 6.2) and other improvements (Table 4) and we add two new sources automatically generated from dialog: scene dialog summary and episode dialog summary. Due to the dataset having a majority of knowl-

Video Description

Amy doesn't like listening to music in the car. Sheldon doesn't want her to be mistaken for a gang member. Amy loves Neil Diamond's music. edge questions, episode dialog summary and plot inputs have higher accuracy than other input sources since they span an entire episode. Our episode dialog summary helps in answering questions better than the plot [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF], bringing an accuracy improvement of 5.4%.

Scene Dialog Summary

QA

Multi-stream results

We evaluate our two multi-stream QA methods introduced in Section 7, namely multi-stream attention and self-attention, comparing them with the following combinations/baselines/competitors:

1. Multi-stream self-attention: combination of multistream attention and self-attention: the output of the latter is weighted by the former. The remaining pipeline is the same as in multi-stream attention.

2. Product: Hadamard product on embeddings of all streams per answer, followed by a linear classifier per answer. The remaining pipeline is the same. tained scores by single-stream classifiers are combined by a multi-stream classifier and another loss function applies. The overall loss all is a linear combination with weight β ω on the multi-stream loss and 1 -β ω uniformly distributed over single-stream losses.

Table 3 shows results for fusion of video, scene dialog summary and episode dialog summary. For modality weighting, we set β ω = 0.7 according to the validation set. Our multi-stream attention outperforms other fusion methods. Besides, it does not require tuning of modality weight hyperparameter β ω or selecting the number of heads and blocks for self-attention. Unless specified, we use multistream attention for fusion by default.

Improvements over [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] We reproduce ROLL [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] using official code by the authors and default parameters. This is our baseline, shown in the first row of Table 4. Then, we evaluate our improvements, adding them one at a time. First, we replace modality weighting with multi-stream attention. Despite its simplicity, its performance is on par, losing only 0.1%, while requiring no hyperparameter tuning. Then, we increase the number of parts of plot summaries from 5 to 10, eliminating information loss by truncation and bringing an accuracy improvement of 1.1%. We change the order of arguments of BERT for episode input sources from f (q, a c +p j ) to f (p j +q, a c ) (5), which is consistent with (3) and improves only slightly by 0.1%. Our new temporal attention mechanism improves accuracy by 0.9%. Replacing plot with episode dialog summary, which is our main contribution, brings an improvement of 5.1%. Finally, the accuracy is improved by 0.6% by using scene dialog summary instead of raw dialog. The overall gain over [START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] is 7.7%. Note that the relative improvement of each new idea depends on the order chosen in Table 4. For instance, the order of BERT arguments brings improvements of up to 2.3% in experiments including the episode dialog summary.

Conclusion

KnowIT VQA is a challenging dataset where it was previously believed that some form of external knowledge was needed to handle knowledge questions, as if knowledge was yet another modality. Our results indicate that much of this required knowledge was hiding in dialog, waiting to be harnessed. It is also interesting that our soft temporal attention helps a lot more with our episode dialog summary than human plot summary, which may be due to the episode dialog summary being longer. This may also explain the astounding performance of episode dialog summary, despite its low overall quality: plot summaries are of much higher quality but may be missing a lot of information.

Figure 3 :

 3 Figure 3: Multi-stream attention visualization. We highlight in blue the part of the source text that is relevant to answering the question. The most attended stream is episode dialog summary for (a), (b), (c) and video description for (d).

Table 1 :

 1 State-of-the-art accuracy on KnowIT VQA. Ours uses the video and scene dialog summary as well as the episode dialog summary that we generate from the dialog of the entire episode. Ours plot also uses human-generated plot summaries, like[START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF]. TVQA uses an LSTM based encoder; all other methods use BERT. Rookies and Masters are humans.

		METHOD	KNOWLEDGE	VIS.	TEXT. TEMP. KNOW.	ALL
		Rookies [11]	-	0.936	0.932	0.624	0.655	0.748
		Masters [11]	✓	0.961	0.936	0.857	0.867	0.896
		ROCKGT [11]	question GT	0.747	0.819	0.756	0.708	0.731
		ROLLhuman [10] question GT	0.708	0.754	0.570	0.567	0.620
		TVQA [20]	-	0.612	0.645	0.547	0.466	0.522
		ROCKfacial [11]	dataset GT	0.654	0.688	0.628	0.646	0.652
		ROLL [10]	plot	0.718	0.739	0.640	0.713	0.715
		Ours	-	0.755	0.783	0.779	0.789	0.781
		Oursplot	plot	0.749	0.783	0.721	0.783	0.773
	Video Description	Scene Dialog Summary	Episode Dialog Summary	Video Description	Scene Dialog Summary	Episode Dialog Summary
	(...)							(...)
	Sheldon at table. Sheldon sitting on chair. Curtain and building behind Sheldon. Sheldon wearing jacket and shoe. Sheldon holding paper.	Sheldon will send him an email when they get back. He needs to read it. (...)	(...) Sheldon forgot his flash drive, so he has to go back and get it. (...)	Dr. Beverly Hofstadter, Sheldon and Penny are holding a laptop at The main building. (...)	Penny wants Sheldon to go to a coffee shop, but he doesn't drink coffee. (...)	Penny wants Sheldon to go to a coffee shop, but he doesn't drink coffee. She wants him to try some cookies, pastries and bear claws.
	(...)							(...)
	QA	Attention Score			QA		Attention Score
	What has Sheldon forgotten here?			What does Sheldon not drink?
		Video Description					Video Description
	A) His flash drive B) His thesis	Scene Dialog Summary		A) milke B) tea			Scene Dialog Summary
	C) His suitcase	Episode Dialog Summary		C) alcohol			Episode Dialog Summary
	D) His laptop				D) coffee
		(a) Knowledge QA					(b) Textual QA
			(...)
			Amy loves Neil Diamond's
			music. Amy loves Neil
			Diamond's song
			"SWEET CAROLINE".
			(...)

Episode Dialog Summary What singer do Amy and Howard discover they both like?

	(...)
	Window and car behind Amy.
	Howard and Amy.
	Amy wearing glass.
	Amy has hand, arm, nose and hair.
	A) Luke Bryan
	B) Willie Nelson
	C) Neil Diamond
	D) Busta Rhythms

Table 2 :

 2 Single-stream QA accuracy on KnowIT VQA.

		D	0.656 0.772 0.570 0.525 0.584
		V	0.629 0.424 0.558 0.514 0.530
		P	0.624 0.620 0.570 0.725 0.685
		D	0.649 0.801 0.581 0.543 0.598
	ROLL [10] †	V	0.625 0.431 0.512 0.541 0.546
		P	0.647 0.554 0.674 0.694 0.667
		P	0.666 0.623 0.593 0.735 0.702
	Ours	S	0.631 0.746 0.605 0.537 0.585
		E	0.676 0.750 0.779 0.785 0.756
	ROLL [10]: as reported; [10] †: our reproduction. Our
	model incorporates the scene dialog and video streams of
	the latter as well as the plot, scene dialog summary and
	episode dialog summary streams. Plot differs between [10] †
	and our model by our temporal attention and other improve-
	ments (Table 4). D: dialog; V: video; P: plot; S: scene dialog
	summary; E: episode dialog summary.
	METHOD		VIS. TEXT. TEMP. KNOW. ALL
	Product		0.743 0.659 0.756 0.751 0.739
	Modality weighting [10] 0.708 0.786 0.767 0.787 0.769
	Self-attention		0.759 0.764 0.767 0.777 0.771
	Multi-stream attention	0.755 0.783 0.779 0.789 0.781
	Multi-stream self-attn.	0.755 0.768 0.756 0.777 0.770

3. Modality weighting

[START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF]

: a linear classifier (4) and loss function is used as in single-stream QA but with transformers frozen for each stream separately. The ob-

Table 3 :

 3 Multi-stream QA accuracy on KnowIT VQA, fusing video, scene dialog summary and episode dialog summary input sources. All fusion methods use soft temporal attention for localization of episode input sources. Top: baseline/competitors. Bottom: ours.

Table 4 :

 4 METHODVIS. TEXT. TEMP. KNOW. ALL ROLL[START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF] † 0.722 0.703 0.709 0.697 0.704 + Multi-stream attention 0.724 0.721 0.721 0.691 0.703 + More parts for plot 0.722 0.703 0.651 0.717 0.714 + New order of plot inputs 0.730 0.710 0.686 0.712 0.715 Accuracy improvements over ROLL[START_REF] Garcia | Knowledge-based video question answering with unsupervised scene descriptions[END_REF]. †: our reproduction. Each row adds a new improvement except the last two, where we replace streams. P: plot; E: episode dialog summary; D: dialog; S: scene dialog summary.

	+ Temporal attention	0.734 0.725 0.663 0.724 0.724
	± Replacing P → E	0.753 0.815 0.814 0.773 0.775
	± Replacing D → S	0.755 0.783 0.779 0.789 0.781

https://the-big-bang-theory.com/

Acknowledgements This work was supported by the European Commission under European Horizon 2020 Programme, grant number 951911 -AI4Media. This work was granted access to the HPC resources of IDRIS under the allocation 2020-AD011012263 made by GENCI.