Aldo Fontana
email: afontana@science.unitn.it

Laura Orsingher

Giacomo Baldi

F Rossi

in permanently densified GeO 2 glasses

Keywords: Raman spectroscopy, neutron scattering, glass permanently densified glass, vitreous germania vitreous germania, permanently densified glass, Raman light scattering, neutron inelastic scattering

The Raman coupling function in permanently densified

GeO2 glasses

Introduction

The vitreous state is a state of matter intermediate between crystals and liquids. The structure is characteristic of the instantaneous structure of a liquid but the atoms are constrained to vibrate, more or less harmonically, around their disordered quasi-equilibrium sites so that the dynamics is characteristic of a solid. Therefore, it is still possible to define a vibrational density of states g(ω), which describes the vibrational characteristics of a disordered system and determines its thermal properties, as in the case of the crystal. An excess of modes over the Debye law in the low frequency regime, called Boson Peak (BP), has been ubiquitously observed in a wide range of materials both in experiments [1][2][3][4] and in numeric simulations [5][6][7]. Modifications of the Boson Peak take place when a hardening of the elastic medium, due to different varying thermodynamic parameters, occurs. For example, the temperature [8,9], the applied pressure [5,10,11] or the permanent densification [12] lead to an increase of the BP position and a decrease of its intensity.

Therefore, in the case of disordered solids the evaluation of the vibrational density of states g(ω) becomes crucial for the study of the vibrational properties and their connection with thermal anomalies. In general one can experimentally deduce the density of states, DoS, from Raman, neutron and specific heat measurements [13], but all these techniques have nevertheless some drawbacks. In the case of inelastic neutron scattering (INS), the incoherent double scattering cross section is directly related to the DoS. In a one-component systems the onephonon incoherent cross section reads [START_REF] Lovesey | Theory of Neutron Scattering from Condensed Matter[END_REF]:

d 2 σ incoh dΩdE = k k N σ i 8πm g(ω) ω Q 2 e -2W (Q) [n(ω) + 1] (1)
where n(ω, T ) is the Bose-Einstein population factor. For a coherent scatterer like germania, we are obliged to calculate the density of states in the framework of the incoherent approximation. The incoherent approximation assumes that the interference term in the coherent scattering function averages to zero at relatively high Q, so that the dynamic structure factor can be described in terms of an average atom, which scatters only incoherently. This works astonishingly well for the two coherently scattering glasses silica and germania [START_REF] Fabiani | [END_REF]. Moreover, it should be noted that in the case of a glass consisting of two or more elements, this density is called the generalized vibrational density of states, to emphasize that it is not the true density of states, but its reflection in the scattering, weighted by the cross sections and masses of the atoms in the sample.

On the other hand, in the case of Raman scattering, the direct evaluation of the vibrational density of states from the experimental spectra is not straightforward because of the presence of a coupling spectral function, i.e. the Raman coupling function C(ω), which measures the efficiency of the coupling between the incident photon and the vibrations of the system.

The first order Stokes Raman scattering intensity, I exp , for harmonic excitations at a given temperature T, is connected to the vibrational density of states [16] by:

I exp (ω) ∝ n(ω, T ) + 1 ω g(ω)C(ω). (2)
The reduced Raman intensity, I red , is obtained by dividing the experimental spectrum for n(ω, T ) + 1 and ω:

I red (ω, T ) = C(ω)g(ω) ω 2 . ( 3
)
Different theoretical models have been so far proposed for the frequency dependence of C(ω) under different approximations. It should be noted however, that even if all the proposed models predict a power-law frequency behavior for C(ω), a complete understanding of the value of the exponents is still lacking. On the experimental side, the direct comparison between Raman scattering and inelastic neutron scattering measurements is the most faithful procedure to evaluate C(ω). Vitreous germania was prepared by the conventional melt-quenching method. The obtained glass block was cut into cylindrical pieces which were densified using an HT-HP multi-anvil apparatus following a procedure described elsewhere [17]. The samples were compressed at 2, 4 and 6 GPa and then heated at 673 K for 2 minutes under pressure. The densities, ρ, were measured at room temperature by a Micrometrics Accupyc 1300 gas pycnometer under helium gas, yielding the values of 3.99 g/cm 3 , 4.21 g/cm 3 and 4.52 g/cm 3 (23% of densification with respect to the normal density of 3.66 g/cm 3 ) respectively. In the following, we will refer to the samples using the pressure at which they were densified.

Raman scattering experiments were performed on a Jobin-Yvon U-1000 spectrometer in vertical-vertical (VV) and horizontal-vertical (HV) polarizations with an incident light of 514.5 nm. The spectra were collected in a wide frequency range (-200/+1500 cm -1 ) with varying frequency resolution, around 2 cm -1 for the low frequency spectra. Such wide range spectra were necessary to have a proper selfconsistent normalization of the spectra corresponding to different samples. The relative Raman intensity has been adjusted imposing the same integrated intensity in the 860-1000 cm -1 region, i.e. where the high frequency bands are not significantly perturbed by the densification process. In order to study the effects of the densification process from the molecular point of view, high frequency Raman scattering was performed on all the available samples [17]. No strong structural modification has been detected with the densification process. This is in agreement with x-ray-absorption near-edge structure (XANES) measurement of in-situ densified GeO 2 glass, which shows that the amorphous-amorphous transition from a fourfold to a sixfold structure takes place at pressure above 7 GPa [18].

INS experiments were performed at the time of flight spectrometer TOF-TOF at FRM II München using a wavelength of 3 Å, which gives a resolution of 0.3 meV and a Q range of 0.4-4 Å-1 [12]. To exactly match the instrument geometry, the samples were piled in an Al cylindrical container, with a wall thickness of 0.1 mm in order to reduce the empty cell signal. The sample mass was about 2 gr for each density. The inelastic neutron data were analyzed following a standard procedure: subtraction of empty cell, correction for the absorption coefficient, normalization to a reference scatterer. Moreover, with an iterative self-consistent procedure, we accounted for the entire inelastic scattering including the multi-phonon terms and the Debye-Waller factor. The convergence function is the one-phonon density of states.

Results and Discussion

Figure 1 shows the reduced Raman intensity (see Eq.( 3)) for all the samples. The neutron scattering data, published in [12], show that the boson peak undergoes a strong decrease in its amplitude and an increase in its frequency as a function of density. This is confirmed by the present Raman data. This assures the validity of our normalization of the Raman intensity previously described. Moreover, for all the samples, Raman and neutron spectra have a qualitatively similar shape. We now report the ratio between the Raman and neutron scattering for all the samples in order to both evaluate the shape of the coupling function and to confirm the close relation between the two experimental quantities. The C(ω) results to be a linear function of the frequency in the BP region, thus explaining the shift towards high frequencies observed in the BP maximum in Raman scattering with respect to the neutron one. The C(ω) relative to the 6 GPa sample is slightly different, at low frequency, with respect to those of 2 and 4 GPa. This difference could be due to a stronger depolarization of light due to the presence of defects in the sample. The experimental data show that C(ω) extrapolates to a non-vanishing value for ω → 0, confirming previous results reported in the literature [19][20][21]. Taking this result into account we can now turn to a quantitative comparison of the BP evolution as a function of density as measured by the two different experimental techniques. In particular, we rescale the x-axis of the Raman spectra using ν = ω/ω s , where ω s is the squeezing frequency obtained imposing the conservation of the total number of vibrational states, g(ν)dν = g(ω)dω. The intensity has been

then rescaled as [22] I

(ν) = I red (ω) × ω 2 s . (4)
where we took into account the linear dependence of C(ω) in the BP frequency range. The so-squeezed spectra are shown in Figure 3: the spectra rescale one on top of the other using the one scaling parameter ω s . A master curve is obtained. The existence of this master curve demonstrates that the g(ω) is unchanged at all investigated densities, the number of low-frequency modes remains the same and hence the decrease of the BP intensity is only an apparent effect. This confirms the results obtained on the rescaling of the BP in the case of neutron data [12]. The same results is reached if the scaling is performed using g(ω)/ω 2 , as shown in Figure 4. As before, imposing the conservation of the total number of vibrational states, the rescaled intensity is: In both Figure 3 and Figure 4 the curve of the 0 GPa GeO 2 doesn not collapse on the master curve. The reason of this discrepancy has been already discussed in [12].

g(ν)/ν 2 = [g(ω)/ω 2 ] × ω 3 s . (5)

Conclusions

In conclusion, in the present work, we have reported Raman and inelastic neutron scattering data on permanently densified GeO 2 glasses. This study has demonstrated that: i) Raman and neutron spectroscopies give similar information on the dynamics in glasses, in the boson peak frequency range; ii) the densification process does not significantly affect the coupling between the probe and the target; iii) the blue shift of the boson peak with density appears also in vitreous GeO 2 , confirming the results found in v-SiO 2 [20,24].

  It is, however, worth noting that this experimental determination of the coupling function C(ω) is based on the assumption that the generalized vibrational density of states measured with INS closely resembles the true DoS. During the last decade, it has been shown for several glasses that C(ω) seems to follow a linear dependence in the BP frequency range. The aim of the present work is to determine the spectral shape of C(ω) by combined Raman and neutron scattering experiments in permanently densified GeO 2 .

Figure 1 .

 1 Figure 1. Room temperature low frequency HV reduced Raman intensity at different densities as indicated in the legend. The peak intensity reduces as the density is increased.

Figure 2 .

 2 Figure 2. The Raman coupling function C(ω), calculated as the ratio between Raman and Neutron intensities at the same densities of figure 1.

Figure 3 .

 3 Figure 3. Reduced Raman intensity rescaled according to the scaling parameter ωs, at the indicated sample densities.

Figure 4 .

 4 Figure 4. Reduced density of vibrational states rescaled according to the scaling parameter ωs, at the indicated sample densities.

http://mc.manuscriptcentral.com/pm-pml Philosophical Magazine & Philosophical Magazine Letters


MD, MPH Nicolino Ruperto
email: nicolaruperto@ospedale-gaslini.ge.it

MD Angela Pistorio

MD Angelo Ravelli

MD Rachana Hasija

MD Dinara Guseinova

MD Giovanni Filocamo

MD Erkan Demirkaya

MD Clara Malattia

MD Alberto Martini

Prof For

Criteria to define response to therapy in paediatric rheumatic diseases

Keywords: juvenile idiopathic arthritis, juvenile dermatomyositis, and juvenile systemic lupus erythematosus, response to therapy, clinical trials

INTRODUCTION

Performing clinical trials in paediatric rheumatic diseases (PRD) is made difficult by the small number of eligible patients, the heterogeneity of disease manifestations, and the lack of standardized criteria to assess clinical response. These situation has improved over the last 10 years thanks to several factors: the implementation of adequate US and Europe pediatric legislation [START_REF]Regulations requiring manufacturers to assess the safety and effectiveness of new drugs and biologic products in pediatrics patients (21 CFR Parts 201, 312, 314, and 601[END_REF][START_REF]the European parliament and of the Council of 12 December 2006 on medicinal products for paediatric use and amending Regulation (EEC) No 1768/92[END_REF][START_REF] Connor | A look at the future of pediatric therapeutics: an investigator's perspective of the new pediatric rule[END_REF][START_REF] Ruperto | for the Paediatric Rheumatology International Trials Organization (PRINTO). Use of unlabelled and off licence drugs in children. A European paediatric rule is needed to protect children[END_REF], the existence of two large not-for-profit networks which facilitated successful trial completion, named Pediatric Rhematology Trials Organsition (PRINTO at www.printo.it or www.pediatricrheumatology.printo.it) [START_REF] Ruperto | International research networks in pediatric rheumatology: the PRINTO perspective[END_REF] and the Pediatric Rheumatology Collaborative Study Group (PRCSG at www.prcsg.org), the availability of standardized and validated criteria for the evaluation of response to therapy in for the PRD.

Indeed the availability of standardized criteria would provide a common basis for comparing different treatment options, permit study and statistical comparison of patients with different disease manifestations, and facilitate comparisons of different clinical trials with meta-analysis.

The purpose of this paper is to review the general methodology and present the criteria used to define response to therapy in the 3 major PRD: juvenile idiopathic arthritis (JIA), juvenile systemic lupus erythematosus (JSLE) and juvenile dermatomyositis (JDM).

Methodological aspects

One of the specific problem for the PRD is the lack of objective measures that can be reliably used to evaluate response to therapy. Hence the need to use some surrogate clinical and laboratory markers to assess the disease activity level.

In general the methodological approach to solve such problem can be divided into 3 main phases: 1) the development of a preliminary core set of measure to evaluate the outcome under consideration (e.g. response to therapy, remission criteria, disease activity or damage etc) through literature review and consensus techniques; 2) a large scale data collection for a prospectively evidence-based

Consensus methodologies

The use of well recognized consensus formation methodologies specifically designed to combine judgments from a group of experts in a particular field in considered necessary whenever there are no objective measure to properly assess the outcome. The 2 major recognised techniques are the Delphi Technique and Nominal Group Technique [START_REF] Ruperto | Consensus procedures and their role in pediatric rheumatology[END_REF][START_REF] Delbecq | Group Techniques for Program Planning. A guide to nominal group and Delphi processes[END_REF][START_REF] Sniderman | Clinical trials, consensus conferences, and clinical practice[END_REF]. In brief, the Delphi Technique utilizes a series of well defined questionnaire based-surveys while Nominal Group Technique is a structured face-to-face meeting designed to facilitate reaching consensus on the topic of study. Consensus formation techniques require that each step is based on the results of the previous steps. These techniques have been used to develop the outcome measures of several chronic rheumatic diseases, including JIA [START_REF] Giannini | Preliminary definition of improvement in juvenile arthritis[END_REF][START_REF] Ruperto | Performance of the preliminary definition of improvement in juvenile chronic arthritis patients treated with methotrexate[END_REF], adult rheumatoid arthritis [START_REF] Felson | American College of Rheumatology preliminary definition of improvement in rheumatoid arthritis[END_REF], adult-onset SLE [START_REF] Strand | Outcome measures to be used in clinical trials in systemic lupus erythematosus[END_REF][START_REF] Smolen | Randomized clinical trials and longitudinal observational studies in systemic lupus erythematosus: Consensus on a preliminary core set of outcome domains[END_REF], and idiopathic inflammatory myopathies [START_REF] Rider | Defining Clinical Improvement in Adult and Juvenile Myositis[END_REF][START_REF] Rider | International consensus on preliminary definitions of improvement in adult and juvenile myositis[END_REF][START_REF] Oddis | International consensus guidelines for trials of therapies in the idiopathic inflammatory myopathies[END_REF] as well as JSLE [START_REF] Ruperto | Preliminary core sets of measures for disease activity and damage assessment in juvenile systemic lupus erythematosus and juvenile dermatomyositis[END_REF][START_REF] Ruperto | The Pediatric Rheumatology International Trials Organization criteria for the evaluation of response to therapy in juvenile systemic lupus erythematosus: Prospective validation of the disease activity core set[END_REF][START_REF] Ruperto | The Pediatric Rheumatology International Trials Organization/American College of Rheumatology provisional criteria for the evaluation of response to therapy in juvenile systemic lupus erythematosus. Prospective validation of the definition of improvement[END_REF] and JDM [START_REF] Ruperto | set for the evaluation of response to therapy in juvenile dermatomyositis: a prospective validation study[END_REF].

What is a core set of measures and why it is needed (e.g. in pediatric rheumatology)

The core set has to be intended as a minimum list of domains/variables to cover all (or most) aspects of the disease and also as a minimum list to be always reported in future studies. Since in pediatric rheumatology we lack true outcome measures (e.g. death, pneumonitis, glucose etc), there is a need of surrogate markers for the evaluation of the outcome under consideration (in the following paragraph briefly referred as outcome) from different perspectives.

Phase 1. The preliminary core set selection

In this initial phase, after a carefully literature review, it is essential to prepare a mailing of people potentially interested to participate to sequential questionnaire-based e-mail surveys using the Delphi Technique. The mailing list can contains up to several hundreds of people [START_REF] Ruperto | Preliminary core sets of measures for disease activity and damage assessment in juvenile systemic lupus erythematosus and juvenile dermatomyositis[END_REF] and constitutes the proper denominator against which evaluate the success of the survey that usually require a response rate equal to 70-80%. The first questionnaire is generic and is used to evaluate, in broad term, which are the variables used in routine clinical practice to assess the outcome. The data from the first survey are then analysed and collated to prepare the second one in which the content is more specific. In this second survey responders can be asked to concentrate only on the variables selected by a minimum threshold of people in the first survey, to select the most important variable (e.g. usually no more than 10) and rank them in order of importance. Other surveys if necessary can be added.

Usually the results of the mailing survey are not sufficient to select the final list of variables to be used. In this case the results of the survey, along with a proper literature search can be used to convene a nominal group technique consensus conference, among a restricted number of people whose goal is indeed to reach the consensus (80% required) on the measures to be included in the preliminary core set. Usually the number of person per table (max 2-3 tables) should not be greater than 10.

Phase 2. The prospective validation of the preliminary core set(s)

The second phase is aimed to formally validate the preliminary consensus based core sets through a prospective, large-scale data collection among the members. The objective of this phase is to further define and validate the preliminary core set variables to document the outcome. A detailed protocol, listing the proper inclusion/exclusion criteria, needs to be set up in order to collect data ideally in a prospective fashion. Data collection should try to mimic what is usually done in a clinical trial, with a baseline assessment at the time of the starting treatment, and a final assessment after a certain follow-up period. Data collection should include all variables in the preliminary consensus based core set, plus all additional variables that are deemed necessary in the analysis phase such as demographic, clinical and laboratory data, treatment options etc. Following the OMERACT filter for outcome measures in rheumatology [21,[START_REF] Bellamy | Clinimetric concepts in outcome assessment: The OMERACT filter[END_REF] the main purposes of the validation phase are to evaluate the following properties [START_REF] Ruperto | Preliminary core sets of measures for disease activity and damage assessment in juvenile systemic lupus erythematosus and juvenile dermatomyositis[END_REF]: feasibility (brevity, simplicity, easy scoring etc); face and content validity (variables easiest to use and most credible); responsiveness (ability of each variable to detect clinically important change between baseline and 6 months) measured through the standardized response mean (SRM) [START_REF] Liang | Comparisons of five health status instruments for orthopedic evaluation[END_REF]; discriminative ability (ability to discriminate patients who improved from those who did not); convergent construct validity to examine whether the construct in question is related to other measures in a manner consistent with a priori prediction; collinearity (or redundancy); internal consistency by Cronbach's coefficient alpha [START_REF] Cronbach | Coefficient alfa and the internal structure of tests[END_REF]; finally, the association between the core measures and response to therapy as judged by the attending physician can be evaluated through a multivariate logistic regression analysis.

Phase 3. The definition of improvement

At the end of the data collection process the related analysis should be presented to a panel of expert (or experienced clinicians) knowledgeable about the outcome, in another nominal group technique consensus based meeting. The purposes of the meeting is to discuss the statistical properties of the measures and confirm/refine the final core set of measures. Another important goal is to establish the amount of change over time, for the variables included in the core set, in order to properly quantify the outcome under consideration (e.g. criteria for response to therapy). The meeting should therefore properly define the final core set of endpoints, using a combination of statistical criteria and consensus formation techniques.

THE EXAMPLES IN PAEDIATRIC RHEUMATOLOGY.

The core sets of measure for JIA, JSLE and JDM.

In table 1 are reported the domains and the suggested variable to measures each domain for JIA [START_REF] Giannini | Preliminary definition of improvement in juvenile arthritis[END_REF][START_REF] Ruperto | Performance of the preliminary definition of improvement in juvenile chronic arthritis patients treated with methotrexate[END_REF], JSLE [START_REF] Ruperto | Preliminary core sets of measures for disease activity and damage assessment in juvenile systemic lupus erythematosus and juvenile dermatomyositis[END_REF][START_REF] Ruperto | The Pediatric Rheumatology International Trials Organization criteria for the evaluation of response to therapy in juvenile systemic lupus erythematosus: Prospective validation of the disease activity core set[END_REF][START_REF] Ruperto | The Pediatric Rheumatology International Trials Organization/American College of Rheumatology provisional criteria for the evaluation of response to therapy in juvenile systemic lupus erythematosus. Prospective validation of the definition of improvement[END_REF] and JDM [START_REF] Ruperto | set for the evaluation of response to therapy in juvenile dermatomyositis: a prospective validation study[END_REF]. Variables suggested include the measures which at the present The domains are divided in the common domains (to be used for all 3 diseases) and specific domains.

The common domains for JIA, JSLE and JDM include 2 global assessment, by the physician and by the parents/patients, usually measured with a 10 cm visual analogue scale (VAS) or a 21 circle VAS [START_REF] Filocamo | Comparison of 21-numbered circle and 10-centimeter horizontal line visual analog scales for physician's and parent's subjective ratings in juvenile idiopathic arthritis[END_REF]. Additionally a measure of functional ability (disability) through the Childhood Health Assessment Questionnaire (CHAQ) [START_REF] Singh | Measurement of health status in children with juvenile rheumatoid arthritis[END_REF][START_REF] Editors | Quality of life in juvenile idiopathic arthritis patients compared to healthy children[END_REF][START_REF] Ruperto | Cross-cultural adaptation and psychometric evaluation of the Childhood Health Assessment Questionnaire (CHAQ) and the Child Health Questionnaire (CHQ) in 32 countries. Review of the general methodology[END_REF] or the Juvenile Arthritis Functionality Scale (JAFS) [START_REF] Filocamo | Development and validation of a new short and simple measure of physical function for juvenile idiopathic arthritis[END_REF] and/or a quality of life measures is also considered usually through the Physical Summary score (PhS) of the Child Health Questionnaire (CHQ) [START_REF] Ruperto | Cross-cultural adaptation and psychometric evaluation of the Childhood Health Assessment Questionnaire (CHAQ) and the Child Health Questionnaire (CHQ) in 32 countries. Review of the general methodology[END_REF][START_REF] Landgraf | The CHQ User's Manual[END_REF] or the Pediatric Rheumatology Quality of Life Scale (PRQL) [START_REF] Filocamo | A new short and simple health-related quality of life measure for paediatric rheumatic diseases: initial validation in juvenile idiopathic arthritis[END_REF].

Disease specific domains for JIA includes 2 measures of joint involvement (the number of joints with active arthritis and the number of joints with limited range of motion) [START_REF] Cassidy | A study of classification criteria for a diagnosis of juvenile rheumatoid arthritis[END_REF][START_REF] Ruperto | Redundancy of conventional articular response variables used in juvenile chronic arthritis clinical trials[END_REF][START_REF] Ravelli | Correlation between conventional disease activity measures in juvenile chronic arthritis[END_REF][START_REF] Bazso | Development and testing of reduced joint counts in juvenile idiopathic arthritis[END_REF] and an index of inflammation. For systemic JIA children, absence of spiking fever (< 2 days of maximal body temperature ≤ 38°C during the week preceding the evaluation) is an additional requirement.

The JSLE specific domains include a global disease activity assessment, either by the Systemic Lupus Erythematosus Disease Activity Index (SLEDAI) [START_REF] Bombardier | Derivation of the SLEDAI. A disease activity index for lupus patients. The Committee on Prognosis Studies in SLE[END_REF] or the Systemic Lupus Activity Measures (SLAM) [START_REF] Liang | Reliability and validity of six systems for the clinical assessment of disease activity in systemic lupus erythematosus[END_REF] or the European Consensus Lupus Activity Measurement (ECLAM) [START_REF] Vitali | Disease activity in systemic lupus erythematosus: report of the Consensus Study Group of the European Workshop for Rheumatology Research. II. Identification of the variables indicative of disease activity and their use in the development of an activity score[END_REF][START_REF] Vitali | Development of a clinical chart to compute different disease activity indices for systemic lupus erythematosus[END_REF] and a measure of kidney involvement (24-hour proteinuria).

Disease specific domains for JDM include a measure of muscle strength as assessed by the Childhood Myositis Assessment Scale (CMAS) [START_REF] Lovell | Development of validated disease activity and damage indices for the juvenile idiopathic inflammatory myopathies -II. The Childhood Myositis Assessment Scale (CMAS): a quantitative tool for the evaluation of muscle function[END_REF][START_REF] Rennebohm | Normal scores for nine maneuvers of the childhood myositis assessment scale[END_REF][START_REF] Huber | Validation and clinical significance of the childhood myositis assessment scale for assessment of muscle function in the juvenile idiopathic inflammatory myopathies[END_REF] or the Manual Muscle Testing (MMT) [START_REF] Hicks | Validation of manual muscle testing (MMT) in the assessment of juvenile dermatomyositis (JDM)[END_REF]; a global disease activity assessment through the Disease Activity Score (DAS) [START_REF] Bode | Disease activity score for children with juvenile dermatomyositis: Reliability and validity evidence[END_REF] or alternatively the Myositis Disease Activity Assessment (MDAA, an instrument [START_REF] Isenberg | International consensus outcome measures for patients with idiopathic inflammatory myopathies. Development and initial validation of myositis activity and damage indices in patients with adult onset disease[END_REF] that combines two partially overlapping tools named the Myositis Disease Activity Assessment Visual Analogue Scale (MYOACT) and the Myositis Intention to Treat Activity Index A-E version (MITAX) [START_REF] Isenberg | International consensus outcome measures for patients with idiopathic inflammatory myopathies. Development and initial validation of myositis activity and damage indices in patients with adult onset disease[END_REF]. Overall the measures include in the 3 core sets were found to be feasible and not redundant, to have good construct validity, discriminative ability, internal consistency, were responsiveness to clinically important change in disease activity, and were associated strongly with treatment outcome [START_REF] Giannini | Preliminary definition of improvement in juvenile arthritis[END_REF][START_REF] Ruperto | Performance of the preliminary definition of improvement in juvenile chronic arthritis patients treated with methotrexate[END_REF][START_REF] Ruperto | Preliminary core sets of measures for disease activity and damage assessment in juvenile systemic lupus erythematosus and juvenile dermatomyositis[END_REF][START_REF] Ruperto | The Pediatric Rheumatology International Trials Organization criteria for the evaluation of response to therapy in juvenile systemic lupus erythematosus: Prospective validation of the disease activity core set[END_REF][START_REF] Ruperto | The Pediatric Rheumatology International Trials Organization/American College of Rheumatology provisional criteria for the evaluation of response to therapy in juvenile systemic lupus erythematosus. Prospective validation of the definition of improvement[END_REF][START_REF] Ruperto | set for the evaluation of response to therapy in juvenile dermatomyositis: a prospective validation study[END_REF].

The definition of improvement for JIA, JSLE and JDM.

In table 2 are reported the definitions of improvement to be used as main outcome in trials JIA [START_REF] Giannini | Preliminary definition of improvement in juvenile arthritis[END_REF][START_REF] Ruperto | Performance of the preliminary definition of improvement in juvenile chronic arthritis patients treated with methotrexate[END_REF], JSLE [START_REF] Ruperto | Preliminary core sets of measures for disease activity and damage assessment in juvenile systemic lupus erythematosus and juvenile dermatomyositis[END_REF][START_REF] Ruperto | The Pediatric Rheumatology International Trials Organization criteria for the evaluation of response to therapy in juvenile systemic lupus erythematosus: Prospective validation of the disease activity core set[END_REF][START_REF] Ruperto | The Pediatric Rheumatology International Trials Organization/American College of Rheumatology provisional criteria for the evaluation of response to therapy in juvenile systemic lupus erythematosus. Prospective validation of the definition of improvement[END_REF] and JDM [START_REF] Ruperto | set for the evaluation of response to therapy in juvenile dermatomyositis: a prospective validation study[END_REF]. The definitions require a minimum level of improvement (30% in JIA, 50% in JSLE and 20% in JDM) in x number of the variables included in the core sets (Table 1)

with no more than one of the remaining variables worsened by more than 30%. For example the JIA definition of improvement for JIA is: at least 30% improvement from baseline in 3 of any 6 variables in the core set with no more than one of the remaining variables worsening by more than 30%. For trials in systemic JIA absence of spiking fever (< 2 days of maximal body temperature ≤ 38°C during the week preceding the evaluation) is also required. The minimum level of improvement has to be reported in all trials as primary outcome but researchers should also report more stringent definition of improvement (50, 70, 90 or 100%) as secondary outcome in the trial results.

The definition of flare for JIA

An innovative trial design in JIA is the use of the so-called randomized, double-blind controlled, withdrawal design. This study design was proposed by Giannini and Lovell for use in pediatric rheumatology studies [START_REF] Giannini | Intravenous immunoglobulin in the treatment of polyarticular juvenile rheumatoid arthritis: a phase I/II study[END_REF] with the rational to avoid the placebo arm for children with chronic A peculiarity of this trial design is therefore the need to have a proper definition of flare to be used in the double-blind withdrawal phase. The flare definition currently used in JIA is essentially the inverse of the ACR Pediatric 30 criteria for response [START_REF] Brunner | Preliminary definition of disease flare in juvenile rheumatoid arthritis[END_REF]. This definition requires that there be at least a 30% worsening in three of the six JIA core set parameters, with no more than one improving by more than 30%. In addition, there are some contingencies requiring that, if either the number of active joints or the number of joints with limitation of motion are used in the flare definition, there must be at least a two joint increase in the number of active joints or those with limited range of motion, respectively. Also, if the physical or parent global scales are used in the flare definition, worsening ≥ 2 cm (on a 0-10 cm scale) is required. Additionally, for systemic JIA patients only, reappearance of spiking fever (> 38°C, lasting for at least 2 days in the week preceding the evaluation) not due to infections would signify that a flare have occurred. These flares criteria have been validated and shown to be sensitive, in terms of limiting the degree of the worsening patient symptoms to avoid potential damage in subjects randomized to placebo and those children losing response to the experimental agent.

The definition of remission for JIA

An additional definition available for JIA is the CARRA/PRINTO/PRCSG criteria for inactive disease [START_REF] Wallace | Preliminary criteria for clinical remission for select categories of juvenile idiopathic arthritis[END_REF] that requires the absence of active arthritis, fever, rash, serositis, splenomegaly, or generalized lymphadenopathy attributable to JIA, active uveitis; normal ESR or CRP; and a physician's global assessment of disease activity rated at the best score possible for the instrument used. The presence 6 continuous months of inactive disease defines clinical remission on medication, while 12 months of inactive disease off all anti-arthritis medications defines clinical remission off medication.

Final remarks and future perspective

The criteria for the evaluation of response to therapy in PRD were designed to ensure that certain minimum criteria/standards are applied to observational and randomized controlled trials and to facilitate comparison with future meta-analysis. Furthermore, they can assist in standardizing outcome measurements in daily clinical practice. The core set includes the indication of instruments that can be used to assess each domain. However the specific instrument can be modified or integrated whenever new valid tools or better laboratory indicators will be developed to measure a particular domain or more suitable for use in the paediatric age. The proposed core sets also combine aspects of the disease that can be measured easily in clinical practice and indeed only measures that can be assessed reliably world-wide were included.

In

F o r P e e r R e v i e w

Response to therapy in pediatric rheumatology. Part of TEDDY Supplement Ruperto et al 12

  preliminary findings; 3) the final development of a validated criteria for the evaluation of the outcome under consideration.

  have the better statistical properties; however in the future other measures appearing in the literature might replace the existing ones if they will demonstrate better statistical properties.

  Response to therapy in pediatric rheumatology. Part of TEDDY Supplement Ruperto et al 9

  conditions when alternative effective treatments are available. Eligible children are treated in an open label fashion with the experimental therapy to be tested in the trial for a few months after which responders (typically those demonstrating an ACR pediatric 30 response) are randomized in a double-blind fashion either to continue the experimental therapy or to switch to placebo. In this Response to therapy in pediatric rheumatology. Part of TEDDY Supplement Ruperto et al 10 segment of the study, called the double-blind withdrawal phase, patients who demonstrate a predefined definition of disease worsening (e.g. "flare") are withdrawn from the double-blind withdrawal phase of the trial and re-treated with the experimental therapy in an open label fashion.

  conclusion the proposed core sets and definitions of improvement incorporate clinically meaningful change in a composite endpoint for the evaluation of global response to therapy in the major PRD. The definitions are proposed for use in PRD clinical trials, and may help physicians to decide if a child has responded adequately to therapy. ACKNOWLEDGMENTS. "This contribution is part of the Task-force in Europe for Drug Development for the Young (TEDDY) Network of Excellence supported by the European Commission's Sixth Framework Program (Contract n. 0005216 LSHBCT-2005-005126)"We are indebted to PRINTO national coordinators and to the members of the PRINTO and of the PRCSG networks, whose enthusiastic participation made these projects possible. The project has been supported by grant from the European Union (contract no. BMH4983531-CA, no. QLG1-CT-2000-00514), by IRCCS G. Gaslini, Genoa, Italy, and by the National Institute of Health (Grant RO3 AI 44046). Dr Dinara Guseinova, is a recipient of a EULAR scholarship.

Table 1 :

 1 Core sets of measure to evaluate response to therapy in JIA, JSLE and JDM. In the first column are reported the overall domain. In the following columns the suggested variables to measure each specific domain in the 3 diseases.

	DOMAINS*	Juvenile idiopathic arthritis (JIA)	Juvenile systemic lupus	Juvenile dermatomyositis (JDM)
			erythematosus (JSLE)
	COMMON DOMAINS
	Global assessment by physicians	1) Physician's global assessment	1) Physician's global assessment	1) Physician's global assessment
		of disease activity on a 10-cm	of disease activity on a 10-cm	of disease activity on a 10-cm
		Visual analogue scale (VAS)	VAS or a 21 circle VAS	VAS or a 21 circle VAS
		or a 21 circle VAS
	Global assessment by	2) Patient/parent's assessment of	2) Patient/parent's assessment of	2) Patient/parent's assessment of
	parents/patients	overall well-being or a 10-cm	overall well-being or a 10-cm	overall well-being or a 10-cm
		VAS or a 21 circle VAS	VAS or a 21 circle VAS	VAS or a 21 circle VAS
	Disability or Health Related	3) Childhood Health Assessment	3) Physical Summary score	3) CHAQ or JAFS
	Quality of Life	Questionnaire (CHAQ) or the	(PhS) of the Child Health	4) CHQ-PhS or PRQL
		Juvenile Arthritis	Questionnaire (CHQ) or the
		Functionality Scale (JAFS)	Pediatric Rheumatology
			Quality of Life Scale (PRQL)
	DISEASE SPECIFIC DOMAINS
	Disease specific measures	4) Number of active joints		5) Childhood Myositis Assess-
		5) Number of joints with limited		ment Scale (CMAS) or
		range of motion		Manual Muscle Testing
				(MMT)
	Global disease activity		4) European Consensus Lupus	6) Disease Activity Score (DAS)
			Activity Measurement	or Myositis Disease Activity
			(ECLAM) or the Systemic	Assessment (MDAA)
			Lupus Activity Measures
			(SLAM) or the SLEDAI
	Laboratory assessment	6) Erythrocite sedimentation rate	5) 24-hour proteinuria
		(ESR) or C-reactive protein
		(CRP)
	Additional	7) Spiking fever (to be used just
		for systemic JIA)

Table 2 :

 2 Definitions to evaluate response to therapy in JIA, JSLE and JDM. Contingencies for JIA flare definition: if either the number of active joints or the number of joints with limitation of motion are used in the flare definition, there must be at least a two joint increase, respectively. Also, if the physical or parent global scales are used in the flare definition, worsening ≥ 2 cm (on a 0-10 cm scale) is required. Additionally, for systemic JIA patients only, reappearance of spiking fever (> 38°C, lasting for at least 2 days in the week preceding the evaluation) not due to infections would signify that a flare have occurred.

	Juvenile idiopathic arthritis	Juvenile systemic lupus	Juvenile dermatomyositis
	(JIA)	erythematosus (JSLE)	(JDM)
	Definition of improvement
	At least 30% (*) improvement	At least 50% (*) improvement	At least 20% (*) improvement
	from baseline in 3 of any 6	from baseline in any 2	from baseline in 3 of any 6
	variables in the core set	among the 5 core set	core set domains
		measures
	with no more than one of the	with no more than 1 of the	with no more than 1 of the
	remaining variables	remaining worsening by	remaining worsening by
	worsening by more than	more than 30%.	more than 30%, which
	30%.		cannot be muscle strength
	For systemic JIA absence of
	spiking fever (< 2 days of
	maximal body temperature ≤
	38°C during the week
	preceding the evaluation) is
	also required
	Definition of flare
	at least a 30% worsening in 3
	of the 6 JIA core set
	parameters, with no more than
	one improving by more than
	30%. (**)
	(*) report as secondary outcome also 50%-70%-90%-100% improvement
	(**)

European Journal of Clinical Pharmacology


PhD Milos Hauskrecht

MSc Michal Valko

MSc Iyad Batal

MD Gilles Clermont

MD Shyam Visweswaran

MD, PhD Gregory F Cooper

Conditional Outlier Detection for Clinical Alerting

We develop and evaluate a data-driven approach for detecting unusual (anomalous) patient-management actions using past patient cases stored in an electronic health record (EHR) system. Our hypothesis is that patient-management actions that are unusual with respect to past patients may be due to a potential error and that it is worthwhile to raise an alert if such a condition is encountered. We evaluate this hypothesis using data obtained from the electronic health records of 4,486 post-cardiac surgical patients. We base the evaluation on the opinions of a panel of experts. The results support that anomaly-based alerting can have reasonably low false alert rates and that stronger anomalies are correlated with higher alert rates.

Introduction

Despite numerous improvements in the health care practice, the occurrence of medical errors remains a persistent and serious problem [START_REF] Lt Kohn | To err is human: Building a safer health system[END_REF][START_REF] Starfield | Is US health really the best in the world?[END_REF]. The urgency and the scope of this problem prompt the development of solutions aimed to aid clinicians in eliminating such mistakes. Current computer tools for monitoring patients are primarily knowledge-based; the ability to monitor depends on the knowledge represented in the computer and extracted a priori from clinical experts. Unfortunately, these systems are time consuming to build and their clinical coverage is not complete. This paper describes a new monitoring and alerting framework we have developed that relies on stored clinical information of past patient cases and on statistical methods for the identification of clinical outliers (anomalies). Our conjecture is that the detection of anomalies corresponding to unusual patient management actions will help to identify medical errors. We believe that such an approach can complement the use of knowledge-based alerting systems, thereby improving overall clinical coverage of alerting. In clinical sub-areas where knowledgebased alerting is not yet available, this new approach can serve as a standalone system. Typical anomaly detection attempts to identify unusual data instances that deviate from the majority of examples in the dataset [START_REF] Chandola | Anomaly Detection -A Survey[END_REF]. Our objective is different; we want to identify anomalies in patient management actions (decisions), where individual actions depend strongly on the condition of the patient. We develop a new detection framework to identify outliers in such settings. Our framework builds upon the conditional outlier approach [START_REF] Hauskrecht | Evidence-based anomaly detection[END_REF] that aims to identify patient management actions for a given patient that are highly unusual with respect to past patients and the condition he/she suffers from. Once a deviation is detected it may be used to generate a patient-specific alert.

To illustrate the idea, consider a post-surgical cardiac patient treated by heparin for the past six days whose platelet count fell below 100 and whose platelet count is decreasing. This patient may have heparin induced thrombocytopenia (HIT), a life threatening condition, which, if not treated properly, may lead to thrombosis and death. Assuming the platelet drop cannot be explained by other causes (e.g., hemorrhaging), a special HPF4 lab assay confirming HIT would typically be ordered. If ordering an HPF4 test is a typical action in such a situation, then seeing no HPF4 test order for such a patient is anomalous, and thus, it becomes the basis for raising an alert for consideration by the clinician(s) who are caring for the patient.

Ideally we would like to have all statistical outliers correspond to medical errors. Hence their detection would lead to a useful clinical alert. However, the reality is that a patient management action that is unusual from a statistical viewpoint does not always correspond to a helpful clinical alert. Nevertheless, our belief (and hypothesis) is that unusual patient management actions that can be found by anomalydetection methods will flag errors often enough to be worth alerting on. We report here our investigation of the relationship between statistical outliers and clinically useful alerts. We conducted experiments on retrospective data obtained from electronic health records (EHRs) of 4,486 post-cardiac surgical patients and by having a subset of the 222 alerts raised by our system evaluated by a panel of 15 experts in critical care. We show that statistical anomaly detection is positively correlated with clinically meaningful alerts, which provides support for our hypothesis that anomaly-based alerting can be clinically useful.

Methodology

This section presents a framework for identifying patient-management anomalies using data from past patients and for using them to generate patient specific alerts.

Conditional anomaly detection

Let x define a vector of attributes (representing a patient's state) and y a patient-management action that is associated with it. Our goal is to decide if the action y is unusual for the patient condition x, by taking into account records for past patients in the database. Our approach for indentifying conditional anomalies consists of three steps: (1) segmentation and transformation of temporal information in EHRs, (2) building of predictive models for different patient management actions, and (3) prospective, application of conditional anomaly detection to identify highly unusual actions in a given patient over the course of that patient's clinical care.

Segmentation and transformation of the EHR

The data in patient medical records consist of complex multivariate time series combining results of lab tests, information about medication orders, procedures performed, diagnoses made, events encountered, and other information. In order to build an outlier detection model for patient-management actions, we first segment the data in every medical record according to discrete segmentation points and convert the time series data observed up to these segmentation points into a vector space representation of patient states at that time. Each patient-state instance is also linked to a vector of patient-management actions (decisions) made in between the current and the next segmentation point. This approach lets us generate multiple patientstate/action pairs per patient. Figure 1 illustrates the idea using 24-hour segmentation; the EHR for case A is segmented into three patient-state instances. Patient management actions are linked to the patient state features that preceded them.

Vector space representation of the patient state

The segmentation divides the patient case into multiple patient-case instances. These instances cover different hospitalization periods and the amount of information in each of them may vary. To make our models independent of these variations, we use a vector space representation of the patient-state, where each state is defined using a fixed set of features and their values. The features attempt to summarize temporal information for each patient instance, and may include features such as last blood glucose measurement, last glucose trend value, and the total time the patient was most recently treated on heparin [START_REF] Hauskrecht | Evidence-based anomaly detection[END_REF][START_REF] Vapnik | The Nature of Statistical Learning Theory[END_REF]. The feature-based representation of time series data is flexible and various features can be built into the model. In this work we use three sets of features representing: laboratory tests (LABs) and their values, medications (MEDs), and procedures performed. We now briefly describe the features generated for each of these categories: Lab Features. For labs with categorical results (e.g., POS/NEG) we used the following 7 features: last value; second to last value; first value; time since last value; a pair of indicators of the test being performed and pending orders. For labs with continuous or ordinal values we used a richer set of features, including features such as the difference between last two measured values, the slope in between the last two values, and their percentage drop/increase. Figure 2 illustrates a subset of features generated for such labs. In this case, the total number of features generated is 26 and it includes features for nadir, apex, baseline values and their differences from the last measured values. Nadir and apex values are the lab values with the smallest and the greatest value recorded up to that point, respectively.

Medication Features.

For each medication we used four features: 1) indicator if the patient is currently on the medication, 2) time since the patient was put on that medication for the first time, 3) time since the patient was last on that medication, and 4) time since last change in the order of the medication.

Procedure Features. Procedure features capture information about procedures such as heart-valve repair. We record three features per procedure: 1) an indicator of whether the procedure has ever been performed during the current hospitalization, 2) the time since the procedure was first performed, and 3) the time since the procedure was last performed.

Representation of patient-management actions

We want to build predictive models that capture the relationship between the patient state and various follow-up patient-management actions. We consider two types of actions: (1) lab order actions with (true/false) values reflecting whether the lab was ordered or not, (2) medication actions with (true/false) values reflecting if the patient was given a medication or not. The values for all action are summarized in an action vector that is linked to the corresponding patient-state vector.

Learning a predictive model

To assess the conditional anomaly of an action y for a patient instance x, we first build (learn) a predictive model of data representing a conditional distribution P(y | x). We can use such a model to determine if an actual action (or a planned action that is recorded) is unusual. Intuitively, the action y is anomalous given x, if P(y | x) is small. We can construct the predictive model P(y | x) using a variety of Machine Learning (ML) methods. In the work reported here we use the support vector machine (SVM) [START_REF] Sollich | Probabilistic methods for support vector machines[END_REF]. The SVM is a discriminative learning model that is popular in the machine learning community primarily thanks to its ability to learn high-quality discriminative patterns in highdimensional datasets. It learns a decision boundary f(x) = 0 that optimally separates examples into two classes. Hence, it does not directly output a probabilistic model of P(y | x). However, multiple probabilistic transformations of the SVM model are possible [START_REF] Ar | Temporal data mining[END_REF]. We adopt the method by Platt in which

P(y=1 | x) = 1/(1+exp(A f(x) +B)),
where A, B are parameters fitted on the training data.

Feature selection

The total number of features in the vector-space patient-state representation is typically huge. It is not feasible to use all these features when learning predictive models. First, a particular action is likely to be influenced by only a relatively limited set of clinical variables (lab values, medications) while other clinical information is often irrelevant for that action. Second, if all features are incorporated into the classification model via parameters, then a high variance of these parameter estimates may negatively influence the quality of the model's predictions. To reduce this effect, a simpler model with a smaller number of features, and hence a smaller number of parameters, is often desirable. We approach the problem by building one model per action, and by selecting features for the model greedily and in groups, where groups are defined by different labs (e.g., hemoglobin features), medications (e.g., features for penicillin) or procedures (e.g., features linked to heart valve replacement). The quality of the features in the same group (e.g,. hemoglobin features) is determined by learning a multivariate model with these features only and by analyzing its predictive power via internal cross-validation in terms of the Area under the Receiver Operating characteristic (AUC) curve, which is equivalent to the Wilcoxon-Mann-Whitney statistic [START_REF] Hanley | The meaning and use of the area under a receiver operating characteristic (ROC) curve[END_REF]. The groups are then combined incrementally into the model, by starting from the best group and greedily adding new feature groups into the model if they lead to the improvement of model's predictive performance. The process stops when no improvement is possible.

Anomaly detection

Predictive models P(y | x) built above capture how well patient instances predict different patientmanagement responses. The model can be applied to a (new) patient instance x and its associated action y to see how different the actual action taken for the patient is from the predicted action.

Anomaly scores. We propose to measure the anomalousness of y given x in terms of an anomaly score Anom(x, y). The most straightforward option is to define the score as Anom(x, y) = 1-P(y | x). Intuitively, the action y is anomalous given x, if P(y | x) is small or, equivalently, when 1 -P(y | x) is large. Other anomaly scores are possible, and we are in the process of exploring them as well.

Outlier detection for clinical alerting

The above anomaly score allows us to measure deviations from usual patient-management patterns. Our hypothesis is that these deviations are often clinically important and may correspond to patientmanagement errors; hence they are worthwhile to be alerted on. But how do we use anomaly scores to alert on data encountered prospectively? The caveat of applying our predictive models to detect anomalies prospectively is that all predictive models relate a patient state x with actions that follow it. Hence, a deviation in the patient management can be assessed only at the end of the follow-up period. But, by that time the patient state may change. To address the problem, we analyze each patient management action with respect to the patient state before and after the follow-up interval and assess their anomalousness. If the action is anomalous for both, then an alert is raised. We implement this idea in terms of the alert score. Let x t be the patient instance at time t (current time), x t-1 be the patient instance in the previous time step (t-1), and y t-1 be the action regarding a lab order or a medication order in between times t-1 and t. We define the score for alerting on action y t-1 given patient state x t at time t as:

Alert(x t , y t-1 ) = min(Anom(x t-1 , y t-1 ,), Anom(x t , y t-1 ,)).

Briefly, the score reflects: (1) the degree of deviation of the action y t-1 as defined by the anomaly score, and (2) the persistence of a high anomaly signal in two consecutive time steps: (t-1) and t. If the score is above a threshold, an alert is raised at time t.

Evaluation study

We evaluated our framework and its ability to raise clinically useful alerts using the data from archived EHRs for 4,486 post-surgical cardiac patients treated at a large teaching hospital in the Pittsburgh area. These EHRs were first divided into two groups: a training set that included all cases seen before 2005, and a test set that included the cases seen on or after 2005. Second, we used the time-stamped data in each EHR to segment the record at 8:00am every day to obtain multiple patient case instances, as illustrated in Figure 1. These patient instances were then converted into: (1) a vector-space representation of the patient state using the feature transformation described in the methodology section, and (2) a vector representation of lab-order and medication actions with true/false values, reflecting whether the lab was ordered or the medication was given within a 24-hour period. The segmentation led to 51,492 patient-state instances, such that 30,828 were used for training the model and 20,664 were used in the evaluation. The vector space representation of the patient state (at any point in time) included 9,282 generated features for 335 labs, 407 medications, and 36 procedure categories. Rare labs, medications and procedures (used in less than 20 patients) were excluded. Additional features we used were demographics (sex, age, race) and indicators of the presence of four heart-support devices. The action vectors linked with each patient-instance included 335 lab-order and 407 medication decisions.

Learning anomaly detection models. The training set was used to build three types of anomaly detection models: (1) models for detecting unexpected lab-order omissions, (2) models for detecting unexpected medication omissions, (3) models for detecting unexpected continuation of medications (commissions).

Selection of alerts for the study. The alerts for the evaluation study were selected as follows. We first applied all the above anomaly detection models to matching patient instances in the test set. The following criteria were then applied. First, only models with AUC of 0.68 or higher were considered. This means that many predictive models built did not qualify and were never used. Second, the minimum anomaly score for all alert candidates had to be 0.15 or lower. Third, for each action, only the strongest 125 anomalies and the strongest 20 alerts obtained from the test data were considered as alert candidates. This lead to 3,768 alert candidates, from which we selected 222 alerts for 100 patients, such that 101 alerts were lab-omission alerts, 55 were medicationomission alerts, and were 66 medication-commission alerts. Figure 3 shows the distribution of alerts in the study according to the alert score. Alert reviews. The alerts selected for the study were assessed by physicians with expertise in post-cardiac surgical care. The reviewers (1) were given the patient cases and model-generated alerts for some of the patient management actions, and (2) were asked to assess the clinical usefulness of these alerts. We recruited 15 physicians to participate in the study, of which 12 were fellows and 3 were faculty from the Departments of Critical Care Medicine or Surgery. The reviewers were divided randomly into five groups, with three reviewers per group, for the total of 15 reviewers. Overall, each clinician made assessments of 44 or 45 alerts generated on 20 different patients. The survey was conducted over the internet using a secure web-based interface [START_REF] Schedlbauer | What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior?[END_REF].

Alert assessments. The pairwise kappa agreement scores for the groups of three ranged from 0.32 to 0.56. We used the majority rule to define the gold standard. That is, an alert was considered to be useful if at least two out of three reviewers found it to be useful. Briefly, out of 222 alerts selected for the evaluation study, 121 alerts were agreed upon by the panel (via the majority rules) as useful alert.

Analysis of clinical usefulness of alerts.

We analyzed the extent to which the alert score from a model was predictive of it producing clinically important alerts. Figure 4 summarizes the results by binning the alert scores (in intervals of the width of 0.2) and presenting the true alert rate per bin. The true alert rates vary from 19% for the low alert scores to 72% for high alert scores, indicating that higher alert scores are indicative of higher true alert rates. This is also confirmed by (1) a positive slope of the line in Figure 4, which was obtained by fitting the results via linear regression, and (2) the results of the ROC analysis. Briefly, all alerts reviewed were ordered according to their alert scores, from which we generated an ROC curve. The AUC for our alert score was 0.64. This is statistically significantly different from 0.5, which is the value one expects to see for random or non-informative orderings. Again, this supports that higher alert scores induce better true alert rates. Finally, we would like to note that alert rates in Figure 4 are promising and despite alert selection restrictions, they compare favorably to alert rates of existing clinical alert systems [START_REF] Schedlbauer | What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior?[END_REF][START_REF] Bates | Ten commandments for effective clinical decision support: Making the practice of evidence-based medicine a reality[END_REF].

Conclusions

We presented a new framework that detects conditional anomalies in patient management actions and raises an alert when an anomaly is found. This work differs from previous studies we reported in [START_REF] Hauskrecht | Evidence-based anomaly detection[END_REF][START_REF] Valko | Conditional anomaly detection methods for patient-management alert systems[END_REF][START_REF] Visweswaran | Identifying deviations from usual medical care using a statistical approach[END_REF], by building and testing the outlier detection models on thousands of patient features and hundreds of actions extracted from EHRs, and by building alerting models prospectively monitoring the patient case in time. It is the most comprehensive and complete evaluation of our conditional outlier methodology for detecting unusual patient management decisions we have conducted so far.

Our results show that outlier detection is a promising methodology for raising clinically useful alerts. In the future we plan to further investigate the framework by eliminating some of the restrictions placed on the selection of alerts in the study, and by using (1) new temporal features that characterize the clinical data, (2) different time discretizations, (3) relations in between actions, and (4) additional evaluations that contain different datasets in different clinical domains.

Figure 1 .

 1 Figure 1. Processing of data in the electronic health record: (1) segmentation of an EHR into multiple patient-state/action instances, (2) transformation of these instances into a vector space representation of patient states and their follow-up actions.

Figure 2 .

 2 Figure 2. Examples of temporal features for continuous lab values: Last value: A, Last value difference = A-B, Last % change = (A-B)/B, Last slope = (A-B) / (t A -t B ), Nadir = D, Nadir difference = A-D, Nadir % difference = (A-D)/D.

  Distribution of alert examples in the evaluation study

Figure 3 .

 3 Figure 3. Histogram of alert examples in the study according to their alert score.

Figure 4 .

 4 Figure 4. The relation between the alert score and the true alert rate. The height of the bins shows true alert rates for alert-score intervals of width 0.2. The line is fitted via linear regression.

Acknowledgements

We would like to thank Drs. Andrew Post and James Harrison for their PROTEMPA case review interface. This research work was supported by grants R21LM009102, R01LM010019, and R01GM088224 from the NIH. Its content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.


Although the restless legs syndrome (RLS) is a common cause of sleep disruption and the prevalence of this condition in primary care is high (15-25%) [START_REF] Nichols | Restless legs syndrome in primary care[END_REF] , most physicians have remained unaware of the disorder. Many patients have remained undiagnosed or misdiagnosed and have not been managed adequately with a subsequently negative impact on sleep and quality of life [START_REF] Hening | Impact, diagnosis and treatment of restless legs syndrome (RLS) in a primary care population: the REST (RLS epidemiology, symptoms and treatment) primary care study[END_REF][START_REF] Allen | Restless legs syndrome. A review of clinical and pathophysiologic features[END_REF][START_REF] Bayard | Restless Legs Syndrome[END_REF] . The RLS diagnostic is clinical and the 4 criteria are: (1) the patient must have an urge to move the legs; (2) symptoms must be aggravated by rest; (3) symptoms must be alleviated by movement, especially by walking; and (4) symptoms must be worse in the evening or night (circadian rhythm) [START_REF] Bayard | Restless Legs Syndrome[END_REF][START_REF] Gamaldo | Restless legs syndrome: a clinical update[END_REF] . This syndrome may be the result of dopamine and iron system abnormalities and the role on RLS pathophysiology has been supported by demonstrated therapeutic benefit, although these relationships are not fully understood [START_REF] Khalid | Refractory restless legs syndrome likely caused by olanzapine[END_REF][START_REF] Kang | Restless legs syndrome and periodic limb movements during sleep probably associated with olanzapine: case series and discussion[END_REF][START_REF] Bahk | Mirtazapine have the propensity for developing a restless legs syndrome? A case report[END_REF] .

A 45 year old woman, with no past medical history, was diagnosed with major depressive disorder and was prescribed 5mg olanzapine and 5mg aripiprazole daily. After ten months the olanzapine was discontinued and the aripiprazole was increased to 15mg daily. Her depressive symptoms were improved but few days after this dosage increase she presented unpleasant aching sensation on both legs with creeping and crawling feelings during the night constantly interrupting the sleep; these symptoms appeared about 3-4 hours after the medication. She had no sudden involuntary muscles contractions, previous legs trauma or akathisia. The symptoms relieved by moving the legs or by walking and they were mild during the early morning. There was no history of caffeine intake or any other medication that may exacerbate this condition.

Physical examination showed no abnormalities and no clinical signs of anemia; peripheral neuropathy or vascular diseases were not found. A diagnosis RLS was made. Lab test were done looking for secondary causes of RLS (iron, ferritin, creatinine, BUN, thyroid hormones, glucose and rheumatoid factor). All findings were normal. Even though aripiprazole was effective for her depressive symptoms it was gradually discontinued causing the RLS symptoms disappearance.

The dopamine agonists have demonstrated their efficacy in the treatment of the RLS. This class of drugs directly stimulates dopamine receptors and has a longer half-life (four to six hours) than levodopa (90 minutes). They are generally superior to levodopa for the treatment of daily RLS. The mechanism by which these dopamine agonists improve the symptoms of RLS is the stimulation of dopamine D 2 receptors.

Aripiprazole and olanzapine are atypical antipsychotics, acting as partial agonist and antagonist of the dopamine D 2 receptor, respectively. Aripiprazole has not been associated with RLS in other case reports [START_REF] Khalid | Refractory restless legs syndrome likely caused by olanzapine[END_REF][START_REF] Kang | Restless legs syndrome and periodic limb movements during sleep probably associated with olanzapine: case series and discussion[END_REF][START_REF] Bahk | Mirtazapine have the propensity for developing a restless legs syndrome? A case report[END_REF] , where antidepressants and antipsychotics induced RLS and it is the first singularity distinguished this case. In theory the difference in the mechanism of action may explain why RLS was not seen with aripiprazole. If antidepressants are necessary, the symptoms of secondary RLS can usually be treated in the same way as primary RLS [START_REF] Nichols | Restless legs syndrome in primary care[END_REF][START_REF] Hening | Impact, diagnosis and treatment of restless legs syndrome (RLS) in a primary care population: the REST (RLS epidemiology, symptoms and treatment) primary care study[END_REF][START_REF] Allen | Restless legs syndrome. A review of clinical and pathophysiologic features[END_REF][START_REF] Bayard | Restless Legs Syndrome[END_REF] . Bupropion is an alternative antidepressant that may be less likely to induce or worsen RLS [START_REF] Bayard | Restless Legs Syndrome[END_REF][START_REF] Gamaldo | Restless legs syndrome: a clinical update[END_REF] . Dopamine antagonist have been shown to exacerbate RLS and the second particularity is the absence of RLS related with olanzapine like others case reports [START_REF] Khalid | Refractory restless legs syndrome likely caused by olanzapine[END_REF][START_REF] Kang | Restless legs syndrome and periodic limb movements during sleep probably associated with olanzapine: case series and discussion[END_REF] when the symptoms resolved after this drug was discontinued.

The third peculiarity, RLS was not developed during the initial prescription of aripiprazole and it appeared during the increase of the dose. It could suggest a probably dose dependent appearance of the RLS associated to aripiprazole. The classification of RLS is shown in table 1.

We suggest future research should focus the antipsychotic induced RLS.

Table 1 Classification of RLS

 1

	Primary	Secondary
	Idiopathic	Iron deficiency
		Pregnancy especially third trimester
		Uremia
		Peripheral neuropathies
		Thyroid disease
		Diabetes
		Rheumatoid arthritis
		Fibromyalgia
		Venous insufficiency
		Spinocerebellar ataxia
		Drugs
		Metoclopramide
		Phenytoin
		Antipsychotics
		Occasionally tricyclic antidepressants
		SSRI, lithium


Ana Macedo

Magí Farré

Josep-E Baños
email: josepeladi.banos@upf.edu

Sir,

We have read with attention the letter of Dr. Tfelt-Hansen regarding our article published in the journal which considered the placebo effect in migraine using meta-analytic techniques 1 . We consider that his comments will help to clarify some of the topics included in our paper. However, we would like to comment some specific points to improve the understanding of our contributions by future readers.

Tfelt-Hansen raises some worries about the definition of headache relief. He states the need of accepting that headache relief should be defined as the decrease in its intensity from moderate [START_REF]Committee for medicinal products for human use (CHMP). Guideline on clinical investigation of medicinal products for the treatment of migraine[END_REF] or severe [START_REF] Tfelt-Hansen | Guidelines for controlled trials of drugs in migraine: second edition[END_REF] to none (0) or mild [START_REF] Macedo | A meta-analysis of the placebo response in acute migraine and how this response may be influenced by some characteristics of clinical trials[END_REF]. In fact, in our article we wrote that 'pain relief means a pain reduction higher than 50%". In the case of severe pain, a pain reduction of 50% is a decrease in headache intensity from severe (3) to mild [START_REF] Macedo | A meta-analysis of the placebo response in acute migraine and how this response may be influenced by some characteristics of clinical trials[END_REF] or none (0). For moderate pain, a decrease in headache intensity to mild (1) or none (0) is considered as a success. Therefore, we consider that our definition of "higher of 50%" is not different from the suggested by Tfelt-Hansen. However, we accept that his definition is clearer than us and also according the Guideline of EMEA for the investigation of medicinal products for the treatment of migraine 2 and the International Headache Society Clinical Trials Subcommittee 3 . But, for practical purposes, it is almost equivalent.

The author also suggests that pain free at 2 h is increasingly used as primary efficacy measure. We fully agree with that point of view. In fact, in all tables of our article 1 we showed pain free and pain relief variables to give the readers the possibility of comparing both of them. The first can be considered as a hard outcome and the second as a soft outcome. However, it is unquestionable that the main objective of treatment should be to leave patients pain free but pain relief can not be considered as a minor issue in clinical practice. As expected, the magnitude of placebo effect was higher when the last was used and decreased when the latter was considered. Unfortunately, although pain free is desirable it is not always achieved nor in clinical practice neither in clinical trials. For instance, in a recent study in acute migraine 4 , the pain freedom at 2h ranged from 17.2% to 31.3% with active treatments, and it was 9.6% with placebo. Values of pain relief at 2 h were from 49.8%% to 56.4% and only 27.7% with placebo. These values were very similar to those obtained in our meta-analysis, i.e. 28.6% of pain relief and 8.8% of pain freedom.

The main objective of our paper was to analyze the significance of placebo effect differences according with clinical trials methodology independently of all the other variables (disease, population and even outcomes). If we assume that methodology makes differences in placebo results, then we might also assume that some aspects associated to clinical practice may influence the therapeutic results of any given treatment in migraine. We consider that our objective was reached but we acknowledge contributions like that of Dr. Tfelt-Hansen which helps to clarify further our results.