Thomas Pradeu

Eric Vivier

The discontinuity theory of immunity

Some biological systems detect the rate of change in a stimulus rather than the stimulus itself only. We suggest that the immune system works in this way. According to the discontinuity theory of immunity, the immune system responds to sudden changes in antigenic stimulation and is rendered tolerant by slow or continuous stimulation. This basic principle, which is supported by recent data on immune checkpoints in viral infections, cancers, and allergies, can be seen as a unifying framework for diverse immune responses.

Key bullet points:

• Like many other biological systems, the immune system can be seen as a changedetection system.

• The immune system detects sudden modifications, but is rendered tolerant to slow changes or chronic antigen expression.

• This view is supported by many recent data and opens up new avenues for research and treatment.

• Further studies of the timing of the immune response, at the interface between immunology and biomathematics, are required.

The discontinuity theory of immunity in the context of change-detection systems in biology

Imagine a toad sitting in the close vicinity of a fly. Whilst the fly remains still, the toad does not move, but as soon as the fly moves, the toad snaps at it. This natural pattern of behavior, which can be observed directly, indicates that the toad detects rapid changes in a visual stimulus, rather than the stimulus itself only. Many biological systems behave in a similar fashion, not perceiving the stimulus itself only, but detecting the rate of change in the stimulus, a feature related to Weber's law [START_REF] Casci | Signalling: Sensing a change[END_REF][START_REF] Trautmann | From kinetics and cellular cooperations to cancer immunotherapies[END_REF]. According to this law, which was first proposed in 1905, our visual perception of a given stimulus depends not on the absolute level of the stimulus, but on its magnitude relative to background levels. It has recently been shown that several cellular signaling pathways, including the ERK2 response to EGF and the influence of β-catenin on Wnt signaling, involve the detection of changes in a stimulus [START_REF] Casci | Signalling: Sensing a change[END_REF].

Here, we argue that change-detection may also be a basic organizing principle underlying the workings of the immune system, constituting a pillar of the "discontinuity theory of immunity" [START_REF] Pradeu | The speed of change: towards a discontinuity theory of immunity?[END_REF]. According to this theory, the immune system detects antigenic variations (Δa) over time (Δt): the Δa/Δt derivative. Consequently, the immune system can detect sudden changes, leading to the triggering of effector immune responses. Such a concept has been pioneered for adaptive immune cells by Grossman and Paul for decades [START_REF] Grossman | Adaptive cellular interactions in the immune system: the tunable activation threshold and the significance of subthreshold responses[END_REF][START_REF] Grossman | Dynamic tuning of lymphocytes: physiological basis, mechanisms, and function[END_REF]. The discontinuity theory of immunity applies to both innate and adaptive immunity. Importantly, the immune system can also adapt to slow or long-lasting modifications in the host, which it then treats as a new reference point. Thus, effector immune responses are induced by sudden antigenic changes, whereas chronic exposure to antigens leads to immune tolerance (3). High, but continuous antigen concentrations and low levels of variation of antigen concentration (Δa/Δt → 0) would, therefore, be predicted to lead to immune tolerance (Table 1). We explore the effects of chronicity, for which large amounts of data have recently been published, in more detail here.

Recent experimental evidence supporting the discontinuity theory of immunity

The timing of exposure to a given immune stimulus, such as an antigen, plays a major role in the triggering of the immune response (4-8), together with other factors, such as the dose and molecular structure of the antigen. The discontinuity theory stems from the demonstration that chronic exposure to immune stimuli can desensitize both adaptive (9) and innate immune cells [START_REF] Tripathy | Continuous engagement of a self-specific activation receptor induces NK cell tolerance[END_REF][START_REF] Wang | Lack of CD47 on nonhematopoietic cells induces split macrophage tolerance to CD47null cells[END_REF]. This theory is supported by an array of recent experimental evidence in three pathological contexts: viral infections, cancers, and allergies. In all these contexts, the immune effector response declines when the immune system is confronted with the chronic presence of an immune stimulus. The tolerogenic mechanisms at work in cases of persistent exposure of the immune system to an antigen have been dissected at length by many groups and may be intrinsic or extrinsic to the immune cell (Table 1). Intrinsic mechanisms include cell surface receptor internalization (e.g., antigen receptors expressed by T and B lymphocytes are internalized upon antigen recognition) and the degradation of key downstream signaling elements. Other cell intrinsic regulatory circuits include the engagement of co-inhibitory receptors that dampen cell activation. These receptors include immune receptor tyrosine-based inhibition motif (ITIM)containing cell surface molecules such as Killer cell Ig-like receptors (KIRs), TIGIT, and CD47, and so-called immune checkpoints such as PD-1, CTLA-4, Lag-3, and Tim-3 [START_REF] Anderson | Lag-3, Tim-3, and TIGIT: Co-inhibitory Receptors with Specialized Functions in Immune Regulation[END_REF]. Although these receptors can be expressed in many different immunological situations (e.g., PD-1 is expressed on functional follicular T helper cells), they are often induced upon sustained cell activation, illustrating a molecular mechanism by which chronic stimulation leads to the desensitization of immune effectors [START_REF] Barber | Restoring function in exhausted CD8 T cells during chronic viral infection[END_REF]. Tolerogenic mechanisms can also be cell-extrinsic, as exemplified by the production of immunoregulatory cytokines, such as IL-10 or TGF-β, by regulatory T cells induced by persistent antigenic stimulation [START_REF] Virgin | Redefining chronic viral infection[END_REF]. Exposure to tissue autoantigens leads to the activation of self-reactive regulatory T cells. These cells persist in tissues and suppress autoimmune responses [START_REF] Rosenblum | Response to self antigen imprints regulatory memory in tissues[END_REF]. Continuous T-cell receptor signaling is critical for both the development of regulatory T cells in the thymus and for the maintenance and function of these cells in the periphery [START_REF] Vahl | Continuous T Cell Receptor Signals Maintain a Functional Regulatory T Cell Pool[END_REF].

In chronic viral infections, the immune system often deteriorates to "exhaustion" (16), a state in which T lymphocytes are first primed and develop effector functions, but then prolonged stimulation leads to a gradual loss of function over time [START_REF] Virgin | Redefining chronic viral infection[END_REF]. In chronic lymphocytic choriomeningitis virus infections in mice, cytotoxic lymphocytes adapt to the persistent infection via a positive feedback pathway between PD-1 and the transcription factor FoxO1, leading to their desensitization [START_REF] Staron | The Transcription Factor FoxO1 Sustains Expression of the Inhibitory Receptor PD-1 and Survival of Antiviral CD8+ T Cells during Chronic Infection[END_REF]. This concept has also been clinically validated for tumors, in which chronic exposure to tumors also induces exhaustion [START_REF] Pauken | Overcoming T cell exhaustion in infection and cancer[END_REF]. Immune checkpoints prevent overt activation of the immune system and protect normal tissues from collateral damage. Tumor cells often evade immune attack, as the ligands for these inhibitory receptors are expressed by the tumor cells themselves or by cells in the tumor microenvironment. There has been a recent paradigm shift in cancer immunotherapy, with the introduction of treatments based on antibodies called immune checkpoint inhibitors, which block the interaction between checkpoints and their ligands. These inhibitors make immune cell activation possible, by blocking a dominant inhibitory mechanism.

Immune checkpoint inhibitors have been shown to be effective and to have manageable safety profiles for several types of cancer [START_REF] Topalian | Immune Checkpoint Blockade: A Common Denominator Approach to Cancer Therapy[END_REF]. The demonstration that such treatments can abolish immunological tolerance to the tumor indicates that it is possible to re-introduce discontinuity in tumor antigen perception.

Another confirmation of the induction of immune tolerance by chronic antigen exposure has been provided by advances in allergen immunotherapy over the last decade, with some degree of desensitization being induced by repeated allergen injections in patients. This method has proved effective for allergic rhinitis, allergic asthma, and food allergies [START_REF] Jutel | International consensus on allergy immunotherapy[END_REF].

In many situations, immune tolerance is thus induced by interactions with antigens to which the body is chronically exposed, consistent with the discontinuity theory of immunity (Table 1). However, no induction of immune tolerance is observed in chronic inflammatory disorders. Immunological stimulation is thought to be persistent in these conditions, but this has yet to be demonstrated. There may be oscillations in the exposure of the immune system to immunological stimuli, preventing the induction of tolerance [START_REF] Pradeu | The speed of change: towards a discontinuity theory of immunity?[END_REF]. Further studies are required to determine whether chronic inflammatory conditions can develop when the immune system is faced with the intermittent presentation of a persistent stimulus.

The discontinuity theory of immunity can also serve as the basis of innovative therapeutic strategies. This theory is consistent with the occurrence of flare-ups in several autoimmune disorders, and suggests that these flare-ups may reflect intermittent interactions between immune receptors and autoantigens. It also suggests that desensitization strategies might be useful in the treatment of autoimmune diseases, to induce tolerance by subjecting the immune system to continuous exposure to the autoimmune stimulus. These predictions generate novel, testable research hypotheses.

A unifying framework for immunology

The discontinuity theory of immunity is based on two founding principles: the important role of time in the immune response, and the frequent detection, by the immune system, of modifications internal to the host. Indeed, the triggering of an immune response is a complex process based on the integration of many different signals, including antigen structure and the context in which recognition occurs [START_REF] Iwasaki | Control of adaptive immunity by the innate immune system[END_REF]. Several different modes of immune recognition can, thus, be distinguished [START_REF] Iwasaki | Control of adaptive immunity by the innate immune system[END_REF][START_REF] Chovatiya | Stress, Inflammation, and Defense of Homeostasis[END_REF]: recognition of patterns (either intracellularly or extracellularly, by pattern recognition receptors), recognition of the absence of a pattern ("missing self" as with NK cells, macrophages or complement), recognition of tissue damage, or recognition of functional modifications (as exemplified by the "guard hypothesis" in plants ( 23)).

We suggest that all these modes of recognition are instances of the general rule that the immune system senses sudden changes in the organism (Figure 1). It is now well-established that, in addition to their classical role in the sensing of "nonself" or "danger", many immune cells detect internal disturbances. This is the case for many "innate" immune cells, including NK cells and macrophages (3), as well as "adaptive" immune cells [START_REF] Grossman | Dynamic tuning of lymphocytes: physiological basis, mechanisms, and function[END_REF]. More generally, the detection of structural modifications to the host seems to have played a major role in the evolution of immune systems. Many organisms seem to make use of such an economical system for detecting internal change rather than a more costly system based on the detection of structural features of pathogens. Several examples of "effector-triggered immunity" have now been described in studies of the so-called "guard hypothesis" in plants [START_REF] Van Der Biezen | Plant disease-resistance proteins and the gene-forgene concept[END_REF], and in invertebrates [START_REF] Stuart | Effector-triggered versus pattern-triggered immunity: how animals sense pathogens[END_REF]. Primitive chordates also have NK-like systems (detecting missing or modified self) [START_REF] Khalturin | Urochordates and the origin of natural killer cells: Identification of a CD94/NKR-P1-related receptor in blood cells of Botryllus[END_REF], lending support to speculations that the immune recognition of major changes in the host has been a driving force in the evolution of the immune system, and that the perception of missing or modified self in vertebrates is a remnant of this evolutionarily ancient system. Many other recent data tend to the same conclusion, including, for example, the demonstrations that an endogenous caspase-11 ligand elicits the release of interleukin-1 from living dendritic cells [START_REF] Zanoni | An endogenous caspase-11 ligand elicits interleukin-1 release from living dendritic cells[END_REF], that an innate pathway activated by disturbance of the mucus layer exerts an antiviral effect preceding the action of interferon [START_REF] Iversen | An innate antiviral pathway acting before interferons at epithelial surfaces[END_REF], and that a guard-like mechanism of pyrin regulation exists in human (28).

Immune mechanisms of change-detection are well known, and some were first described decades ago. However, the processes underlying the recognition of patterns, an absence of pattern, tissue damage, and functional modifications have generally been considered to be separate phenomena, with different explanations. The discontinuity theory of immunity encompasses all these different modes of recognition in a single explanatory framework, according to which immune systems have been selected during evolution for their capacity to Herbert Feigl [START_REF] Feigl | Minnesota Studies in the Philosophy of Science[END_REF] once pointed out that "the aim of scientific explanation throughout the ages has been unification, i.e. the comprehending of a maximum of facts and regularities in terms of a minimum of theoretical concepts and assumptions." In a similar vein, the discontinuity theory of immunity seeks to provide a single, simple explanation for many different immunological phenomena.

One key lesson to be drawn from the framework suggested here is the need for more detailed studies of the time course of the immune response. Along this line, a pivotal element in the organization of the immune system resides in its capacity to adapt to slow or long-lasting modifications in the host, which it then treats as a new reference point. The duration of the stimulation needed for inducing immune responses must be thus compared to the time required for the immune system to adapt to the same stimulation. An illustration of the importance of these kinetics parameters resides in the killing of MHC class I-negative targets by NK cells raised in a negative environment to kill the same targets. Adoptive transfer experiments indicate that it takes several days to a whole population of NK cells to adapt to the chronic loss of MHC Class I in their environment, although it takes a few hours for a whole population of NK cells to kill MHC Class I-negative cells in vivo [START_REF] Joncker | Mature natural killer cells reset their responsiveness when exposed to an altered MHC environment[END_REF][START_REF] Elliott | MHC class I-deficient natural killer cells acquire a licensed phenotype after transfer into an MHC class I-sufficient environment[END_REF]. Quantitative investigations on the impact of time and antigenic dose would be facilitated by much needed collaboration between immunologists and biomathematicians, extending beyond traditional disciplinary boundaries [START_REF] May | Uses and Abuses of Mathematics in Biology[END_REF].

  sense sudden modifications in the body. The question whether mechanisms of immune recognition diverged from a common mechanism of change recognition or many mechanisms of immune recognition converged on a common strategy of change recognition remains open.

Figure legends Figure 1 .

 legends1 Figure legends

Figure 1

 1 Figure 1

Acknowledgments T.P.'s work is supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme -grant agreement n° 637647 (IDEM Starting Grant) and by the IDEX program of the University of Bordeaux. The E.V. laboratory is supported by the European Research Council (THINK Advanced Grant), the Ligue Nationale contre le Cancer (Equipe Labellisée) and by institutional grants from INSERM, CNRS and Aix-Marseille University to CIML. E.V. is a scholar of the Institut Universitaire de France. E.V. is a cofounder and a shareholder of Innate-Pharma. We thank Sébastien Jaeger for stimulating discussions.

Effector

Sudden appearance of tumor antigens

Sudden metabolic modification

Table 1. Weak or chronic stimulations lead to a tolerogenic immune response while sudden modifications lead to an effector immune response. On the graphs, the blue line represents antigen concentration and the red line represents the immune response.


Abdenbi Mohand Ousaid

Tianming Lu

Cécile Pacoret
email: pacoret@isir.upmc.frregnier

Stéphane Régnier

Vincent Hayward
email: hayward@upmc.fr

Dual Stage Options for Interface Designs Suitable for Haptic Interaction at the Micro-Nano Scales

Keywords: Haptic interface, Dual-stage architecture, Pantograph, Transparency, Tactile transducer, Signal crossover

Direct, manual interaction with the micro/nano scales is not straightforward because the objects at this scale obey unituitive physics. For instance, in ambient conditions at the micro-scale, capillary forces dominate over many other forces. When the scale becomes smaller, Brownian motion becomes pervasive. Haptic interfaces give us the option to bring the experience of this physics with the direct reach of the human sensorimotor capabilities. To cope with the limitations of conventional force feedback devices, we present here two alternative dual-stage designs suitable to address the needs of the interaction with the micro/nano scales. The first one features very low apparent inertia, a large dynamics range and a wide bandwidth. This properties are obtained by coupling a large actuator to a small one via a viscous coupler. Feedback can then be used to achieve nearly perfect transparency. The second is a conventional force feedback device augmented with a tactile transducer. The two channels are frequency compensated to achieved a flat response fro DC to one kHz.

Introduction

At the nano and micro scales, the mechanical behavior of objects no longer is dominated by gravity and friction. At short range, forces including electrostatic, capillary, and van der Waals forces are preeminent [START_REF] Gauthier | Robotic microassembly[END_REF]. As a result, the physics of the micro/nano scales differs completely from that of the macro scale and are not accessible to humans sensory and motor capabilities.

Like a microscope magnifies viewed objects, haptic devices can be employed to magnify interaction between objects at a scale at which humans are naturally Contributed equally to this work.

effective. Most haptic devices described in the past fail to match the human sensorimotor capacity and thus act as an obstacle between the hand and the phenomenon that could be accessed. Conventional interfaces are subjected to inherent friction and/or high inertia that affect the interface transparency, dynamic range, and bandwidth.

At the micro/nano scales, three issues arise in term of interface design [START_REF] Hayward | Performance measures for haptic interfaces[END_REF]. Effective scaling systems must provide high transparency to convey the most of what can be sensed by humans and not taint it with noise; they must have a large dynamic range since human can motorically and sensorially deal with 4-5 orders of magnitude; and they must have wide bandwidth since the haptic capabilities cover from dc to about 1 kHz. Transparency thus quantifies a haptic interface's capacity to reproduce micro/nano interactions to a human operator. The dynamic range is related to the force resolution of the interface. It quantifies the ratio of the largest to the smallest forces that can be commanded, and thus sensed, through the interface. Bandwidth refers to the capacity of the interface to convey mechanical signals over a large frequency range, in order to render complex phenomena, such as Brownian motion.

In the past 25 years, several haptic interfaces have been proposed to deal with interaction with small objects. They enhance human interaction capabilities at the microscale using either electrodynamic transducers [START_REF] Hollis | Toward a tele-nanorobotic manipulation system with atomic scale force feedback and motion resolution[END_REF][START_REF] Hunter | Manipulation and dynamic mechanical testing of microscopic objects using a telemicro-robot system[END_REF], or single stage systems based on conventional robotic architectures, see [START_REF] Gaponov | Telerobotic system for cell manipulation[END_REF], among several other examples. Such interfaces are subjected to inherent friction and high inertia that contribute spurious forces that mask the mechanical signals to be felt by the users. With single stage systems, sacrifices must be made regarding transparency, dynamic range, or the bandwidth in favor of other factors such as maximum force. To cope with these limitations, we describe here two alternative dual-stage haptic device designs (see Fig. 1) suitable to address the needs of interactions at the micro/nano scales.

Viscously coupled dual stage design

The first design, see Fig. 1(a), focuses on transparency and dynamic range. It is based on a dual-stage architecture [START_REF] Millet | The ultimate haptic device: First step[END_REF][START_REF] Mohand-Ousaid | Haptic interface transparency achieved through viscous coupling[END_REF]. The large, proximal motor provides power and the small, distal motor reproduces the force transients. The two stages are connected to each other by a passive viscous coupler based on Foucault (eddy) currents. Such coupler transforms slip velocity into torque with exacting accuracy and exhibits a linear behavior over a large velocity range. The viscous torque is proportional to the relative velocity between proximal and distal motor,

τ coupler = b( θ1 -θ2 ).
The handle is driven by the sum of two torques, that of the coupler and that of the distal motor. A fundamental advantage is the possibility to decouple the output torque from the inertia of the motor. Referring to Fig. 2, a pole-placement controller, C(z), designed by polynomial methods was shown to be able to achieve a near perfect transparency [START_REF] Mohand-Ousaid | Haptic interface transparency achieved through viscous coupling[END_REF].

Interface evaluation

The objective was to verify that the high-level degree of transparency of the dual-stage had a measurable effect on human performance when using the interface. Human observers to performed a signal detection task using the stimulus depicted Fig. 4(a). One component had a fixed magnitude and the other varied from imperceptible values to easily detectable values. To create an experimental condition that resembled actual use, observers detected the presence of weak force perturbations at random locations in the workspace. The stimuli was felt through two identical handles, one connected to the large motor, and other to the small motor, see Fig. 4(b). The observers were not aware of the condition under which they were working, that is with the dual-stage interface (condition A) or with the single motor (condition B), similar to a conventional interface. All subjects produced typical psychometric curves as shown in Fig 5 . Despite inter-subject variations, it could be concluded that the performance detection of the observers was improved by almost an order of magnitude. As a whole, the interface achieves a large dynamic range which goes from 5.2 mN to 5.7 N that is three orders of magnitude. Reader can find in [START_REF] Mohand-Ousaid | Haptic interface transparency achieved through viscous coupling[END_REF] more details about the evaluation procedure.

Haptic interaction with a water droplet

To test the ability of the haptic interface to bring the physics of micro scales to a human scale perception, we experimented with a actual interaction at micro scales. The test was carried out on a complex case for microscales force sensing, which entails measuring and feeling the interaction of a thin glass probe with a water droplet. The droplet was first approached towards the probe using the position of the interface handle as a set point for the droplet micro-positioner. Then, the droplet was contacted and retracted in approach-retraction cycle. The interaction force was measured by using a micro force sensor [START_REF] Ousaid | Active electrostatic force-sensing probe in the microNewton range[END_REF][START_REF] Ousaid | H-infinity optimal control enabled micro-force sensing[END_REF] and fed back to the user through the dual-stage haptic interface. The dual-stage interface conveys the amplified interaction forces between the glass probe and water droplet at a scale where the operator could interact through his/her natural touch perception. Figures 6(a) and 6(b) represent respectively the force measured by the sensor over a cycle of approach-retraction and the force transmitted to the operator by a probe with a diameter of 80 µm. The force felt by the user through the interface handle was exactly the force measured by the sensor amplified by a coefficient α f =0.05 10 6 . his system shows the capacity of the haptic interface to let humans to go beyond their sensory limitations. The system was also tested by several users with no experience in microscale phenomena. Various interaction forces, like pull-in and pull-off forces, are correctly rendered to the operators. Readers can refer to [START_REF] Ousaid | Stability and transparency analysis of a teleoperation chain for microscale interaction[END_REF] for more details about the set-up and the experiment.

Discussion

We described a new type of electromagnetic drive for use in haptic interfaces and other applications. The interface increases its dynamic range and its transparency by decoupling the delivered maximum torque from its effective inertia. As a whole, the interface achieves a large dynamic range which goes from 5.2 mN to 5.7 N. In terms of transparency for a human user, the interface achieves two orders of magnitude of improvement over existing designs. However, the capacity of the haptic interface to convey large frequency signals remains limited, particularly tactile sensations that can be employed to feel fast phenomana Brownian motion. To address this need, augmenting a conventional force feedback device with a vibrotactile transducer is an interesting option. Such as approach is described in the next section.

Haptic interface augmented with tactile transducer

A significant challenge is to increase the bandwidth of an interface to enable key aspects of the rich information available in the micro/nano scales to be felt [START_REF] Gauthier | Robotic microassembly[END_REF][START_REF] Romano | Creating realistic virtual textures from contact acceleration data[END_REF][START_REF] Sitti | Micro and nano-scale robotics[END_REF][START_REF] Finio | An ultra-high precision, high bandwidth torque sensor for microrobotics applications[END_REF]. The problem arises from factors such as the presence of resonances in the mechanical structure of articulated systems of a scale sufficient to interact with a moving hand. Because of these resonances, typical haptic interfaces must be limited in bandwidth using appropriate low-pass filters [START_REF] Campion | Fundamental limits in the rendering of virtual haptic textures[END_REF]. To fix ideas, the best performing Sensable Phantom 1.0A device has a useful mechanical bandwidth of only 30 Hz. As a result, high frequency phenomena such as Brownian motion, van der Waals and electrostatic forces cannot be reproduced by the interface. The mechanics of cantilevered structures of appropriate scales preclude any device to achieved a bandwidth able to transmit the signals arising from the micro/nano scales. In order to address this problem, we constructed a dual-stage design based on channel separation using signal crossover to separate haptic signals into low and high frequency channels that recombine at the tip of the device.

The main stage was a 'Pantograph' haptic interface, which is a planar parallel mechanism of two actuated degrees of freedom. It provided a stiff and light set of linkages driven without transmission [START_REF] Hayward | Design And Multi-objective optimization of a linkage for a haptic interface[END_REF][START_REF] Campion | The pantograph Mk-II: A Haptic instrument[END_REF]. It has the desirable features of light inertia, high stiffness, compact design but large operating workspace. The mechanical properties give rise to uniform magnitude performance on the workspace. A vibrotactile transducer, called a 'Haptuator' [START_REF] Yao | Design and analysis of a recoil-type vibrotactile transducer[END_REF], was embedded inside the finger interface to reproduce fast oscillations. The Haptuator, based on the principle of conservation of momentum, is designed to work in the high frequency regions and is limited in the low frequency range because of displacement saturation. The resulting system can be viewed in Fig. 1(b). Each component operated in its optimal range of frequencies. From the knowledge of the identified system's dynamics, a compensation/crossover scheme could achieve nearly ideal, uniform magnitude performance over the whole frequency band, from DC up to 1.0 kHz and up. The overall system block diagram is shown in Fig. 7.

LP compensator pantograph

HP compensator haptuator input output Fig. 7. System block diagram with compensation and crossover, L-P and H-P stand for low-pass and high-pass respectively.

Interface evaluation

The magnitude performance of each subsystem was first experimentally determined using identification by the third of octave band method. Then, the compensation scheme and the crossover filter were implemented in a real-time computational system. The magnitude response over the required frequency bands, from DC to 1.0 kHz, is reported in Fig. 8. It can be seen that the interface exhibited a flat response in the desired frequency region.

Optical tweezers application

The interface was employed in a simulation scenario for an optical tweezers application to demonstrate its capabilities [START_REF] Ni | 2d high speed force feedback teleoperation of optical tweezers[END_REF][START_REF] Ni | Asynchronous event based high speed vision for micro-particles tracking[END_REF], see Fig. 9(a). The scenario involved a nano-scale spherical object placed in a liquid environment which was subjected to nothing else but the Brownian motion. Its movements were recorded but a novel high-speed image capture system [START_REF] Ni | 2d high speed force feedback teleoperation of optical tweezers[END_REF]. During operation, operators were able to feel high frequency oscillations. In order to verify that the interface was capable of rendering the Brownian motion sensed by the optical tweezers simulation, a histogram was plotted to show the distribution of the signals recorded by an accelerometer mounted on the finger interface, see Fig. 9(b). It can be concluded that the measurements reflected the expected normal distribution and that the haptic interface was capable of rendering Brownian motion. Observers invited to test the interface reported that the haptic sensations where novel and provided new intuition about what Brownian motion is. The results suggest the measurements conformed to a Gaussian distribution and the interface was able to render Brownian motion.

Discussion

A haptic teleoperation interface was developed by combining a Pantograph and an Haptuator, which has uniform performance over a large frequency band from DC to 1.0 kHz. By applying the interface to data collected from optical tweezers, the observers were able to experience the Brownian motion present in the micro/nano world. The presented system demonstrates high bandwidth capability and reliability over determined frequency band. It gives rise to potential possibilities for complex applications which require uniform performance over large frequency regions. Further research aims at combining this concept with the viscously coupled dual stage design so that wide bandwidth, high transparency and dynamic range could be achieved all in one single system.

Conclusion

Dual-stage architectures were proposed to overcome the performance limitations of conventional haptic interfaces. A knob-type high fidelity haptic interface with one degree of freedom based on viscous coupling is advantageous when high torques are required since inertia of large motors are decoupled from operator experience. A dual stage design based on signal crossover and compensation filters could extend conventional interface bandwidth considerably. In future works, the complementary between the first and second design could be exploited to design high-performance and compact haptic interfaces suitable for dealing with micro/nano scales interactions.

Acknowledgments

This work was supported in part by the eurostar project REMIQUA.

Fig. 1 .

 1 Fig. 1. (a) Dual-stage haptic interface with viscous coupling, (b) Direct drive haptic interface augmented with high-frequency tactile transducer.

Fig. 2 .

 2 Fig. 2. The reference torque, τ d , is compared to the torque produced by the coupler, τ coupler . The large motor is enslaved by the compensator C(z) to zero the torque error. The small motor compensates for the slower response of the large motor.

Figure 3 (

 3 Figure 3(a) shows the response to a null reference torque. The torque felt by the operator, τ out , is nearly zero despite rapid movements of the handle. The inertia of the proximal motor is entirely masked by the feedback control. Mechanically speaking, the power required to move the large motor is entirely supplied by the power amplifier and not by the operator. Only the inertia of the small motor is felt (Fig3b).

Fig. 3 .

 3 Fig. 3. (a) Interface response to a null reference. (b) What is felt by the user.

Fig. 4 .

 4 Fig. 4. Dual-stage interface validation. (a) In condition A, the handle was connected to the output of the dual-stage drive. In condition B, an identical handle was connected directly to the shaft of the proximal motor and the coupler was disconnected. (b) Stimulus. A small burst of oscillation of size Θ, amplitude h0, and spatial period λ, was superposed onto a larger friction force, τC.

Fig. 5 .

 5 Fig. 5. Results and fits for conditions A and B a typical observer.

Fig. 6 .

 6 Fig.6. Illustration of the main phases during the interaction, approach, pull-in, penetration, retraction and pull-off. Measuring and feeling the interaction between a glass probe and water droplet. (a) Interaction force measured by the sensor over a cycle of approach-retraction according to time. (b) Haptic force feedback felt by the operator through the dual-stage haptic interface. .

Fig. 8 .

 8 Fig. 8. Acceleration magnitude response of the interface with tactile feedback. The result shows a flat response up to 1.0 kHz.

Fig. 9 .

 9 Fig. 9. (a) A virtual scenario created from records of an optical tweezers. The 200 nm spherical object was placed in a liquid environment and thus was subjected to Brownian motion. (b) Histogram of measured acceleration had a quasi-normal distribution. The results suggest the measurements conformed to a Gaussian distribution and the interface was able to render Brownian motion.


Joscha Diehl

Peter K Friz

Paul Gassiat

Stochastic control with rough paths

Keywords: Dyn. 08, 23, 2008]. 1991 Mathematics Subject Classification. Primary 60H99 Stochastic control, Duality, Rough Paths

  L'archive ouverte pluridisciplinaire

Introduction

In classical works [START_REF] Doss | Liens entre équations différentielles stochastiques et ordinaires[END_REF][START_REF] Sussmann | On the gap between deterministic and stochastic ordinary differential equations[END_REF] Doss and Sussmann studied the link between ordinary and stochastic differential equations (ODEs and SDEs, in the sequel). In the simplest setting, consider a nice vector field σ and a smooth path B : [0, T ] → R, one solves the (random) ordinary differential equation Ẋ = σ (X) Ḃ, so that X t = e σBt X 0 , where e σBt denotes flow, for unit time, along the vector field σ (•) B t . The point is that the resulting formula for X t makes sense for any continuous path B, and in fact the Itô-map B → X is continuous with respect to • ∞;[0,T ] . In particular, one can use this procedure for every (continuous) Brownian path; the so-constructed SDE solution then solves the Stratonovich equation,

dX = σ (X) • dB = σ (X) dB + 1 2 (σσ ) (X) dt.
When B = B (ω) : [0, T ] → R d is a multidimensional Brownian motion, which we shall assume from here on, this construction fails and indeed the Itô-map is notorious for its lack of (classical) continuity properties. Nonetheless, many approximations, B n → B, examples of which include the piecewise linear -, mollifier -and Karhunen-Love approximations, have the property that the corresponding (random) ODE solutions, say to dX n = b (X n ) dt+σ (X n ) dB n with σ = (σ 1 , . . . , σ d ), converge to the solution of the Stratonovich equation dX = b (X) dt + σ (X) • dB.

(At least in the case of piecewise linear approximations, this result is known as Wong-Zakai theorem 1 ). It was a major step forward, due to T. Lyons [START_REF] Lyons | Differential equations driven by rough signals[END_REF], to realize that the multidimensional SDE case can also be understood via deterministic differential equations (known as rough differential equations, short: RDEs); they do, however, require more complicated driving signals (known as rough paths) which in the present context are of the form

B (ω) : [0, T ] → R d ⊕ so (d) ,
and "contain", in addition to the Brownian sample path, Lévy stochastic area, viewed as process with values in so (d). It is known that B (ω) enjoys p-variation rough path regularity, for any p ∈ (2, 3). Given the increasing amount of text books on the subject, [START_REF] Lyons | System control and rough paths[END_REF][START_REF] Lyons | Differential equations driven by rough paths: École d'été de probabilités de Saint-Flour XXXIV-2004[END_REF][START_REF] Friz | Multidimensional stochastic processes as rough paths[END_REF][START_REF] Friz | A Course on Rough Paths: With an Introduction to Regularity Structures[END_REF], we shall keep further background to a minimum. Let us just mention that, among the many applications of rough paths to stochastic analysis, (i) all Wong-Zakai type results follow from B n → B in p-variation rough path metric (essentially thanks to continuity of rough integration with respect to rough path metrics) and (ii) the (rough)pathwise resolution of SDE can handle immediately situations with anticipating randomness in the coefficients; consistency with anticipating SDE in the sense of Nualart, Ocone, Pardoux [START_REF] Nualart | Stochastic calculus with anticipating integrands[END_REF][START_REF] Ocone | A generalized Itô-Ventzell formula. Application to a class of anticipating stochastic differential equations[END_REF] was established in [START_REF] Coutin | Good rough path sequences and applications to anticipating stochastic calculus[END_REF].

Given the deep insights of rough paths analysis to deterministic and stochastic differential equations, it is a natural question what such a (rough) pathwise point of view can contribute to our understanding of controlled deterministic, and then stochastic, differential equations.

An obstacle, both technical and conceptual, to overcome is that a pathwise approach to (eventually stochastic) control problem naturally leads to anticipating controls (and then potentially to serious measurability issues, as was seen e.g. in [START_REF] Buckdahn | Pathwise stochastic control problems and stochastic HJB equations[END_REF]) as well as the question of how to relate such anticipating controls to classical adapted controls. We feel that our results here constitute convincing evidence that rough path analysis is ideal to formulate and analyze these problems. In particular, one is able to write down, in a meaningful and direct way, all the quantities that one wants to write down -without any headache related to measurability and anticipativity technicalities : throughout, all quantities depend continuously on some abstract rough path η and are then -triviallymeasurable upon substitution η ← B (ω).

Loosely stated, our main results are as follows.

Theorem 1. (cf. Section 3) Let η : [0, T ] → R d ⊕ so (d) be a rough path (of p-variation regularity, p < 3) and µ = (µ t ) a control. (i) The controlled rough differential equation

(1.1) dX = b (X, µ) dt + σ (X) dη, X t = x
has a unique solution.

(ii) The value function

v (t, x) = sup µ T t f (s, X s , µ s ) ds + g (X T )
1 ... although, strictly speaking, the multi-dimensional case is due to M. Clark and Stroock-Varadhan.

satisfies a non-linear (rough) partial differential equation, in the sense of Caruana et al. [START_REF] Caruana | A (rough) pathwise approach to a class of nonlinear SPDEs[END_REF]. Moreover, a (rough) Pontryagin type maximum principle (based on a controlled, backward RDE) gives a necessary condition for a pair X, μ to be optimal.

It should be noted that the controlled rough differential equation in (i) does not immediately fall into the standard framework of rough path theory. But, as we lay out in the appendix, it can be easily treated as an infinite dimensional rough differential equation. The required rough PDE theory [START_REF] Caruana | A (rough) pathwise approach to a class of nonlinear SPDEs[END_REF] is far from the standard setting of rough paths and requires a subtle combination of rough path analysis with stochastic viscosity solution in the spirit Lions-Souganidis [START_REF] Lions | Fully nonlinear stochastic pde with semilinear stochastic dependence[END_REF][START_REF] Lions | Fully nonlinear stochastic partial differential equations: non-smooth equations and applications[END_REF].

When applied with η = B (ω), the above theorem provides information about the optimal control of the following Stratonovich equation2 dX = b (X, µ) dt + σ (X) • dB, considered up to time T . On the other hand, any optimal control for the deterministic control problem in (ii) above is of the form µ * t = µ * t (η) = µ * t (B (ω)), depending on σ (B t : t ≤ T ), and thus fails to satisfy the crucial adaptedness condition in stochastic control theory. Moreover, the ω-wise optimization has (at first glance) little to do with the classical stochastic control problem in which one maximizes the expected value, i.e. an average over all ω's, of a payoff function.

Nonetheless, stochastic and pathwise control are intimately connected and we are able to prove the following duality result.

Theorem 2. (cf. Section 4) (i) Write X µ,η for the solution to the controlled RDE (1.1). Let ν be an adapted control. Then

X ν = X µ,η | µ=ν(ω),η=B(ω) solves the (classical, non-anticipating) Stratonovich equation dX ν = b (X ν , ν) dt + σ (X ν ) • dB.
(ii) The stochastic optimal control problem

V (t, x) = sup ν adapted E T t f (s, X ν s , ν s ) ds + g (X ν T )
admits a dual representation, given by

V (t, x) = inf z E   sup µ T t f (s, X µ,η s , ν s ) ds + g (X µ,η T ) + z (µ, η) η=B(ω)  
where z ranges over suitable classes of penality functionals for non-adapted controls. Special choices of such classes lead to the Davis-Burstein duality [START_REF] Davis | A deterministic approach to stochastic optimal control with application to anticipative optimal control[END_REF] and to an extension of Rogers [START_REF] Rogers | Pathwise Stochastic Optimal Control[END_REF] to continuous time (as conjectured in that paper).

Readers familiar with [START_REF] Davis | A deterministic approach to stochastic optimal control with application to anticipative optimal control[END_REF] will recall a similar setup to ours, with control only in the drift term, b = b (X, ν) but not in the volatility term, σ = σ (X). On the other hand, time discretization of general controlled diffusion, i.e. with control in both drift and volatility, leads to the setting of controlled Markov chains discussed in [START_REF] Rogers | Pathwise Stochastic Optimal Control[END_REF]. One may then wonder (question posed to us by M. Davis) why the continuous time formulation does not allow for controlled volatility. From a technical point of view, the problem is that there is no satisfactory extension of (i) to a setting of rough differential equations with controlled volatility coefficients. (In a sense, there is missing information between the control, itself a rough signal, and the actual driving signal). If one attempts to bypass this by considering piecewise constant controls, with vanishing mesh size, one is led to the observation that (in non-degenerate situations) one simply obtains trivial upper duality bounds. (See the Remarks 13 and 21).

Let us briefly review some past works in the area of pathwise versus stochastic control. Wets [START_REF] Wets | On the relation between stochastic and deterministic optimization. Control theory, numerical methods and computer systems modelling[END_REF] first observed that stochastic optimization problems resemble deterministic optimization problems up to the nonanticipativity restriction on the choice of the control policy. Davis-Burstein, Davis-Karatzas, Rogers and then also Brown et al [START_REF] Davis | A deterministic approach to stochastic optimal control with application to anticipative optimal control[END_REF][START_REF] Davis | A deterministic approach to optimal stopping[END_REF][START_REF] Rogers | Monte Carlo valuation of American options[END_REF][START_REF] Rogers | Pathwise Stochastic Optimal Control[END_REF][START_REF] Brown | Information relaxations and duality in stochastic dynamic programs[END_REF] all implement duality type results in various settings.

At last, we mention Mazliak-Nourdin [START_REF] Mazliak | Optimal control for rough differential equations[END_REF] for a first investigation of controlled rough differential equations. While this obviously relates to this paper, they stay on "level 1" of rough path theory, i.e. the case handled by Young integration, and so do not cover any situations with Brownian or Brownian-like noise. In particular, our paper seems to be the first proper use of rough path analysis in the important field of (stochastic) optimal control.

Notation

For p < 3 denote by C 0,p-var = C 0,p-var (E) the space of geometric p-variation rough paths in E, where E is a Banach space chosen according to context. 3 On this space, we denote by ρ p-var the corresponding inhomogenous distance. For p < 2 this is just the p-variation norm ||.|| p-var .

Let U be some separable metric space (the control space). Denote by M the class of measurable controls µ : [0, T ] → U .

When working on a filtered probability space (Ω, F, F t , P), A will denote the class of progressively measurable controls ν : Ω × [0, T ] → U .

BU C(R e ) denotes the space of bounded, uniformly continuous functions that is usually employed in viscosity theory for partial differential equations.

Deterministic control with rough paths

Let η : [0, T ] → R d be a smooth path. Write X t,x,µ for the solution to the controlled ordinary differential equation

(3.1) dX t,x,µ s = b X t,x,µ s , µ s ds + σ X t,x,µ s dη s , s ≥ t, X t,x,µ t = x ∈ R e .
Classical control theory allows to maximize

T t f (s, X t,x,µ s , µ s ) ds+g (X µ T ) over a class of admissible controls µ. As is well-known, (3.2) v (t, x) := sup µ T t f s, X t,x,µ s , µ s ds + g X t,x,µ
T is the (under some technical conditions: unique, bounded uniformly continuous) viscosity solution to the HJB equation

-∂ t v -H (t, x, Dv) -σ (x) , Dv η = 0, v(T, x) = g(x)
.

where H acting on v is given by

(3.3) H (t, x, p) = sup u { b (x, u) , p + f (t, x, u)} .
Now, (3.1) also makes sense for a driving rough path (including a controlled drift terms to the standard setting of RDEs is fairly straight-forward; for the reader's convenience proofs are given in the appendix). This allows to consider the optimization problem (3.2) for controlled RDEs.

3.1. HJB equation. The main result here is that the corresponding value function satisfies a "rough PDE". Such equations go back to Lions-Souganidis ( [START_REF] Lions | Fully nonlinear stochastic partial differential equations: non-smooth equations and applications[END_REF] considers a pathwise stochastic control problem and give an associated stochastic HJB equation, see also [START_REF] Buckdahn | Pathwise stochastic control problems and stochastic HJB equations[END_REF]; these correspond to η = B(ω) in the present section). However their (non-rough) pathwise setup is restricted to commuting diffusion vector fields σ 1 , . . . , σ d (actually, [START_REF] Lions | Fully nonlinear stochastic partial differential equations: non-smooth equations and applications[END_REF] considers constant vector fields). Extensions to more general vector fields via a rough pathwise approach were then obtained in [START_REF] Caruana | A (rough) pathwise approach to a class of nonlinear SPDEs[END_REF] (see also [START_REF] Diehl | Regularity theory for rough partial differential equations and parabolic comparison revisited[END_REF]).

Definition 3. Let η ∈ C 0,p-var be a geometric rough path, p-var ≥ 1. Assume F, G, φ to be such that for every smooth path η there exists a unique BUC viscosity solution to

-∂ t v η -F (t, x, v η , Dv η , D 2 v η ) -G(t, x, v η , Dv η ) ηt = 0, v η (T, x) = φ(x).
We say that v ∈ BU C(R e ) solves the rough partial differential equation

-dv -F (t, x, v, Dv, D 2 v)dt -G(t, x, v, Dv)dη t = 0, v(T, x) = φ(x),
if for every sequence of smooth paths η n such that η n → η in rough path metric we have locally uniformly

v η n → v.
Remark 4.

(1) We remark that uniqueness of a solution, if it exists, is built into the definition (by demanding uniqueness for the approximating problems).

(2) In special cases (in particular the gradient noise case of the following theorem) it is possible to define the solution to a rough PDE through a coordinate transformation (if the vector fields in front of the rough path are smooth enough). This approach is followed in [START_REF] Lions | Fully nonlinear stochastic pde with semilinear stochastic dependence[END_REF].

The two definitions are equivalent, if the coefficients admit enough regularity (see [START_REF] Caruana | A (rough) pathwise approach to a class of nonlinear SPDEs[END_REF]). In the following theorem the coordinate transformation is not applicable, since σ is only assumed to be Lip γ instead of Lip γ+2 . Theorem 5. Let η ∈ C 0,p-var (R d ) be a rough path, p ∈ [2, 3). Let γ > p. Let b : R e × U → R e be continuous and let b(

•, u) ∈ Lip 1 (R e ) uniformly in u ∈ U . Let σ 1 , . . . , σ d ∈ Lip γ (R e ). Let g ∈ BU C(R e ). Let f : [0, T ] × R e × U →
R be bounded, continuous and locally uniformly continuous in t, x, uniformly in u.

For µ ∈ M consider the RDE with controlled drift 4 (Theorem 29),

(3.4) dX t,x,µ,η = b X t,x,µ,η , µ dt + σ X t,x,µ,η dη, X t,x,µ,η t = x.
4 An extension to a time-dependent b is straightforward, as an inspection of the proof of Theorem 29 shows. A time-dependent σ can be treated immediately by adding a time-component to the rough path, although this leads to strong regularity assumption in t. For a more nuanced approach one could adopt the ideas from [START_REF] Friz | Multidimensional stochastic processes as rough paths[END_REF]Chapter 12].

It is also possible to consider the controlled hybrid RDE/SDE dX = b(X, µ)dt + σ(X, µ)dW + σ(X)dη, see [START_REF] Diehl | Topics in Stochastic Differential Equations and Rough Path Theory[END_REF].

Then

v (t, x) := v η (t, x) := sup µ∈M T t f s, X t,x,µ,η s , µ s ds + g X t,x,µ,η
T is the unique bounded, uniformly continuous viscosity solution to the rough HJ equation

-dv -H (x, Dv) dt -σ (x) , Dv dη = 0, v(T, x) = g(x). (3.5)
Proof. The case f = 0, σ ∈ Lip γ+2 appears in [START_REF] Diehl | Topics in Stochastic Differential Equations and Rough Path Theory[END_REF]. The general case presented here is different, since we cannot use a coordinate transformation. Let a smooth sequence η n be given, such that

η n → η in C 0,p-var . Let v n (t, x) := sup µ∈M Ξ t,x [η n , µ],
where

Ξ t,x [γ, µ] := T t f (s, X t,x,µ,γ s , µ s ) ds+g X t,x,µ,γ T
for any (rough) path γ. By classical control theory (e.g. Corollary III.3.6 in [START_REF] Bardi | Dolcetta: Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations[END_REF]) we have that v n is the unique bounded, continuous viscosity solution to

-dv n -H (x, Dv n ) dt -σ (x) , Dv n dη n = 0, v n (T, x) = g(x). Then |v n (t, x) -v(t, x)| ≤ sup µ∈M |Ξ t,x [η, µ] -Ξ t,x [η n , µ]| .
Note that Ξ is continuous in γ uniformly in µ (and (t, x)) by Theorem 29. Therefore, v n converges locally uniformly to v and then, by Definition 3, v solves (3.5).

Example 6. In the case with additive noise (σ(x) ≡ Id) and state-independent gains / drift (f (s, x, u) = f (s, u), b(x, u) = b(s, u)), this rough deterministic control problem admits a simple solution. Indeed, if v 0 is the value function to the standard deterministic problem for η ≡ 0, i.e.

v 0 (t, x) = sup µ∈M T t f (s, µ s )ds + g x + T t b(s, µ s )ds ,
then one has immediately (since η only appears in the terminal gain)

v η (t, x) = v 0 (t, x + η T -η t ).
When v 0 has a nice form, this gives simple explicit solutions. For instance, assuming in addition f ≡ 0, U convex and b(s, u) = u, v 0 is reduced to a static optimization problem and

v η (t, x) = sup u∈U g x + η T -η t + (T -t)u .

Pontryagin maximum principle.

If η is smooth, then Theorem 3.2.1 in [START_REF] Yong | Stochastic controls: Hamiltonian systems and HJB equations[END_REF] gives the following optimality criterium.

Theorem 7. Let η be a smooth path. Assume b, f, g be C 1 in x, such that the derivative is Lipschitz in x, u and bounded and let σ, g be C 1 with bounded, Lipschitz first derivative. Let X, μ be an optimal pair for problem (3.2) with t = 0. Let p be the unique solution to the backward ODE

-ṗ(t) = Db( Xt , μt )p(t) + Dσ( Xt ) ηt p(t) + Df ( Xt , μt ), p(T ) = Dg( XT ). Then b( Xt , μt )p(t) + f ( Xt , μt ) = sup u∈U b( Xt , u)p(t) + f ( Xt , u) , a.e. t ∈ [0, T ].
Let now η be rough. We have the following equivalent statement.

Theorem 8. Let η ∈ C 0,p-var be a geometric rough path, p ∈ [2, 3). Assume the same regularity on b, f, g, σ as in Theorem 7. Let X, μ be an optimal pair. Let p be the unique solution to the controlled, backward RDE (Remark 31)

-dp(t) = Db( Xt , μt )p(t)dt + Dσ( Xt )p(t)dη t + Df ( Xt , μt )dt, p(T ) = Dg( XT ). Then b( Xt , μt )p(t) + f ( Xt , μt ) = sup u∈U b( Xt , u)p(t) + f ( Xt , u) , a.e. t.
Remark 9. This is the necessary condition for an admissible pair to be optimal. In the classical setting there do also exist sufficient conditions (see for example Theorem 3.2.5 in [START_REF] Yong | Stochastic controls: Hamiltonian systems and HJB equations[END_REF]). They rely on convexity of the Hamiltonian and therefore will in general (unless σ is affine in x) not work in our setting because, informally, the dη-term switches sign all the time.

Define for µ ∈ M J(µ) := T 0 f (r, X 0,x0,µ,η r , µ r )dr + g(X 0,x0,µ,η T ),
so that v(0, x 0 ) = inf µ∈M J(µ). We prepare the proof with the following Lemma.

Lemma 10. Let X, μ be an optimal pair. Let µ be any other control. Let I ⊂ [0, T ] be an interval with |I| = ε. Define

µ ε (t) := 1 I (t)µ(t) + 1 [0,T ]\I (t)μ(t).
Let X ε be the solution to the controlled RDE (3.4) corresponding to the control µ ε . Let Y ε be the solution to the RDE

Y ε t = t 0 Db( Xr , μr )Y ε r dr + t 0 Dσ( Xr )Y ε r dη r + t 0 b( Xr , µ r ) -b( Xr , μr ) 1 I (r)dr. Then sup t |X ε t -Xt | = O(ε), sup t |Y ε t | = O(ε), (3.6)
sup t |X ε t -Xt -Y ε t | = O(ε 2 ), (3.7)
J(µ ε ) -J(μ) = Dg( XT ), Y ε T + T 0 Df ( Xr , μr ), Y ε r + f ( Xr , µ r ) -f ( Xr , μr ) 1 I (r) dr + O(ε 2 ). (3.8)
Proof. We have

T 0 ||b(•, μr ) -b(•, µ ε r )|| Lip 1 dr ≤ cε.
Hence looking at the proof of Theorem 33 (note that X ε and X have the same initial condition), we get

||X ε -X|| ∞ ≤ ||X ε -X|| p-var ≤ cε,
which proves the first equality in (3.6). The second one follows analogously.

Now D ε t := X ε t -Xt -Y ε t satisfies dD ε t = b(X ε t , µ ε t ) -b( Xt , μt ) -Db( Xt , μt )Y ε t -b( Xt , µ t ) -b( Xt , μt ) 1 I (t) dt + σ(X ε t ) -σ( Xt ) -Dσ( Xt )Y ε t dη t = t 0 Db( Xt + θ(X ε t -Xt ), µ ε t ) -Db( Xt , µ ε )dθ(X ε t -Xt ) + Db( Xt , µ ε t ) -Db( Xt , μt ) (X ε t -Xt ) + Db( Xt , μt )D ε t dt + 1 0 Dσ( Xt + θ(X ε t -Xt )) -Dσ( Xt )dθ(X ε t -X t )dη t + Dσ( Xt )D ε t dη t = dA ε t + Db( Xt , μt )D ε t dt + dM t D ε t
where

dA ε t = t 0 Db( Xt + θ(X ε t -Xt ), µ ε t ) -Db( Xt , µ ε )dθ(X ε t -Xt ) + Db( Xt , µ ε t ) -Db( Xt , μt ) (X ε t -Xt ) dt + 1 0 Dσ( Xt + θ(X ε t -Xt )) -Dσ( Xt )dθ(X ε t -X t )dη t dM t = Dσ( Xt )dη t .
It is straightforward to check that A ε = O(ε 2 ) as a path controlled by η (see Remark 34). The result then follows from Lemma 35.

Proof of Theorem 8. We follow the idea of the proof of Theorem 3.2.1 in [START_REF] Yong | Stochastic controls: Hamiltonian systems and HJB equations[END_REF]. Fix

x 0 ∈ R e .
Since η is geometric, we have

Dg( XT ), Y ε T = p T , Y ε T -p 0 , Y ε 0 = - T 0 Df ( Xr , μr ), Y ε r dr + T 0 p(r), b( Xr , µ r ) -b( Xr , μr ) 1 I (r) dr.
Here, Y ε and I are given as in Lemma 10.

Let any u ∈ U be given. Let µ(t) ≡ u. Let t ∈ [0, T ) and let ε > 0 be small enough such that

I ε := [t, t + ε] ⊂ [0, T ].
Then, combined with Lemma 10 we get

0 ≥ J(µ ε ) -J(μ) = Dg( XT ), Y ε T + T 0 Df (x r , μr ), Y ε r + f ( Xr , µ r ) -f ( Xr , μr ) 1 I (r) dr + o(ε) = - T 0 Df ( Xr , μr ), Y ε r dr + T 0 p(r), b( Xr , µ r ) -b( Xr , μr ) 1 I (r) dr + T 0 Df (x r , μr ), Y ε r + f ( Xr , µ r ) -f ( Xr , μr ) 1 I (r) dr + o(ε) = t+ε t p(r), b( Xr , u) -b( Xr , μr ) + f ( Xr , u) -f ( Xr , μr )dr + o(ε).
Dividing by ε and sending ε → 0 yields, together with the separability of the metric space, the desired result. Proposition 11. Under the assumptions of Theorem 5, the map

ω → sup µ∈M T t f (s, X µ,η , µ s ) ds + g (X µ,η T ) η=B(ω)
is measurable. In particular, the expected value of the pathwise optimization problem,

(3.9) v (t, x) = E   sup µ∈M T t f (s, X µ,η , µ s ) ds + g (X µ,η T ) η=B(ω)   is well-defined.
Proof. The lift into rough pathspace, ω → B (ω), is measurable. v η as element in BUC space depends continuously (and hence: measurably) on the rough path η. Conclude by composition.

Remark 12. Well-definedness of such expressions was a non-trivial technical obstacle in previous works on pathwise stochastic control; e.g. [START_REF] Davis | A deterministic approach to stochastic optimal control with application to anticipative optimal control[END_REF][START_REF] Buckdahn | Pathwise stochastic control problems and stochastic HJB equations[END_REF]. The use of rough path theory allows to bypass this difficulty entirely.

Remark 13. Let us explain why we only consider the case where the coefficient σ(x) in front of the rough path is not controlled. It would not be too difficult to make sense of RDEs

dX = b(t, X, u)dt + σ(X, u)dη t ,
assuming good regularity for σ and (u s ) s≥0 chosen in a suitable class (for instance : u piecewise constant, u controlled by η in the Gubinelli sense,...). However, in most cases of interest the control problem would degenerate, in the sense that we would have

v η (t, x) = sup µ∈M T t f s, X t,x,µ,η s , µ s ds + g X t,x,µ,η T = T t (sup µ,x f (s, x, µ))ds + sup x g(x).
The reason is that if σ has enough u-dependence (for instance if d = 1, U is the unit ball in R e and σ(x, u) = u) and η has unbounded variation on any interval (as is the case for typical Brownian paths), the system can essentially be driven to reach any point instantly.

In order to obtain nontrivial values for the problem, one would need the admissible control processes to be uniformly bounded in some particular sense (see e.g. [START_REF] Mazliak | Optimal control for rough differential equations[END_REF] in the Young case, where the (µ s ) need to be bounded in some Hölder space), which is not very natural (for instance, Dynamic Programming and HJB-type pointwise optimizations are then no longer valid).

Duality results for classical stochastic control

We now link the expected value of the pathwise optimization problem, as given in (3.9), to the value function of the (classical) stochastic control problem as exposed in [START_REF] Krylov | Controlled diffusion processes[END_REF][START_REF] Fleming | controlled markov processes and viscosity solutions[END_REF],

(4.1) V (t, x) := sup ν∈A E T t f s, X t,x,ν s , ν s ds + g X t,x,ν T .
Her A denotes the class of progressively measurable controls ν : Ω × [t, T ] → U , where we use notation in not specifying t. There are well-known assumptions under which V is a classical (see [START_REF] Krylov | Controlled diffusion processes[END_REF]) resp. viscosity (see [START_REF] Fleming | controlled markov processes and viscosity solutions[END_REF]) solution to the HJB equation, i.e. the non-linear terminal value problem

-∂ t V -F t, x, DV, D 2 V = 0 V (T, •) = g;
uniqueness holds in suitable classes. In fact, assume the dynamics

dX s,x,ν t = b (X s,x,ν t , ν t ) dt + d i=1 σ i (X s,x,ν t ) • dB i t , = b (X s,x,ν t , ν t ) dt + d i=1 σ i (X s,x,ν t ) dB i t , X s,x,ν s = x, (4.2)
where

b(x, u) = b(x, u) + 1 2 d i=1 (σ i • Dσ i )(x)
is the corrected drift. Then the equation is semilinear of the form

-∂ t V -H (t, x, DV ) -LV = 0, V (T, •) = g. (4.3) where LV = 1 2 Tr[(σσ T )D 2 V ]
and H is given by H (t, x, p) = sup u b (x, u) , p + f (t, x, u) .

Let us also write

L u V = b (•, u) DV + LV, u ∈ U.
As before M is the class of measurable functions µ : [t, T ] → U , with the topology of convergence in measure (with respect to dt), where we abuse notation by not specifying t.

Inspired by results in discrete time [START_REF] Brown | Information relaxations and duality in stochastic dynamic programs[END_REF], we have the following duality result.

Theorem 14. Let Z F be the class of all mappings z : C 0,p-var × M → R such that • z is bounded, measurable and continuous in η ∈ C 0,p-var uniformly over

µ ∈ M • E[z(B, ν)] ≥ 0, if ν is adapted Let b : R e × U → R e be continuous and let b(•, u) ∈ Lip 1 (R e ) uniformly in u ∈ U and such that u → b(•, u) is Lipschitz. Let σ 1 , . . . , σ d ∈ Lip γ (R e ), for some γ > 2, g ∈ BU C(R e ) and f : [0, T ] × R e × U →
R bounded, continuous and locally uniformly continuous in t, x, uniformly in u. Then we have

V (t, x) = inf z∈Z F E   sup µ∈M T t f (r, X t,x,µ,η r , µ r )dr + g(X t,x,µ,η T ) + z(η, µ) η=B(ω)   .
Where B denotes the Stratonovich lift of the Brownian motion to a geometric rough path and X t,x,µ,η is the solution to the RDE with controlled drift (Theorem 29)

dX t,x,µ,η = b X t,x,µ,η , µ dt + σ X t,x,µ,η dη, X t,x,µ,η t = x.
Remark 15. Every choice of admissible control ν ∈ A in (4.1) leads to a lower bound on the value function (with equality for ν = ν * , the optimal control). In the same spirit, every choice z leads to an upper bound. There is great interest in such duality results, as they help to judge how much room is left for policy improvement. The result is still too general for this purpose and therefore it is an important question, discussed below, to understand whether duality still holds when restricting to some concrete (parametrized) subsets of Z F .

Proof. We first note, that the supremum inside the expectation is continuous (and hence measurable) in η, which follows by the same argument as in the proof of Theorem 5. Since it is also bounded, the expectation is well-defined.

Recall that X t,x,ν is the solution to the (classical) controlled SDE and that X t,x,µ,η is the solution to the controlled RDE. Let z ∈ Z F . Then, using Theorem 29 to justify the step from second to third line,

V (t, x) = sup ν∈A E T t f (s, X t,x,ν s , ν s )ds + g(X t,x,u T ) ≤ sup ν∈A E T t f (s, X t,x,ν s , ν s )ds + g(X t,x,ν T ) + z(B, ν) = sup ν∈A E   T t f (s, X t,x,µ,η s , µ s )ds + g(X t,x,µ,η T ) + z(η, µ) µ=ν,η=B   ≤ E   sup µ∈M T t f (s, X t,x,µ,η s , µ s )ds + g(X t,x,µ,η T ) + z(η, µ) η=B   ,
and to show equality, let

z * (η, µ) := V (t, x) - T t f (s, X t,x,µ,η s , µ s )ds + g(X t,x,µ,η T ).
Then z * ∈ Z F and equality is attained. 4.1. Example I, inspired by the discrete-time results of Rogers [START_REF] Rogers | Pathwise Stochastic Optimal Control[END_REF]. We now show that Theorem 14 still holds with penalty terms based on martingale increments. Theorem 16. Under the (regularity) assumptions of Theorem 14 we have

V (t, x) = inf h∈C 1,2 b E   sup µ∈M T t f s, X t,x,µ,η s , µ s ds + g X t,x,µ,η T -M t,x,µ,η,h t,T η=B(ω)   ,
where

M t,x,µ,η,h t,T := h T, X t,x,µ,η T -h t, X t,x,µ,η t - T t (∂ s + L µ s ) h s, X t,x,µ,η s ds.
That is, Theorem 14 still holds with Z F replaced by the set {z :

z(η, µ) = M t,x,µ,η,h t,T , h ∈ C 1,2 b }. Moreover, if V ∈ C 1,2 b the infimum is achieved at h * = V .
Proof. We have

V (t, x) ≤ inf h∈C 1,2 b E   sup µ∈M T t f (s, X t,x,µ,η s , µ s )ds + g(X t,x,µ,η T ) -M t,x,µ,η,h t,T η=B   = inf h∈C 1,2 b h(t, x) + E   sup µ∈M T t f (s, X t,x,µ,η s , µ s ) + (∂ s + L µ s )h(s, X t,x,µ,η s )ds + g(X t,x,µ,η T ) -h(T, X t,x,µ,η T ) η=B   ≤ inf h∈C 1,2 b h(t, x) + T t sup x∈R e ,u∈U [f (s, x, u) + (∂ s + L u )h(s, x)] ds + sup x∈R e [g(x) -h(T, x)] ≤ inf h∈S + s h(t, x) + T t sup x∈R e ,u∈U [f (s, x, u) + (∂ s + L u )h(s, x)] ds + sup x∈R e [g(x) -h(T, x)] ≤ inf h∈S + s h(t, x).
where the first inequality follows from (the proof of) Theorem 14 and S + s denotes the class of smooth bounded super solutions of the HJB equation. Note that S + s ⊂ C 1,2 b , which yields the third to last inequality.

But in fact the infimum of smooth supersolutions is equal to the viscosity solution V , all inequalities are actually equalities and the result follows. This can be proved via a technique due to Krylov [START_REF] Krylov | On the rate of convergence of finite-difference approximations for Bellman's equations with variable coefficients[END_REF] which he called "shaking the coefficients". For the reader's convenience let us recall the argument.

Recall that b is the corrected drift, see (4.2). Extending by continuity b, σ and f to t ∈ (-∞, ∞), define for ε > 0,

F ε (t, x, p, X) := sup u∈U,|s|,|e|≤ε b(x + e, u), p + 1 2
Tr(σσ T (x + e)X) + f (t + s, x + e, u) , and consider V ε the unique viscosity solution to

-∂V ε ∂t -F ε (t, x, DV ε , D 2 V ε ) = 0, V ε (T, •) = g.
By (local) uniform continuity of b, σ, f one can actually show that V → V ε as ε → 0, locally uniformly. This can be done for instance by interpreting V ε as the value function of a stochastic control problem. Now take some smoothing kernel ρ ε with R e+1 ρ ε = 1 and supp(ρ ε ) ⊂ [-ε, ε] e+1 , and define

V ε := V ε * ρ ε . Clearly by definition of F ε , for each |s|, |e| ≤ ε, V ε (• -s, • -e) is a supersolution to the HJB equation -∂ t V -F ε (t, x, DV, D 2 V ) = 0. Since F is convex in (DV, D 2 V ) it follows that V ε = [-ε,ε] e+1 V ε (• -s, • -e)ρ ε (s, e)dsde
is again a (smooth) supersolution (for the details see the appendix in [START_REF] Barles | On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman equations[END_REF]). Finally it only remains to notice that |V -

V ε | ≤ |V -V * ρ ε | + |(V -V ε ) * ρ ε | → 0 (locally uniformly). Remark 17. Note that V h (t, x) := E   sup µ∈M T t f s, X t,x,µ,η , µ s ds + g X t,x,µ,η T -M t,x,µ,η,h t,T η=B(ω)   ,
for fixed x, t, is precisely of the form (3.9) with f resp. g replaced by f resp. g, given by

f (s, •, µ) = f (s, •, µ) + (∂ s + L µ ) h (s, •) , g (•) = g (•) + h (T, •) -h (t, x) .
The point is that the inner pathwise optimization falls directly into the framework of Section 3.

Remark 18. For η a geometric rough path, we may apply the chain rule to h(s, X s ) and obtain

h(T, X T ) -h(t, x) = T t Dh(s, X s ), b(s, X s , µ s )ds + σ(X s )dη s .
It follows that the penalization may also be rewritten in a (rough) integral form

M t,x,µ,η,h t,T = T t Dh(s, X s ), σ(X s )dη s + T t (b -b)(s, X s , µ s ), Dh(s, X s ) - 1 2 Tr[(σσ T )D 2 h](s, X s ) ds.
Note that for η = B and adapted ν, this is just the Itô integral T t Dh(s, X s ), σ(X s )dB s . Remark 19. If one were to try anticipating stochastic calculus, in the spirit of [START_REF] Davis | A deterministic approach to stochastic optimal control with application to anticipative optimal control[END_REF], to implement Roger's duality in continuous time, then -leaving aside all other technical (measurability) issues that have to be dealt with -more regularity on the coefficient will be required. This is in stark contrast to the usual understanding in SDE theory that rough paths require more regularity than Itô theory.

Example 20. From Example 6 we can see that in some special cases this method gives explicit upper bounds. Assume :

• additive noise (σ ≡ Id),

• state-independent drift b ≡ b(u),

• running gain f (s, x, u) = f 0 (u) -∇h(x) • b(u), with h superharmonic (∆h ≤ 0). Then for the penalty corresponding to h(t, x) = h(x), the inner optimization problem is given by

sup µ∈M T t (f 0 (µ s ) -∇h(X t,x,µ,η s ), b(µ s ) )ds + (g -h) X t,x,µ,η T + h(x) + T t ( ∇h(X t,x,µ,η s ), b(µ s ) + 1 2 ∆h(X t,x,µ,η s ))ds ≤ sup µ∈M T t f 0 (µ s ) ds + (g -h) X t,x,µ,η T + h(x) = h(x) + V 0,h (t, x + η T -η t ),
where V 0,h is the value function to the standard control problem

V 0 (t, x) = sup µ∈M T t f 0 (µ s )ds + (g -h) x + T t µ s ds .
From Theorem 16, we then have the upper bound

V (t, x) ≤ h(x) + E V 0,h (t, x + B T -B t ) .
Remark 21. As in Remark 13, one can wonder how Theorem 16 could translate in the case where σ depends on u. As mentioned in that remark, under reasonable conditions on σ the control problem degenerates so that for any choice of h, say for piecewise-constant controls µ, we can expect that

E   sup µ T t f (s, X t,x,µ,η s , µ s ) + (∂ t + L µ s )h(s, X t,x,µ,η s )ds + g(X t,x,µ,η T ) -h(T, X t,x,µ,η T ) η=B   = T t sup x∈R e ,u∈U [f (s, x, u) + (∂ t + L u )h(s, x)] ds + sup x∈R e [g(x) -h(T, x)] .
In other words there is nothing to be gained from considering the (penalized) pathwise optimization problem, as we always get

V (t, x) ≤ h(t, x) + T t sup x∈R e ,u∈U [f (s, x, u) + (∂ t + L u )h(s, x)] ds + sup x∈R e [g(x) -h(T, x)]
which is in fact clear from a direct application of Itô's formula (or viscosity comparison). [START_REF] Davis | A deterministic approach to stochastic optimal control with application to anticipative optimal control[END_REF]. We now explore a different penalization, possible under concavity assumptions.

Example II, inspired by Davis-Burstein

Theorem 22. Let g be as in Theorem 14 and assume f = 0; furthermore make the (stronger) assumption that b ∈ C 5 b , σ ∈ C 5 b , σσ T > 0, and that (4.1) has a feedback solution u * 5 which is continuous, C 1 in t and C 4 b in x, taking values in the interior of U . Assume that U is a compact convex subset of R n . 5 That is, the optimal control is given as a deterministic function u * of time and the current state of the system. This is also called a Markovian control.

Let Z t,x,η be the solution starting from x at time t to (denote b

u := ∂ u b) dZ = b (Z, u * (t, Z)) dt + σ (Z) dη -b u (Z, u * (t, Z))u * (t, Z)dt, (4.4) let W (t, x) := W (t, x; η) := g(Z t,x,η T
) and assume that

∀(t, x), u → b(x, u), DW (t, x; B)
is strictly concave, a.s. (4.5)

Then V (t, x) = inf λ∈A E[ sup µ∈M g(X t,x,µ,η T ) + T t λ(r, X t,x,µ,η r , η), µ r dr η=B(ω)
].

Where A is the class of all λ : [0, T ] × R e × C 0,p-var → R d such that

• λ is bounded and uniformly continuous on bounded sets • λ is future adapted, i.e. for any fixed t, x, λ(t, x, B) ∈ σ(B s : s ∈ [t, T ])

• E[λ(t, x, B)] = 0 for all t, x. That is, Theorem 14 still holds with Z F replaced by the set

{z : z(η, µ) = T t λ(s, X t,x,µ,η r , η), µ s ds, λ ∈ A}.
Moreover the infimum is achieved with λ * (t, x, η) := t b u (t, u * (t, x))DW (t, x; η).

Remark 23.

The concavity assumption is difficult to verify for concrete examples. It holds for the linear quadratic case, which we treat in Section 4.3.

Remark 24. The case of running cost f is, as usual, easily covered with this formulation. Indeed, let the optimal control problem be given as

dX = b(X, ν)dt + σ(X) • dW, V (t, x) = sup ν E[ T t f (X, ν)dr + g(X T )].
Define the new component

dX d+1 t = f (X, u)dt, X d+1 t = x.
Then the theorem yields that the penalty

λ * (t, x) := (b u , f u ) • (D x 1...d g(Z T ) + D x 1...d Z e+1 T , D xe+1 Z e+1 T ) = (b u , f u ) • (D x 1...d g(Z T ) + D x 1...d Z e+1 T , 1). is optimal, where dZ = b(Z, u * )dt + σ(Z)dη -b u (Z, u * )u * dt, dZ e+1 = f (Z, u * )dt -f u (Z, u * )u * dt.
Proof. From (the proof of) Theorem 14 we know

V (t, x) ≤ inf λ∈A E[. . . ]. The converse direction is proven in [8] by using 6 λ * (t, x, η) = t b u (t, u * (t, x))DW (t, x).
For the reader's convenience we provide a sketch of the argument below.

Sketch of the Davis-Burstein argument. We have assumed that the optimal control for the stochastic problem (4.1) is given in feedback form by u * (t, x). Write X t,x, * := X t,x,u * .

Recall that Z t,x,η is the solution starting from x at time t to :

dZ = b (Z, u * (t, Z)) dt + σ (Z) dη -b u (Z, u * (t, Z))u * (t, Z)dt. Assume that W (t, x) := W (t, x; η) = g(Z t,x,η T
) is a (viscosity) solution to the rough PDE

-∂ t W -b(x, u * (t, x)) -b u (x, u * (t, x))u * (t, x), DW -σ (x) , DW η = 0, (4.6)
and assume that W is differentiable in x.

We assumed that

∀(t, x), u → b(x, u), DW (t, x) is strictly concave. It then follows that b(x, u * (t, x)) -b u (x, u * (t, x))u * (t, x), DW = sup u∈U b(x, u) -b u (x, u * (t, x))u, DW . (4.7)
Because of (4.7) the PDE (4.6) may be rewritten as

-∂ t W -b(x, u * (t, x)), DW -u * (t, x), λ * (t, x; η) -σ (x) , DW η = -∂ t W -sup u∈U { b(x, u), DW -u, λ * (t, x; η) } -σ (x) , DW η = 0.
By verification it follows that actually W is also the value function of the problem with penalty λ * , and the optimal control is given by u * , i.e.

W (t, x) = W (t, x; η) = sup µ∈M g(X t,x,µ,η T ) - T t λ * (s, X t,x,µ,η s ; η), µ s ds = g(X t,x,u * ,η T ) - T t λ * (s, X t,x,u * ,η s ; η), u * (s, X t,x,u * ,η s ) ds
Then, by Theorem 29 we have (if the convexity assumption (4.5) is satisfied a.s. by η = B(ω))

W (t, x; B) = g(X t,x, * T ) - T t λ * (s, X t,x, * s ; B), u * (s, X t,x, * s ) ds.
It follows in particular that for the original stochastic control problem

V (t, x) = sup ν∈A E g(X t,x,ν T ) = E g(X t,x, * T ) = E g(X t,x, * T ) - T t λ * (s, X t,x, * s ; B), u * (s, X t,x, * s ) ds = E sup µ∈M g(X t,x,µ,η T ) - T t λ * (s, X t,x,µ,η s ; η), µ s ds | η=B .
Here we have used that λ * (t, x, B) is future adapted and E[λ * (t, x, B)] = 0 ∀t, x, which is shown on p. 227 in [START_REF] Davis | A deterministic approach to stochastic optimal control with application to anticipative optimal control[END_REF].

Remark 25. The two different penalizations presented above are based on verification arguments for respectively the stochastic HJB equation and the (rough) deterministic HJB equation. One can then also try to devise an approach based on Pontryagin's maximum principles (both stochastic and deterministic). While this is technically possible, the need to use sufficient conditions in the rough Pontryagin maximum principle means that it can only apply to the very specific case where σ is affine in x, and in consequence we have chosen not to pursue this here.

4.3.

Explicit computations in LQC problems. We will compare the two optimal penalizations in the case of a linear quadratic control problem (both for additive and multiplicative noise).

4.3.1.

LQC with additive noise. The dynamics are given by7

dX = (M X + N ν)dt + dB t (4.8)
and the optimization problem is given by

V (t, x) = sup ν∈A E 1 2 T t ( QX s , X s + Rν s , ν s )ds + 1 2 GX T , X T . (4.9)
This problem admits the explicit solution (see e.g. Section 6.3 in [START_REF] Yong | Stochastic controls: Hamiltonian systems and HJB equations[END_REF])

V (t, x) = 1 2 P (t)x, x + 1 2 T t
Tr(P (s))ds, (4. [START_REF] Diehl | Regularity theory for rough partial differential equations and parabolic comparison revisited[END_REF] where P is the solution to the matrix Riccati equation

Ṗ = -P (t)M -t M P (t) + P N R -1t N P (t) -Q P (T ) = G,
and the optimal control is then given in feedback form by

ν * (t, x) = -R -1t N P (t)x.
Proposition 26. For this LQ control problem the optimal penalty corresponding to Theorem 22 is given by

z 1 (η, µ) = - T t λ 2 (s; η), µ s ds,
where

λ 1 (t; η) = -t N T t e t M (s-t) P (s)dη s .
The optimal penalty corresponding to Theorem 16 is given by

z 2 (η, µ) = z 1 (η, µ) + γ R (η), where γ R (η) = T t P (s)X 0 s , dη s - 1 2 T t
Tr(P (s))ds, X 0 denoting the solution to the RDE dX = M Xdt + dη starting at (t, x). In particular, these two penalizations are equal modulo a random constant (not depending on the control) with zero expectation.

Proof. The formula for z Hence we see that this penalization can be written as 2 = z 1 + γ R (η), where γ R (η) does not depend on the chosen control. One can check immediately that E[γ R (η)| η=B(ω) ] = 0. 4.3.2. LQC with multiplicative noise. Let the dynamics be given by

dX = (M X + N ν)dt + n i=1 C i X • dB i t (4.11) = ( M X + N ν)dt + n i=1 C i XdB i t . (4.12)
Denote by X t,x,µ,η the solution starting from x at time t to dX t,x,µ,η s = (M X t,x,µ,η s

+ N µ)dt + n i=1 C i X t,x,µ,η s dη i t
and by Γ t,s the (matrix) solution to the RDE

d s Γ t,s = M Γ t,s ds + n i=1 C i Γ t,s dη s , Γ t,t = I Then X t,x,µ,η s = Γ t,s x + s t Γ r,s N µ r dr. (4.13)
For simplicity we now take d = n = 1: the general case is only notationally more involved.

The optimization problem is given by

V (t, x) = sup ν∈A E 1 2 T t (QX 2 s + Rν 2 s )ds + 1 2 GX 2 T . (4.14)
By Section 6.6 in [START_REF] Yong | Stochastic controls: Hamiltonian systems and HJB equations[END_REF] the value function is again given as

V (t, x) = 1 2 P t x 2
and the optimal control as

u * (t, x) = -R -1 N P t x,
where

Ṗt + 2P t M + 2P t C 2 + Q -N 2 R -1 P 2 t = 0, P T = G. (4.15)
We can then compute explicitely the Davis-Burstein and Rogers penalties :

Proposition 27. For t ≤ r ≤ T , define Θ r = T r P s Γ 2
r,s (dη s -Cds).

Then the optimal penalty corresponding to Theorem 22 is given by

z 1 (η, µ) = 2CN x T t Γ t,s Θ s µ s ds + CN 2 T t T t
Γ r∧s,r∨s Θ r∨s µ r µ s drds, while the optimal penalty corresponding to Theorem 16 is given by

z 2 (η, µ) = CΘ t x 2 + z 1 (η, µ).
Proof. The optimal penalty stemming from Theorem 22 (see also Remark 24) is given by T t λ * (r, x)µ r dr, where

λ * (r, x) = N GZ 1 T ∂ x Z 1 T + ∂ x Z 2 T -N P (r)x,
where

dZ 1 = M Z 1 ds + CZ 1 dη t , Z 1 r = x; dZ 2 = 1 2 Q -N 2 R -1 P (s) 2 (Z 1 ) 2 ds, Z 2 r = 0. Since Z 1 s = Γ r,s x, this is computed to λ * (r, x) = N x GΓ 2 r,T + T r Q -N 2 R -1 P (s) 2 Γ 2 r,s ds -P (r) . = N x P (s)Γ 2 r,s T s=r + T r -Ṗ (s) -2M P (s) -2C 2 P (s) Γ 2 r,s ds = 2N x T r P (s)Γ 2 r,s (Cdη s -C 2 ds) = 2N CxΘ r .
Using the formula (4.13), we immediately obtain

z 1 (η, µ) = T t λ * (s, X t,x,µ,η s )µ s ds = 2CN x T t Γ t,s Θ s µ s ds + CN 2 T t T t
Γ r∧s,r∨s Θ r∨s µ r µ s drds.

For the optimal penalty corresponding to Theorem 16, we apply again Remark 18 to see that the optimal penalty is given by

M t,x,µ,η,V t,T = T t DV (s, X s ), CXdη s - T t Tr[C 2 X 2 D 2 V (s, X s )]ds = T t P r C|X t,x,µ,η r | 2 dη r - T t C 2 |X t,x,µ,η r | 2 P r dr.
It only remains to perform straightforward computations expanding the quadratic terms and applying Fubini's theorem.

Remark 28. Let us also draw attention to the linear-quadratic Gaussian stochastic control studied in [START_REF] Duncan | Linear-quadratic fractional Gaussian control[END_REF], noting however that the type of penalization proposed in Theorem 16 and Theorem 22 will not work for this, since they rely on a Markovian setting.

Appendix: RDEs with controlled drift

Theorem 29 (RDE with controlled drift). Let p ∈ [2, 3). Let η ∈ C 0,p-var a geometric p-variation rough path. Let γ > p. Let U be the subset of a separable Banach space. Let b : R e

× U → R e such that b(•, u) ∈ Lip 1 (R e ) uniformly in u ∈ U (i.e. sup u∈U ||b(•, u)|| Lip 1 (R e ) < ∞) and such that u → b(•, u) is measurable. Let σ 1 , . . . , σ d ∈ Lip γ (R e ). Let µ : [0, T ] → U be measurable, i.e. µ ∈ M.
(i) There exists a unique Y ∈ C 0,p-var that solves

Y t = y 0 + t 0 b(Y r , µ r )dr + t 0 σ(Y )dη r .
Moreover the mapping

(x 0 , η) → Y ∈ C 0,p-var
(t, x, u) i = k ∂ x k (φ -1 ) i (t, φ(t, x))b k (φ(t, x), u).
Remark 30. In the last case, i.e. point (iv), we can immediately use results in [START_REF] Friz | Multidimensional stochastic processes as rough paths[END_REF] (Theorem 10.53) to also handle linear vector fields.

Remark 31. Given a rough path η on [0, T ], the time-inverted object ←η t := η T -t is again a rough path. We can hence solve controlled, backward RDEs using the previous Theorem by inverting time.

Proof. Denote for µ ∈ M

Z µ t (•) := t 0 b(•, µ r )dr,
which is a well defined Bochner integral in the space Lip 1 (R e ) (indeed, by assumption on b,

t 0 ||b(•, µ r )|| Lip 1 (R e ) dr < ∞). Then Z µ ∈ C 1-var ([0, T ], Lip 1 (R e )). Indeed ||Z µ || 1-var ≤ T 0 ||b(•, µ r )|| Lip 1 (R e ) dr ≤ T 0 sup u∈U ||b(•, u)|| Lip 1 (R e ) dr, (5.3) independent of µ ∈ M.
By Theorem 33 we get a unique solution to the RDE

dY = f (Y )dZ µ + σ(Y )dη,
where f : R e → L(Lip 1 (R e ), R e ) is the evaluation operator, i.e. f (y)V := V (y). This gives existence of the controlled RDE as well as continuity in the starting point and in η. By (5.3), this is independent of µ ∈ M and we hence have shown (i). Concerning (ii), assume now that U u → b(•, u) ∈ Lip 1 is Lipschitz. Using the representation given in the proof of (i) it is sufficient to realize that if

µ n → µ ∈ M in measure, then Z µ n → Z µ in C 1-var ([0, T ], Lip 1 (R e )).
Concerning (iii): First of all, we can regard ν as a measurable mapping from (Ω, F) into the space of all measurable mappings from [0, T ] → U with the topology of convergence in measure. Indeed, if U is a compact subset of a separable Banach space, then this follows from the equivalence of weak and strong measurability for Banach space valued mappings (Pettis Theorem, see Section V.4 in [START_REF] Yosida | Functional Analysis[END_REF]). If U is a general subset of a separable Banach space, then define ν n : Ω → M with ν n (ω) t := Φ n (ν(ω) t ). Here Φ n is a (measurable) nearest-neighbor projection on {x 1 , . . . , x n }, the sequence (x k ) k≥0 being dense in the Banach space. Then ν n is taking values in a compact set and hence by the previous case, is measurable as a mapping to M. Finally ν is the pointwise limit of the ν n and hence also measurable.

Hence Y | µ=ν,η=B is measurable, as the concatenation of measurable maps (here we use the joint continuity of RDE solutions in the control and the rough path, i.e. continuity of the mapping (5.1)). Now, to get the equality (5.2): we can argue as in [START_REF] Friz | A Course on Rough Paths: With an Introduction to Regularity Structures[END_REF] using the Riemann sum representation of stochastic integral.

(iv) This follows from Theorem 1 in [START_REF] Diehl | Topics in Stochastic Differential Equations and Rough Path Theory[END_REF] or Theorem 2 in [START_REF] Crisan | Robust Filtering: Correlated Noise and Multidimensional Observation[END_REF].

Remark 32. One can also prove "by hand" existence of a solution, using a fixpoint argument, like the one used in [START_REF] Gubinelli | Controlling Rough Paths[END_REF]. This way one arrives at the same regularity demands on the coefficients. Using the infinite dimensional setting makes it possible to immediately quote existing results on existence, which shortens the proof immensely. We thank Terry Lyons for drawing our attention to this fact.

In the proof of the previous theorem we needed the following version of Theorem 6.2.1 in [START_REF] Lyons | System control and rough paths[END_REF].

Theorem 33. Let V, W, Z be some Banach spaces. Let tensor products be endowed with the projective tensor norm. 8 Let p ∈ [2, 3) η ∈ C 0,p-var (W ) and Z ∈ C q-var ([0, T ], V ) for some 1/q > 1 -1/p. Let f : Z → L(V, Z) be Lip 1 , let g : Z → L(W, Z) be Lip γ , γ > p. Then there exists a unique solution Y ∈ C 0,p-var (Z) to the RDE dY = f (Y )dZ + g(Y )dη, in the sense of Lyons. 9 Moreover for every R > 0 there exists C = C(R) such that ρ p-var (Y, Ȳ ) ≤ C||Z -Z|| q-var .

whenever (Z, X) and ( Z, X) are two driving paths with ||Z|| q-var , || Z|| q-var , ||X|| p-var ≤ R.

Proof. Since Z and X have complementary Young regularity (i.e. 1/p+1/q > 1) there is a canonical joint rough path λ over (Z, X), where the missing integrals of Z and the cross-integrals of Z and X are defined via Young integration. So we have λ s,t = 1 + Z s,t X s,t + t s Z s,r ⊗ dZ r t s Z s,r ⊗ dη r t s η s,r ⊗ dZ r t s η s,r ⊗ dη r Then, by Theorem 6.2.1 in [START_REF] Lyons | System control and rough paths[END_REF], there exists a unique solution to the RDE dY = h(Y )dλ, where h = (f, g).

We calculate how λ depends on Z. For the first level we have of course ||λ (1) λ(1) || p-var ≤ ||Z -Z|| q-var . For the second level we have, by Young's inequality, Together this gives ρ p-var (λ, λ) ≤ c||Z -Z|| q-var .

8 See [START_REF] Lyons | Differential equations driven by rough paths: École d'été de probabilités de Saint-Flour XXXIV-2004[END_REF] p. 18 for more on the choice of tensor norms which, of course, only matter in an infinite dimensional setting.

9 See e.g. Definition 5.1 in [START_REF] Lyons | Differential equations driven by rough paths: École d'été de probabilités de Saint-Flour XXXIV-2004[END_REF].

Plugging this into the continuity estimate of Theorem 6.2.1 in [START_REF] Lyons | System control and rough paths[END_REF] we get ρ p-var (Y, Ȳ ) ≤ C||Z -Z|| q-var as desired.

Remark 34. The following Lemma as well as the proof the Pontryagin principle use the concept of controlled rough paths in the sense of Gubinelli [START_REF] Gubinelli | Controlling Rough Paths[END_REF]. Even though it is usually set up in Hölder spaces, the modification to variation spaces poses no problem (see [START_REF] Perkowski | Pathwise stochastic integrals for model free finance[END_REF]Section 4.1]). Let X ε solve dX ε t = dA ε t + A t dη t X ε 0 = 0.

(5.4)

Then ρ p-var (X ε , 0) = O(ε).

Proof. For every ε there exists a unique solution solution to (5.4), and it is uniformly bounded in ε ∈ (0, 1]. This follows from [START_REF] Friz | Multidimensional stochastic processes as rough paths[END_REF]Theorem 10.53], by considering (A ε , η) as a joint rough path. We can hence, uniformly in ε, replace the linear vector field by a bounded one. We then get from [START_REF] Friz | A Course on Rough Paths: With an Introduction to Regularity Structures[END_REF]Theorem 8.5] ρ p-var (X ε , X 0 ) ≤ c ρ p-var ((A ε , η), (0, η)).

where X 0 denotes the unique solution to (5.4) with A ε replaced by the constant 0-path. Obviously X 0 ≡ 0 and it easy to see that ρ p-var ((A ε , η), (0, η)) = O(ε), which yields the desired result.

3. 3 .

 3 Pathwise stochastic control. We can apply Theorem 5 to enhanced Brownian motion, i.e. take η = B (ω), Brownian motion enhanced with Lévy's stochastic area which constitutes for a.e. ω a geometric rough path. The (rough)pathwise unique solution to the RDE with controlled drift, X µ,η | η=B(ω) then becomes a solution to the classical stochastic differential equation (in Stratonovich sense) (Theorem 29).

  is locally Lipschitz continuous, uniformly in µ ∈ M.(ii) Assume that u → b(•, u) is Lipschitz. If we use the topology of convergence in measure on M, thenM × R e × C 0,p-var → C 0,p-var (µ, x 0 , η) → Y, (5.1) is continuous. (iii) Assume that u → b(•, u) is Lipschitz. If ν : Ω × [0, T ] → U isprogressively measurable and B is the Stratonovich rough path lift of a Brownian motion B, then Y | µ=ν,η=B = Ỹ , P -a.s., (5.2) where Ỹ is the (classical) solution to the controlled SDE Ỹt = y 0 + t 0 b( Ỹr , ν r )dr + t 0 σ( Ỹ ) • dB r . (iv) If σ 1 , . . . , σ d ∈ Lip γ+2 (R e ), we can write Y = φ(t, Ŷt ) where φ is the solution flow to the RDE φ(t, x) = x + t 0 σ(φ(r, x))dη r , and Ŷ solves the classical ODE Ŷt = x 0 + t 0 b(r, Ŷr , µ r )dr, where we define componentwise b

  r dZ r -t s Zs,r d Zr | ≤ | t s Z s,r d Z r -Zr | + | t s Z s,r -Zs,r d Zr | ≤ c||Z|| q-var;[s,t] ||Z -Z|| q-var;[s,t] + c||Z -Z|| q-var;[s,t] ||Z|| q-var;[s,t] . r d Zr | ≤ c||X|| p-var;[s,t] ||Z|| q-var;[s,t] .

Lemma 35 .

 35 In particular we define for a path Y controlled by η with derivative Y ||Y, Y || η,p-var := ||Y || p-var + ||R|| p/2-var , where R s,t := Y s,t -Y s η s,t . Let η be a geometric p-variation rough path, p ∈ (2, 3). Let M be controlled by η and for ε ∈ (0, 1] let (A ε , (A ) ε ) be controlled by η with ||A ε , (A ) ε || η,p-var = O(ε).

  1 is in fact already computed in [8, sec. 2.4], so that it only remains to do the computation for the Rogers penalization.

	It follows from Remark 18 that
	M t,x,µ,η,V t,T	=	t	T	DV (s, X s ), dη s -	1 2		t	T	Tr(D 2 V (s, X s ))ds
		=	t	T	P (s)X µ s , dη s -	1 2	t	T	Tr(P (s))ds
		=	t	T	P (s)(X 0 s +	0	s	e M (s-r) N µ r dr), dη s -	1 2	t	T	Tr(P (s))ds
		=	t	T	µ r , ( t N	r	T	e	t M (s-r) P (s)dη s ) dr +	t	T	P (s)X 0 s , dη s -	1 2	t	T	Tr(P (s))ds.

See e.g.[START_REF] Friz | Multidimensional stochastic processes as rough paths[END_REF][START_REF] Friz | A Course on Rough Paths: With an Introduction to Regularity Structures[END_REF] for consistency of rough integration again B (ω) with Stratonovich integration.

For p < 2 these are just continuous E-valued paths with finite p-variation. For p ∈ [2, 3) additional "area" information is necessary. We refer to[START_REF] Lyons | System control and rough paths[END_REF] and[START_REF] Friz | Multidimensional stochastic processes as rough paths[END_REF] for background on rough path theory.

The paper of Davis-Burstein predates rough path theory relies heavily on anticipating stochastic calculus.

This equation admits an obvious pathwise SDE solution (via the ODE satisfied by X -B) so that, strictly speaking, there is no need for rough paths here.

This work was commenced while all authors were affiliated to TU Berlin. The work of JD was supported by DFG project SPP1324 and DAAD/Marie Curie programme P.R.I.M.E. PKF and PG have received partial funding from the European Research Council under the European Union's Seventh Framework Programme (FP7/2007-2013) / ERC grant agreement nr. 258237 "RPT". PKF acknowledges support from DFG project FR 2943/2.

1


Abbas Bahri

Yanyan Li

Olivier Rey

On a variational problem with lack of compactness: the topological effect of the critical points at infinity


est destinée au dépôt

The topological impact of critical points at infinity in a variational problem with lack of compactness: the dimension 3

Olivier Rey

To cite this version:

Olivier Rey. The topological impact of critical points at infinity in a variational problem with lack of compactness: the dimension 3. Advances in Differential Equations, 1999, 4, pp.581-616. hal-00943490