Etienne Balmes
email: balmes@sdtools.com

Shapes & DOF: on the use of modal concepts in the context of parametric non-linear studies

Physical responses tend to lie within restricted subspaces even for parametric problems. For a given subspace, the choice of a basis defines Degree Of Freedom (DOF) and this choice may give interesting meaning to the associated amplitudes. Classical modal analysis builds subspaces combining modeshapes and static responses. Parametric loads for non-linear, damped, variable, ... structures are discussed to extend the theory and illustrated for test and simulation cases. Challenges in shape extraction and basis generation techniques are then detailed. Introducing the ability to manipulate models with variable junction properties, component material and geometry, load and operating conditions, ... opens new questions on the quantification and tracking of changes and objectives throughout design exploration. The definition of a reference linear system and the use of global and/or local modal DOF are shown to provide an interesting perspective.

Introduction

Modal analysis has been used extensively for many decades. The base notion is that the response is dominated by modal contributions corresponding to a linear combination of shapes constant in space and associated amplitudes depending on time, which are known as Degrees of Freedom (DOF) in mechanics and states in system modeling. Linear time invariant systems are no longer considered difficult, so the focus of this paper will be on parametric problems. Non-linearities, damping treatments, junctions, geometric variability, ... are such parameters that challenge current practice. In such studies the choice of relevant shapes and associated DOF plays a useful role in both speeding computations and providing ways to analyze tests.

As illustrated by figure 1, design groups use direct problem resolution: starting from geometry in the form of CAD models, well defined mechanical model equations are generated in the form of finite element models for most industrial applications. From these models, reduction techniques are used to extract shapes and generate system models that allow the prediction of frequency or time responses. Experimental groups go in the opposite direction, that is solve inverse problems. As shown in figure 2, time responses are measured, signal processing tools are then used to obtain experimental characterizations such as transfer functions, then multiple sensor reading are combined to identify system models containing shape information. Finally test and FEM models can be combined in modeshape expansion and updating processes. The meeting point between computational and experimental activities is a system with inputs and outputs as detailed in section 2. A key difficulty of system modeling is the definition of states/DOF and associated shapes. Bandwidth and load objectives are introduced to allow correct system representations. Within a given subspace the ability to choose DOF is discussed. Parametric problems and associated loads are then introduced. Finally current challenges in subspace techniques are detailed: alternative learning and classification strategies, interface reduction, sparsity preservation.

Considering cases where parametric models are available, section 3 addresses tools needed to analyze their evolution. A parallel with control theory shows the relevance of "coupling" studies that correspond to the adaptation of root locus computations. The definition of a reference linear system is used to track property changes in parametric computations and test. Finally, the issue of understanding component properties when integrated in a system is discussed and illustrated.

A system perspective of structural dynamics 2.1 Definition of system models

Considering vibration problems through the point of view of system dynamics is a useful perspective. In a system model, one considers

• inputs {u(t)}, which in vibration problems will typically be applied loads or enforced displacements.

In experiments inputs are measured as electrical signals and thus limited to a series of scalar quantities, which may a notable approximation if the true loads are non-uniform fields.

• states {x(t)}, which are all the quantities needed to write a differential equation describing the evolution of the system in time. In a fairly general setting the evolution equation can be written as a non-linear state-space equation

{ ẋ} = f (x(t), u(t), p, t) (1)
For mechanical engineers, displacement Degrees of Freedom (DOF) {q} are used and the state vectors combines displacement {q} and velocity { q}. Additional internal states are also introduced for nonlinear material laws (plasticity, viscoelasticity, ...). Figure 3 illustrates a simple modified Oberst test which can be considered as a single DOF system characterizing the feature of interest which is the frequency and damping of the first bending mode, but will require many FEM DOF to allow the prediction of the strain energy field shown as color. Modal analysis or by extension model reduction, discussed in section 2.2, provides a clean strategy to relate these two definitions of DOF for the same system.

• outputs {y(t)}, which are measurable quantities in test (displacements, strains, loads, ...) and predictable quantities in models. Since states describe all the quantities describing the evolution of the system, outputs can be estimated at a given time through an observation equation

{y} = g(x(t), u(t), p, t) (2)
In Out • environment variables or design points corresponding to parameters that are either constant or assumed to be independent of states. Section 2.3 will detail parameters of interest in vibration design problems.

Environment Design point

• features are quantities that can be extracted from the response such as resonance frequencies, shapes, damping, mean levels, ...

The objective of a system model is to provide accurate prediction of the input/output relation and the associated features for a range of design points. Side objectives are to minimize the cost of obtaining and operating such a model and to have a proper understanding of its limitations.

Classical modal analysis/model reduction

Classical modal analysis, which is taken here to include Component Mode Synthesis (CMS), assumes Linear Time Invariant (LTI) systems so that the evolution equation can be written using constant matrices. Although the theory applies to all LTI problems, the paper will focus on mechanical problems of the form

[M ] {q} + [C] { q} + [K] {q} = [b] {u(t)} (3)
In vibration problems, modeling typically considers two objectives: the accurate representation of the response to a restricted set of loads within a bandwidth. In an undamped LTI system, normal modes are harmonic solutions in the absence of excitation and thus verify

-M ω 2 j + K {φ j } = 0. (4)
To achieve the bandwidth objective the system model must be able to represent all modeshapes within this band. Strategies to eliminate modes within the band can be based on modal participation factors or balanced realization arguments [START_REF] Girard | Structural Dynamics in Industry[END_REF][START_REF] Kammer | Selection of Component Modes for Craig-Bampton Substructure Representations[END_REF], but in practice such truncation is nowadays rarely necessary for performance. The argument to use modes above the bandwidth, the common 1.5 times the bandwidth or any other factor, is actually quite weak and adding more modeshapes is generally sub-optimal in terms of model size.

To achieve the loads objective, it is also necessary to account for the additional associated information. This is achieved by computing static responses to loads

[q Residual ] = [K F lex ] -1 [b] (5)
or by using the common alternative of static responses to enforced displacements on the areas of load application, which is known as Guyan condensation. The need for static terms is widely acknowledged: the contribution is called upper residuals in experimental modal analysis [START_REF] Heylen | Modal Analysis Theory and Testing[END_REF], attachment mode or residual flexibility for free mode component mode synthesis [START_REF] Craig | A Review of Time-Domain and Frequency Domain Component Mode Synthesis Methods[END_REF], D term in the observation equation of state-space models, residual vector in many FEM solvers, ... and yet many recognize to have difficulties with the notion.

Figure 4 illustrates a case with strong static contribution [START_REF] Balmes | Validation of a vibration and electric model of honeycomb panels equipped with piezoelectric patch actuators[END_REF]. A piezo-actuator is glued on a honeycomb panel. At 10 Hz, the bending of the honeycomb skin is much larger than the overall panel bending. The response can be decomposed in modal (a), (b) and static (c) contributions. The first key here is to understand that static contribution comes from the piezo load, and thus is completely independent from the structural modes which do not depend of any load.

Once the necessity of static correction/residual flexibility understood, the challenge remains in the definition of reference loads [b] for which a static correction is needed, discussed in section 2.3 and the understanding of how wrong results are when it is ignored.

Figure 4: Sample case with strong static contribution [START_REF] Balmes | Validation of a vibration and electric model of honeycomb panels equipped with piezoelectric patch actuators[END_REF] While modes and static responses are results of interests, one of the key aspects of modal analysis is that they span the subspace where the response lies. For any basis T of that subspace, one defines generalized DOF {q R } and seeks an approximation of the form

[q F ull (t)] = [T ] {q R (t)} (6)
leading in a Ritz-Galerkin approach to a reduced model still of the form (3) but using reduced matrices M R = T T M T , ... The two basics bases [START_REF] Craig | A Review of Time-Domain and Frequency Domain Component Mode Synthesis Methods[END_REF][START_REF] De Klerk | General Framework for Dynamic Substructuring : History, Review and Classification of Techniques[END_REF] are McNeal (free modes + static response to loads φ 1:N M K -1 b ) and Craig-Bampton (enforced static displacement /Guyan+ fixed interface modes), but any combination of mode and static response will give achieve the bandwidth and load objectives and thus lead to correct results. Conceptual limitations of most of the literature are

• the use of explicit formulas for bases rather than the implementation of numerical basis generation strategies to go from arbitrary sets of assumed vectors to bases [START_REF] Boiteau | Modal Solvers and resolution of the generalized problem (GEP)[END_REF][START_REF] Sternchüss | Multi-level parametric reduced models of rotating bladed disk assemblies[END_REF]. Thus many have rejected static corrections because it tends to generate poorly conditioned basis if iterative orthogonalization of loads b is not enforced. But the combination of free modes and Guyan vectors is also valid although rarely implemented in software packages.

• the use of modes computed on a single component rather than the trace of modes computed on a larger system (see section 2.4.3).

• the failure to recognize that the accuracy of a reduced model depends on the subspace spanned by T not on the base vectors and associated DOF used to generate that subspace. For example, a modal model in physical coordinates can be defined by taking the first few modes [φ 1:N R ], retaining a subset of physical DOF R, and spanning the subspace with the basis defined as follows

[q F ull (t)] N = [φ 1:N R ] N ×N R [ φ 1:N R | R ] -1 N R×N R {q R (t)} N R (7)
Starting from physical DOF, basis generation techniques based in incomplete LU factorization will lead to generalized coordinates that still correspond to physical DOF, while eigenvalue/SVD based techniques will lead to modal coordinates.

Parametric problems of interest beyond LTI modal analysis

While LTI modal analysis remains very useful the challenge has now moved to the handling of parametric problems. The common approach is to continue using classical CMS methods and to reformulate the problem as a linear reduced model with all other aspects of interest (coupling, non-linearity, and environment) implemented as external load or feedback. This approach is well mastered for point to point connections. Thus many multi-body simulation software support flexible component reduction using Craig-Bampton reduction with residual terms for viscous loads, distributed inertia (inertial relief modes [START_REF] Craig | A Review of Time-Domain and Frequency Domain Component Mode Synthesis Methods[END_REF]). But much more should be done. Selecting an underlying linear system, associated with constant matrices, M 0 , C 0 , K 0 , one can always rewrite the equations of motion (3) by the defining a parametric load {f p } in the right hand side of the equation

[M 0 ] {q} + [C 0 ] { q} + [K 0 ] {q} = [b] {u(t)} + {f p (M (p) -M 0 , K(p) -K 0 , q, q, p)} (8)
Figure 12 illustrates a parametric contact problem. The contact between a main press formed plate and a small "cable guide" component is modeled at gauss points shown in (a). Due to manufacturing the contact does not occur on the whole surface and different colored dots correspond to different surfaces. In the frequency tracking, cross markers correspond to variable surface computations. They overlay nearly exactly with solid lines which correspond to variable distributed stiffness. The selected modes illustrate that fairly flat lines correspond to global bending of the plate while the frequency traversing the full frequency range from 700 to 1900 Hz corresponds to a local bending shape of the cable guide.

Figure 6: Influence of contact on the modal properties of a brake part [START_REF] Martin | Review of model updating processes used for brake components[END_REF]. Left: Zoom on cable guide, right: frequency evolution with contact surface/stiffness

The idea of parametric reduced models is to construct a basis independent of the amplitude of parameter loads just as traditional CMS is independent of the amplitude of external inputs. Here the parameter loads are forces applied on all contact points. As there are many points, computing a static correction ( 5) for all contact loads will lead to large reduced models. Alternatives [START_REF] Hammami | Numerical design and test on an assembled structure of a bolted joint with viscoelastic damping[END_REF] are to build a multi-model basis from modes computed using different contact conditions or the compute the static correction to contact loads found for the nominal modes (first order correction). These techniques are similar to interface reduction discussed in 2.4.1.

The representation of static correction corresponds to the use of additional out of band DOF. These have a strong impact on zeroes (by opposition to in band modes which generate poles) of transfers associated with parameter loads and thus on the prediction of closed loop configurations where these have a significant impact. A key result is that the added shapes are very different from the following modes so that seeking convergence by just adding more modes is rarely a good idea. A challenging question is the analysis of the minimum number of out of band DOF necessary to represent all configurations shown in figure 12 right.

A very partial list of problems that can be addressed by including parametric loads into classical CMS are

• Non-linear responses are parametric in the sense that local properties depend on inputs. Figure 5 thus illustrates the amplitude dependence of a junction problem [START_REF] Chevallier | Design of an Assembly for Nonlinear Vibration Reduction, IMAC[END_REF] where the parametric loads are contact/friction forces distributed on junction surfaces also shown in the figure.

• Structures damped with viscoelastic materials have parametric volume loads associated with viscoelastic stresses which depend on temperature and frequency (figure 11 gives an illustration). Thickness variations, often used as design parameters, can also be represented as a parametric load.

• Structural dynamic modification techniques [START_REF] Corus | Amélioration de méthodes de modification structurale par utilisation de techniques d'expansion et de réduction de modèle[END_REF] seek to combine a linear test model with parametric models of possible modifications.

• Geometry variations due to design or variability in the manufacturing process are also of major interest [START_REF] Franc ¸a De Paula | On the influence of geometry updating on modal correlation of brake components[END_REF][START_REF] Martin | Méthodes de corrélation calcul/essai pour l'analyse du crissement[END_REF].

Open challenges in shape extraction and basis generation techniques

Space/time separation (6) considering bandwidth and input objectives were shown to be the base ingredients of CMS. This leaves many choices open and this section seeks to illustrate areas still under development. Section 2.4.1 addresses alternative learning and vector classification techniques. Section 2.4.2 discusses issues with interface reduction, which occur in mechanical problems where applied loads correspond to fields. Finally, section 2.4.3 touches basis sparsity and associated memory/CPU issues.

Alternative learning and vector classification strategies

In traditional CMS methods, shapes are obtained by a combination of simple computations of modes and static responses. Even modes are actually computed through a series of static computations in Krylov iteration based strategies such as the Lanczos algorithm.

Another category of approaches uses learning techniques where a full order response is analyzed a posteriori. Such techniques can be used both in computations (for example to compute the high frequency impedance of a piezo [START_REF] Balmes | Effects of temperature on the impedance of piezoelectric actuators used for SHM[END_REF]) and experiments [START_REF] Martin | Squeal measurement with 3D Scanning Laser Doppler Vibrometer: Handling of the time varying system behavior and analysis improvement using FEM expansion[END_REF]. Figure 7 illustrates a time varying resonance found in a squeal test. From the spectrogram of each sensor, a response at the frequency of the maximum amplitude can be extracted. The set of learning vectors is thus a matrix with spatial rows (different sensors) and temporal columns associated with the maximum response at different instants.

Given a set of learning vectors or the full possible space, a natural strategy is to classify directions by their relative contribution. Here the Singular Value Decomposition is used to extract shapes, shown in red, and associated time amplitudes (DOF) shown in blue. The usefulness of these amplitudes will be further discussed in section 3.2. Classification of vectors in a large subspace, followed by truncation, is a key ingredient of most strategies. Modal analysis considers low frequency modes which for equal kinetic energy the shapes of interest are those that minimize strain energy [START_REF] Balmes | Optimal Ritz vectors for component mode synthesis using the singular value decomposition[END_REF]. Using strain and kinetic energy norms is not the only possibility and Ref [START_REF] Kerschen | The method of proper orthogonal decomposition for dynamical characterization and order reduction of mechanical systems: An overview[END_REF] clearly points out that the literature is full of eigenvalue based subspace classification algorithms: SVD Singular Value Decomposition, KLD Karhunen-Loève Decomposition, PCA Principal Component Analysis, ...

In parametric problems, the body of literature on Proper Generalized Decomposition arguments that rather than limiting the decomposition to space and time, variable separation can made on space, time and parameter variations [START_REF] Chinesta | Encyclopedia of Computational Mechanics Second Edition[END_REF]. Finally, many other fields, including the very active image and text classification fields, also address dimensionality reduction [START_REF] Roweis | Nonlinear Dimensionality Reduction by Locally Linear Embedding[END_REF]. It should be noted however that a lot of this literature deals with data reduction (representation of a parametric surface through variable separation) rather than model reduction (generation of small differential equations).

Interface reduction

The global application of subspace classification algorithms without reference to loads leads to modes. To continue addressing the objective of correct representation of inputs, local or interface problems must be considered. Interfaces corresponds to the areas where external or parametric loads are applied (surfaces in junctions, volumes for viscoelastic models, volumes for geometry changes, ...).

Taking the crankshaft shown in figure 5, there are about 5 000 DOF connected to the bearing interfaces (red dots) and the FEM model contains 2 million DOF to allow accurate prediction of stresses. If no special care is taken to build vectors that are zero on large parts (possible here by considering interfaces that contain all nodes of a perpendicular surface), the static part of the basis shown in green will use 2e6 × 5e3 doubles ≈ 74GB. The reduced matrix also has a full block associated with the interface leading to a 5e3 × 5e3 matrix requiring 120 MB storage which must be passed from computer memory to processor at each time step of a time integration thus creating a computational bottleneck.

The Craig-Bampton approach however considers unit displacements at DOF (indicated by the red identity in basis T): one of the interface nodes moves while no other does. Unit displacements guarantee independence but any basis on the interfaces will achieve the same objective. Analytic decompositions (Fourier, orthogonal polynomials) and eigenvalue problems solved on a restricted mesh close to the interface provide much more bases of independent unit displacements that are more easily truncated. Such interface modes are used in automated procedures such as AMLS [START_REF] Bennighof | Meeting the NVH Computational Challenge: Automated Multi-Level Substructuring, International Modal Analysis Conference[END_REF], but can also be quite useful in analyzing loads transmitted through surfaces in contact [START_REF] Vermot Des Roches | Compatibility measure and penalized contact resolution for incompatible interfaces[END_REF][START_REF] Balmes | From physical to macro-models of local assemblies for predictions of global dynamics[END_REF]. Similar techniques can be used to deal with periodic structures [START_REF] Sternchüss | Multi-level parametric reduced models of rotating bladed disk assemblies[END_REF][START_REF] Arlaud | Receptance of railway tracks at low frequency: Numerical and experimental approaches[END_REF]. The limited success of interface reduction methods outside automated procedures may be due to issues with the choice of coordinates (see [START_REF] Boiteau | Modal Solvers and resolution of the generalized problem (GEP)[END_REF] for a bypass) and the need to automate precision checks (see, for example, the work on residue iteration [START_REF] Bobillot | Méthodes de réduction pour le recalage : Application au cas d'Ariane 5[END_REF]).

Piece wise decompositions and sparsity

The discussion of interface modes motivated that computation and memory costs are critical objectives for computational model reduction methods. The variable separation techniques (6) need to be extended by acknowledging that piecewise decompositions will lead to much sparser matrices and thus lower memory and/or CPU costs. Figure 9a illustrates that a full model response {q} can be represented as a combination of bases on components (here individual blade sectors shown with different colors) and associated generalized DOF {q Ri }. If the DOF sets associated with each component are disjoint (this is always possible), the reduced matrix is sparse as shown in 9b. In the test case [START_REF] Sternchüss | Multi-level parametric reduced models of rotating bladed disk assemblies[END_REF], reduction is essential both to allow computations and to store 1500 modes on a 60e6 DOF model which would require 670 GB. Obviously a model reduced this way should still achieve the general load/bandwidth accuracy objectives introduced in section 2.1. This requires fairly automated procedures to be applicable without expert knowledge. When performing a spatial decomposition, loads will be transmitted between components and if the components are reduced independently, as traditionally assumed [START_REF] Craig | A Review of Time-Domain and Frequency Domain Component Mode Synthesis Methods[END_REF], large interfaces are found and truncation, discussed in section 2.4.2, does not solve everything.

A simple procedure to obtain exact interface reduction in the nominal model consists in starting with a basis T valid for the full model and using the subspace generated by the trace of this basis on each component at component basis T i . Obviously the basis is no longer exact if parameters vary, but parameter variations can be accounted for by enriching T . For more details see applications to multi-stage cyclic symmetry [START_REF] Sternchüss | Multi-level parametric reduced models of rotating bladed disk assemblies[END_REF], brake squeal [START_REF] Vermot | Frequency and time simulation of squeal instabilities[END_REF], train/track interaction [START_REF] Arlaud | Receptance of railway tracks at low frequency: Numerical and experimental approaches[END_REF].

It is finally interesting to note that sparsity issues can also be applied to the time domain (Harmonic Balance Method) or to PGD strategies.

Parametric problems :coupling, modal interaction, generalized DOF

While different orientations for reduced model representations were discussed in section 2.4, more interesting challenges are associated with the analysis of parametric problems.

Figure 10 illustrates a brake squeal instability study. Model complexity is due to the fact that lower squeal frequencies (1-4kHz) tend to have significant interactions with the suspension system. A state-of-the art industrial model thus features between 2 and 5 million DOF (geometric complexity), and between 300 and 600 modes in the frequency band of interest below 16 kHz (dynamic complexity). 8 to 15 components, which exhibit material and geometric variability, are coupled by interfaces whose properties vary with operating conditions requiring additional DOF (parametric complexity). Design exploration typically considers sampling over a subset of sensitive parameters to search for robust countermeasures or update parameters when test is available. A full complex eigenvalue extraction for a design point can require a few hours of computation time, challenging relevant design exploration strategies. Here computing 1000 design points would require over 80 days and generate 22 TB of data without post-treatment -clearly unacceptable in terms of design cycles timing and storage capabilities. Model reduction is key to make the model practical in the sense that rather than doing a few hundred computations lasting a couple hours, one spends a couple hours reducing the model and computes a few thousands computations lasting tens of seconds. Another aspect of model reduction is that if you perform restitution on the fly (when you click on a point of (a) you launch the restitution of the shape in (b)), you do not deal with TB of data but just hundreds of MB. There are thus no theoretical limitations to making parametric tools accessible on workstation, rather than large cluster.

The figure however illustrates that the raw result of a parametric experiment is not very understandable. Questions of interest are : is one of those points closer to my experiment, is the experiment sensitive to small changes, is one of the designs better and robust (meaning with little sensitivity to things that are not controlled).

A key question in analyzing parametric problems is thus to have objective strategies to quantify what remains constant and what corresponds to small or large changes. Section 3.1 introduces the idea that many parametric problems can be analyzed as system coupling with variable stiffness studies providing very relevant information about damping, non-linear or variability effects. Tools to track shape changes are then addressed in section 3.2. Finally, section 3.3 addresses the relation between local and global modes.

Coupling : a measure of change in parametric dynamic problems

In control applications, it is common to compare open and closed loop performance. Taking the system perspective, the equivalent of a feedback loop is the part of the dynamics not included in the nominal LTI system. Change can thus be measured through the impact of parameter loads and the section seeks to illustrate how this applies to structural problems.

Following resonance frequencies is a measure of change that is very natural to modal analysts. Figure 11 illustrates the impact of random excitation level on steam exchanger structure with contact non-linearity and the impact of temperature and viscoelastic material thickness in a bracket junction. In both cases, the modal frequency increases with a parameter and in the range where the frequency sensitivity is highest, higher damping levels are achieved. As argumented in [START_REF] Hammami | Numerical design and test on an assembled structure of a bolted joint with viscoelastic damping[END_REF], frequency sensitivity is very much related to the level of energy in the design area (here the contact non-linearity or the viscoelastic joint). The notion is also used to estimate electromechanical coupling in active systems [START_REF] Hagood | Modelling of Piezoelectric Actuator Dynamics for Active Structural Control[END_REF], hence the name coupling retained here.

Figure 11: Left : effect of excitation level for a structure with impact non-linearity [START_REF] Thénint | Definition of a linear equivalent model for a non-linear system with impacts[END_REF]. Right : effect of temperature in a structure with viscoelastic damping [START_REF] Hammami | Numerical design and test on an assembled structure of a bolted joint with viscoelastic damping[END_REF] As motivated in section 2.3, it always possible to represent non-linear parametric problems as an underlying LTI problem (at design point p 0 ) with parametric loads. It is further possible to use modal states of this LTI problem to write system equations under the form

[I + ∆M (p)] {q R } + [∆C(p)] { qR } + \ s 2 + ω 2 j (p 0 ) \ + ∆K(p) {q R (s)} = {f p (q R , qR , p)} (9)
In this formulation, one distinguishes the nominal LTI model, and perturbations on mass ∆M (p), damping ∆M (p) and stiffness ∆K(p). These are however not unique since their effect can also be represented by the parametric force f p . In the contact problem of figure 11 left, the linear system can be considered to correspond to the tube without contact, the tube in full contact or a tube with an intermediate contact stiffness that depends on amplitude. From the point of view of resonances visible in the spectra, the variable contact stiffness/damping provides a good representation of the frequency transition or an envelope for the non-linear response.

A key consequence of illustrations shown here is that in many parametric problems, initial design can simply be performed by doing multiple linear elastic computations with variable stiffness, which is much more accessible than full-fledged non-linear or damping computations.

Observing Degree of Freedom associated with shapes

While observing frequency shifts is very useful, properly understanding modal crossing can be very necessary to understand responses. Section 2.3 illustrated how this can be done in experiments but more efficient tools can be inspired by analysis.

Computing modal amplitudes is much more efficient than using the MAC (shape correlation) for tracking.

Using the orthogonality condition for modes of the reference system {φ j (p 0 )} T [M (p 0 )] {φ k (p 0 )} = δ jk leads to the estimation of modal amplitudes from non-reduced response {q} as

{q jR (t)} = [φ(p 0 )] -1 j {q(t)} = {φ j (p 0 )} T [M (p 0 )] {q(t)} (10)
which corresponds to the perfect modal filter [START_REF] Bianchi | Using modal damping for full model transient analysis[END_REF].

On the vibro-impact case of figure 11 Other analysis applications of modal amplitude tracking are tracking unstable modes in time simulation of brake squeal [START_REF] Vermot | Frequency and time simulation of squeal instabilities[END_REF], study of blade mistuning [START_REF] Sternchüss | Multi-level parametric reduced models of rotating bladed disk assemblies[END_REF], tracking of waves propagating in the soil [START_REF] Arlaud | Receptance of railway tracks at low frequency: Numerical and experimental approaches[END_REF].

Experimentally, modal amplitude estimation was illustrated for time varying squeal (figure 7 and [START_REF] Martin | Squeal measurement with 3D Scanning Laser Doppler Vibrometer: Handling of the time varying system behavior and analysis improvement using FEM expansion[END_REF]), is applied to auto-resonance techniques in [START_REF] Davis | Automatic vibration mode selection and excitation; combining modal filtering with autoresonance[END_REF]. Expansion [START_REF] Balmes | Review and Evaluation of Shape Expansion Methods[END_REF] in subspaces of modes or static responses are related linear observation schemes, which use the current measured shapes to estimate the generalized amplitudes. These techniques can however be usefully extended to estimate more DOF than sensors in the frequency domain (Minimum Dynamic Residual Expansion or Error in Consitutive Law) or in the time domain (Kalman Filtering possibly extended with the estimation of parameters).

Component within a system, local and global modes

The next issue of interest is that one typically redesigns a component with the objective of improving the system performance. However, the component modes of interest are usually not the fixed interface modes found in a Craig-Bampton model. The Component Mode Tuning (CMT) method [START_REF] Martin | Méthodes de corrélation calcul/essai pour l'analyse du crissement[END_REF][START_REF] Vermot | Frequency and time simulation of squeal instabilities[END_REF] was thus introduced to provide a reduced model with DOF that explicitly correspond to the amplitude of free-free component modes. In this algorithm, components are assumed disjoint (this is always possible if it is accepted to use an element layer as interface) and for each component one generates a mass and stiffness orthogonal basis spanning the subspace combining N C component free modes and N G trace on the component of system modes

[T c ] = φ local 1:N C φ global c 1:N G ⊥ (11)
Any appropriate numerical orthogonalization procedure can then be used to generate a well-conditioned basis T c .

In this basis, the first generalized DOF of each component correspond to its retained free modes and the last to corrections needed to recover the nominal system modes exactly. Figure 13 illustrates that in the resulting basis, the mass is an identity matrix, the stiffness is diagonal with squared component mode frequencies ω 2 jc for the first elements, and component coupling leads to block diagonal terms respecting the topology of component connections. The tuning in the name CMT reflects the fact the specific form of matrices makes it simple to perform sensitivity studies on the frequency of component modes. Similarly it is quite simple to sum modal energies to display component energies as illustrated in figure 13 right. The idea of tracking modal amplitude is quite useful in this context. The CMT basis decomposition provides a sound basis to define local modes : significant fraction of energy associated with a given component. For small components, such as the cable guide of figure 6, free component modes may not be relevant and variants of the base CMT procedure were introduced in [START_REF] Martin | Méthodes de corrélation calcul/essai pour l'analyse du crissement[END_REF].

The notions of local and global modes shown here are very much related with the introduction of stochastic models in Ref [START_REF] Arnoux | Stochastic reduced order computational model of structures having numerous local elastic modes in low frequency dynamics[END_REF][START_REF] Soize | A model and numerical method in the medium frequency range for vibroacoustics predictions using the theory of structural fuzzy[END_REF]. These discussions paved the way for the work in [START_REF] Loukota | Modeling of Nonstructural Components in the Dynamic Behavior of Aeronautical Structures[END_REF] which demonstrates how a large number of randomly distributed local modes can lead to significant damping : actually the major source of damping in aircraft modes. The fact that such modes come from so called non structural mass opens a very interesting design perspective.

Conclusion

The presentation sought to illustrate notions related to shapes and modes. Physical responses tend to lie within restricted subspaces even for parametric systems. DOF are amplitudes associated with base vectors generating the subspace of interest. Since a given subspace has an infinite number of bases, DOF are arbitrary in the sense that they can be changed without affecting the response. But some choices make more sense. Modal amplitude DOF can be defined at the system and component levels and tracking these amplitudes is of interest both in tests and analysis. Amplitude extraction seems particularly promising for non-linear, damped, and time varying problems which commonly share the fact that the varying parameter induces coupling. On the computation side, DOF selections leading to sparse bases can lead to gains in CPU and memory requirements which become interesting when the one or more order of magnitude is gained.

Problems of interest go beyond the traditional theory of linear time invariant systems. Extensions of traditional CMS, considers parametric loads, interface reduction, constant basis reduction for parametric models, and sparsity enhancing basis generation. With such models parametric studies with thousands of design points become routine and lead to new challenges in defining procedures to base decisions on large data sets. Finally, the possibility of state (expansion) and parameter (updating) estimation is very much linked to the ability to build a reduced parametric model but the relation between the two needs to be clarified. This presentation will conclude with a few general comments, partially inspired by challenges in A. Pfeiffer's ISMA/USD 2016 keynote.

• The main cost of a model is not computational time but engineering time spent : preparing the right computation, navigating between results of a parametric experiment, designing and analyzing tests, progressing in a design cycle. This implies that each design group should have a digital work-flow allowing parametric studies.

• When everything goes well, the new routine practice is to deal with tens of configurations each containing thousands of computations. The need for companies to manage learning costs typically implies that software vendors pre-package GUI operated processes. But new processes devised for parametric studies typically imply the writing of custom steps (computation of criteria, automated generation of images, ...). A trade-off thus exists between process standardization, implemented as a GUI, and adaptability implemented in the form of a script. On the academic side, the need to focus on teaching GUI or script operation will certainly continue to be an object of debate.

• Outside user process, a few software issues with parametric studies are: the need to perform reduced order computations on the fly using interaction with initial data; the computational architecture between off-line computations (cluster, background) and workstation near real time interaction; scalability of licensing models devised for a few computations rather than thousands.

• It is unlikely that any software provider will ever cover more than 50-70% needs. This is even not desirable since it would mean other groups would stop developing new intelligent strategies. Efficient inter-operability between software packages is thus key. Open source efforts are very good at broad problems with a very large number of users, but often lack vision and focus to answer very technical needs of niche markets. And the noise and vibration community is a collection of such niches. So the inter-operability impulse should come from major user companies forcing/giving incentives to the software providers to develop open API that allow access not only at pre-processing and postprocessing but also at solver levels.

• Non-linearity, damping, junction surfaces, variability, local modes associated with non-structural mass, ... correspond to both challenges (they are difficult to account for in test and analysis) and opportunities (since they have a strong impact on the dynamics, they can be used to improve performance).

• Academics will not convince industry that they are dealing with real problems when using massspring-beam models. Distinguishing geometric complexity (a FEM with the true geometry and thus many DOF rather than analytic equations), system complexity (a CMS model accounting for both bandwidth and loads), and parametric complexity (multiple designs, boundary conditions, operating conditions, ...) is an often ignored path to demonstrating scalability.

Figure 1 :

 1 Figure 1: Direct problem: CAD, FEM modes/reduced models, transfers, time responses

Figure 2 :

 2 Figure 2: Inverse problem : a) time response, b) transfer function and identification, c) analysis of modeshapes, expansion

Figure 3 :

 3 Figure 3: Left : parametric system representation. Right : sample case modified Oberst test

Figure 5 :

 5 Figure 5: Formulation as a combination of LTI problem with additional loads representing parametric effects.

Figure 7 :

 7 Figure 7: Shape and amplitude extraction in a time varying squeal event[START_REF] Martin | Squeal measurement with 3D Scanning Laser Doppler Vibrometer: Handling of the time varying system behavior and analysis improvement using FEM expansion[END_REF]

Figure 8 :

 8 Figure 8: Redefining unit displacements : (a) sample crankshaft example, (b) topology of reduced basis, (c) unit nodal displacement/interface mode (d) topology of reduced matrix

Figure 9 :

 9 Figure 9: Reduction and sparsity: (a) reduced bases with disjoint support (b) topology of reduced matrix[START_REF] Sternchüss | Multi-level parametric reduced models of rotating bladed disk assemblies[END_REF]

Figure 10 :

 10 Figure 10: Left : poles for a 1000 design points, center : sample mode, right : tabular representation of sensitivity parameters. Test case courtesy of Daimler AG.

  left, the estimation from operational deflection shapes shown in figure 12, clearly illustrates that the shape of the dominant resonance transitions from the low frequency bending with no effect of the support plate to an high frequency shape combining the first two modes of the free-free tube. During the transition some torsion effects become visible. The shape transition corresponds directly with the resonance transition visible in 11 left (5 Hz for low amplitude to 30 Hz at high amplitude with additional damping visible during the transition).

Figure 12 :

 12 Figure 12: a) Evolution of modal participations with excitation amplitude, b) Low amplitude shape, c) high amplitude shape.

Figure 13 :

 13 Figure 13: Left : reduced matrix topology in a CMT reduced basis. Right : sample post-processing of component energies

Acknowledgements

The author would like to thank former students from Ecole Centrale Paris and Arts and Metiers ParisTech (ENSAM), whose work helped construct the perspective presented here and provided most illustrations. Special thanks also go to the SDTools engineers J.M. Leclère, J.P. Bianchi, G. Vermot des Roches, G. Martin who contributed to the development of a software environment where ideas can be tested on real life problems. As always, funding is key and thanks must go to those who funded the work presented here : PSA, Chassis Brakes International, EDF, Safran Aircraft Engines, Daimler, ... Finally, the team at KU Leuven should be recognized for the work implied in organizing the top conference on modal analysis and uncertainty.


Trong Phuc Truong

Vincent Nozick
email: vincent.nozick@u-pem.fr

Hideo Saito
email: saito@hvrl.ics.keio.ac.jp

Semi-independent Stereo Visual Odometry for Different Field of View Cameras

Keywords: Stereo visual odometry, Field of View, 3D Reconstruction

This paper presents a pipeline for stereo visual odometry using cameras with different fields of view. It gives a proof of concept about how a constraint on the respective field of view of each camera can lead to both an accurate 3D reconstruction and a robust pose estimation. Indeed, when considering a fixed resolution, a narrow field of view has a higher angular resolution and can preserve image texture details. On the other hand, a wide field of view allows to track features over longer periods since the overlap between two successive frames is more substantial. We propose a semi-independent stereo system where each camera performs individually temporal multi-view optimization but their initial parameters are still jointly optimized in an iterative framework. Furthermore, the concept of lead and follow camera is introduced to adaptively propagate information between the cameras. We evaluate the method qualitatively on two indoor datasets, and quantitatively on a synthetic dataset to allow the comparison across different fields of view.

Introduction

Visual odometry (VO) and simultaneous localization and mapping (SLAM) have been popular research topics in the past decades, and have recently become a prominent part in many emerging technologies such as self-driving car, drone delivery, virtual and augmented reality. Monocular cameras are widely used for these challenging tasks due to their low hardware cost and relatively small size. However, the absolute scale is not observable by using monocular camera approaches without introducing priors, and thus leading to scale drift [START_REF] Engel | Lsd-slam: Large-scale direct monocular slam[END_REF][START_REF] Mur-Artal | Orb-slam: a versatile and accurate monocular slam system[END_REF]. Stereo camera configurations allow to resolve this scale ambiguity by computing the depth from a known fixed-baseline [START_REF] Nistér | Visual odometry[END_REF]. In many stereo VO and SLAM using cameras with overlapping fields of view, two identical cameras are often considered to estimate more efficiently correspondences. The second camera is only used to perform static depth estimation [START_REF] Mur-Artal | Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras[END_REF][START_REF] Paz | Large-scale 6-dof slam with stereoin-hand[END_REF] and/or to add a static constraint term in the optimization [START_REF] Engel | Direct sparse odometry[END_REF][START_REF] Pire | Stereo parallel tracking and mapping for robot localization[END_REF][START_REF] Wang | Stereo dso: Large-scale direct sparse visual odometry with stereo cameras[END_REF]. Temporal multi-view stereo is thus neglected for the latter since the gain of information would not be worth the computation cost.

This paper presents a proof of concept on how a strong constraint on the focal length difference between the two cameras can result in both a higher reconstruction robustness and accuracy. Indeed, when using cameras with different fields of view, performing temporal multi-view stereo for both cameras can become meaningful as the stereo system will be able to exploit more independent source of data when compared to the case of an identical pair of cameras. In theory, a wider field of view allows to avoid occlusion and it is more likely that the visible part of the scene contains well-suited information for visual methods. Visual odometry and SLAM using large field of view fish-eye cameras [START_REF] Caruso | Large-scale direct slam for omnidirectional cameras[END_REF][START_REF] Rituerto | Comparison of omnidirectional and conventional monocular systems for visual slam[END_REF] demonstrate more robust pose estimation, notably during rapid motion, as there is more overlap between subsequent images such that landmarks can be tracked over longer periods. However, the angular resolution of the image decreases as the FOV increases for a fixed image resolution. In [START_REF] Zhang | Benefit of large field-of-view cameras for visual odometry[END_REF], Zhang et al. study the impact of the field of view for visual odometry, and show that large field of view camera should be used in confined environment since features are more evenly distributed which stabilizes the pose estimation and can be tracked for a longer time. On the other hand, due to the loss of angular resolution of higher FOV, the triangulation error is amplified with the depth range especially for large scale outdoor environment such that small FOV cameras should be preferred. In this paper, we propose a semi-independent for stereo visual odometry using cameras with different fields of view so that it can take advantage of both the large and small fields of view properties by performing temporal multi-view stereo for both of them.

Related Work

Using a stereo camera configuration, the scale becomes directly observable given the fixed-baseline, but the implied triangulation can only be estimated for correspondences from both images. As a result, a lot of stereo systems consider a configuration where the common field of view area is maximized.

An early seminal work using a stereo camera setup was proposed by Nister et al. [START_REF] Nistér | Visual odometry[END_REF], where static triangulation and sequential frame-to-frame matching for sparse features were used to estimate the motion with the correct scale of the stereo rig. In [START_REF] Paz | Large-scale 6-dof slam with stereoin-hand[END_REF], Pax et al. present an approach based on extended Kalman filter that considers information from both close and far features. The former provides scale information through the stereo baseline and the latter are represented with an inverse depth parametrization that is useful to obtain angular information. More recently, Mur-Artal et al. present ORB-SLAM2 [START_REF] Mur-Artal | Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras[END_REF], an extension of their monocular SLAM framework based on ORB features [START_REF] Mur-Artal | Orb-slam: a versatile and accurate monocular slam system[END_REF]. The system can work with different configurations such as stereo cameras. It includes loop closing, relocalization, map reuse and follows a similar strategy to [START_REF] Paz | Large-scale 6-dof slam with stereoin-hand[END_REF] by treating differently close and far points.

While these methods are solely based on sparse interest points, recently proposed semi-direct and direct methods have gained popularity due to their ability to circumvent this limitation by exploiting information from all intensity gradients in the image [START_REF] Newcombe | Dtam: Dense tracking and mapping in real-time[END_REF][START_REF] Forster | Svo: Fast semi-direct monocular visual odometry[END_REF][START_REF] Engel | Lsd-slam: Large-scale direct monocular slam[END_REF]. Foster et al. present SVO [START_REF] Forster | Svo: Semidirect visual odometry for monocular and multicamera systems[END_REF], a semi-direct visual odometry, that exploits both photometric error to estimate the initial motion, and geometric error to jointly optimize the camera poses as well as sparse landmarks positions over a window of frames. This method can be easily extended to multiple cameras as the motion estimation and bundle adjustment can be generalized to include measurements from other cameras given their relative pose. On the other hand, full direct methods that only optimize the photometric error also demonstrate state-of-the art results. Based on the work of Engel et al. LSD-SLAM [START_REF] Engel | Lsd-slam: Large-scale direct monocular slam[END_REF], and DSO [START_REF] Engel | Direct sparse odometry[END_REF], extension to stereo camera systems have been presented in [START_REF] Engel | Large-scale direct slam with stereo cameras[END_REF], and [START_REF] Wang | Stereo dso: Large-scale direct sparse visual odometry with stereo cameras[END_REF], in which the authors couple temporal stereo and static stereo in their optimization problem.

Motivation and Contribution

In this work, we propose a framework for stereo visual odometry using cameras with different yet overlapping fields of view. In particular, we consider the combination of a wide-angle (∼ 80 • ) and a medium telephoto lens pinhole camera (∼ 30 • ). With this stereo configuration, our system is able to recover the scale by estimating the depth using static stereo matching from the common FOV as illustrated in Figure 1.

While our method is based on DSO [START_REF] Engel | Direct sparse odometry[END_REF], we extend it to work with a stereo configuration such that information between the two different FOV cameras can be shared. Furthermore, it differs from the stereo implementation presented in [START_REF] Wang | Stereo dso: Large-scale direct sparse visual odometry with stereo cameras[END_REF] as we do not directly introduce any constraint from static stereo in the windowed bundle adjustment pipeline. Instead, the back-end optimization is performed individually for each camera as if it were two independent monocular systems to avoid instability that could arise from photometric error depending on the difference of FOV between the cameras. In other words, the temporal multiple-view optimization is performed by both cameras allowing to take advantage of their respective properties; e.g., angular resolution and robust tracking. Furthermore, we introduce an iterative optimization pipeline such that the least reliable camera is initialized in a way that it is more likely to lie in the basin of attraction of the cost function. The front-end part is also modified to initialize the depth variance of each keyframe with static stereo matching and share the depth map used for tracking such that scale drift can be reduced.

Therefore, the proposed method is designated as semi-independent since the two cameras independently execute monocular VO but their initial parameters are jointly optimized. Our main contributions include:

-A stereo visual odometry using different fields of view that can fully exploit, on the one hand, the precision and robustness of the pose of the large FOV camera, and on the other hand, the angular resolution of the small FOV camera, while recovering the reconstruction scale from the known baseline. -An iterative optimization procedure with efficient front-end frame and point management to avoid the joint optimization of two dissimilar cameras. -Quantitative evaluations on a synthetic dataset with a comparison with DSO and ORB-SLAM2.

Stereo Matching

Estimating the depth using images from different physical cameras but taken at the same time (i.e. static stereo) is an important part of stereo VO since it gives information about the scale as the relative position between both cameras is known. Many stereo systems such as [START_REF] Mur-Artal | Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras[END_REF][START_REF] Pire | Stereo parallel tracking and mapping for robot localization[END_REF][START_REF] Wang | Stereo dso: Large-scale direct sparse visual odometry with stereo cameras[END_REF] use rectified images as input so that the correspondences search can be performed efficiently along horizontal epipolar lines. However, when considering cameras with notably different focal lengths, there is a loss of the FOV for the wide-lens camera due to distortion or cropping depending on the rectification method used. As a result, rectified images cannot be directly input to our system, but they are still used during static stereo matching for computation time. It can be noted that the 3D point computed from the disparity given rectified images needs to be transformed into the original camera frame since unrectified images are fed as input for the VO pipeline.

Since commonly used matching cost functions were not robust to the difference of resolution between the two rectified images, we define an empirical cost function combining NCC with BRIEF binary descriptor [START_REF] Calonder | Brief: Binary robust independent elementary features[END_REF]. It allows to avoid local maxima for the NCC by taking into account a sparse but bigger region using BRIEF descriptors. Furthermore, since rectification practically removes any rotation and scale variance, BRIEF provides a good performance under image blur for a low computation time [START_REF] Heinly | Comparative evaluation of binary features[END_REF]. By defining B n as the min-max normalized L1 distance between the pixel point p 1 in image I 1 and p 2 in image I 2 , and similarly N CC n as the min-max normalized NCC for the same points, the final cost function C is defined as follows

C(p 1 , p 2 , I 1 , I 2 ) = 1 -B n (p 1 , p 2 , I 1 , I 2 ) + λN CC n (p 1 , p 2 , I 1 , I 2 ), ( 1
)
where λ is a weighting factor to balance the influence between NCC and BRIEF.

In our experiment, we use NCC with a 5 x 7 neighborhood, BRIEF with 256 location pairs, and λ = 1.

Fig. 2. Overview of our system, blue parts represent our contributions. After initialization, a lead camera is decided and is firstly optimized with the monocular visual odometry pipeline. Given additional information from the active window of the lead camera, the second camera runs, in turn, the monocular pipeline.

Stereo VO with Wide and Narrow FOV Cameras

We present a stereo visual odometry method using a wide-angle and a narrowangle lens camera that combines multi-view stereo from both cameras and static stereo matching from the overlapping FOV. DSO [START_REF] Engel | Direct sparse odometry[END_REF] is used as the backbone visual odometry framework since it can benefit from its direct and sparse aspects.

In fact, direct method can use every points with high gradient as features so that we can achieve higher resolution point cloud by exploiting the narrow-angle lens camera. Moreover, the sparse nature of DSO allows to save stereo computation time as correspondences are required for a smaller amount of points than dense methods. An overview of our system is presented in Figure 2, where the blue parts represent our contributions.

Direct Sparse Odometry back-end

We adopt the DSO framework as the core visual odometry in our system. DSO proposes a direct probabilistic model with joint optimization of all model parameters including camera poses, camera intrinsic and geometric parameters represented by inverse depths. It is a sparse method that does not incorporate geometric prior so that the Hessian matrix can be solved efficiently using the Schur complement. In [START_REF] Engel | Direct sparse odometry[END_REF], the photometric error for a point p in the reference frame I i , observed in a target frame is defined as the weighted SSD over a 8-point neighborhood N p and is formulated as

E pj = p∈Np w p (I j [p ] -b j ) - t j e aj t i e ai (I i [p] -b i ) γ , ( 2
)
where p is the warped point of p in I j ; t i , t j are the exposure times of the images I i , I j ; a i , a j , b i , b j are brightness affine transfer function parameters; . γ is the Huber norm and w p is a gradient-dependent weighting defined as

w p = c 2 c 2 + ∇I i (p) 2 2
.

(

) 3
Each time a new keyframe is created, it is added to the active window which results in an additional energy factor for every points that can be observed by another keyframe in the window as defined [START_REF] Caruso | Large-scale direct slam for omnidirectional cameras[END_REF]. The full energy is optimized using Gauss-Newton method, and in order to keep the sliding window of bounded size, marginalization is employed to remove the old keyframes.

Semi-independent Stereo VO

Iterative Pipeline. Given a stereo configuration with different fields of view, we propose an iterative approach to avoid the uncertainty coming from the difference of resolution during the photometric error optimization. In fact, when comparing the pixel intensity in a reference frame and the one in a target frame from a camera with a different focal length, the impact of noisy pose or depth estimations can result in the non-convergence of the highly complex optimization. The complexity is even more accentuated as we want to perform temporal multi-view stereo for both cameras. For these reasons, we decouple the problem by performing iteratively two monocular visual odometry pipelines with independent windowed optimizations as illustrated in Figure 2. At each incoming frame, the most reliable camera (lead camera) is first optimized such that its refined parameters can be thereafter shared with the visual odometry front-end of the other camera (follow camera). As a result, the follow camera, that is considered less reliable, is more likely to converge during its back-end optimization process.

Visual Odometry Front-end. The front-end part of the system handles how frames and points are managed. In particular, it decides which frames and points are added and removed from the windowed optimization. Similarly to DSO, new keyframes are required when the current image becomes too distinctive compared to the last keyframe. It is based on three criteria: when the field of view is significantly different, when the translation part of the motion is high, and when the camera exposure time considerably changes. Each time a keyframe is created, well distributed candidate points with sufficient gradient are selected and their inverse depth variance is directly initialized using static stereo. The front-end also provides initializations for new parameters (camera pose, affine transform parameters, and inverse depth of candidate points) required to optimize the highly non-convex optimization in the windowed optimization.

It differs from stereo DSO [START_REF] Wang | Stereo dso: Large-scale direct sparse visual odometry with stereo cameras[END_REF] by taking advantage of having two semiindependent systems running iteratively. When a new keyframe is created, all active points from both windowed optimizations are projected into the latter and then dilated to create a semi-dense depth map used for tracking the pose of new frames. Since feature points are selected to be well distributed, they can be substantially different for the same area of the two cameras due to the difference of FOV, and thus resulting in denser depth map. This depth map is used to track the camera pose of new frames fed into to the system by minimizing the photometric error using direct image alignment. During this optimization, the inverse depth values are fixed and the two-frame direct image alignment is performed on an image pyramid in a coarse-to-fine order.

Moreover, instead of assuming a constant motion model for the follow system, it is directly initialized using the optimized pose of the lead camera given their constant relative pose. This process is particularly important for the narrow FOV camera as it can easily lose its tracking with respect to the last keyframe during fast motion.

Lead and Follow Camera Selection. The selection of the lead camera is critical as an incorrect pose initialization for the follow camera can result in a divergence from the optimal solution. We propose a straightforward metric to select the lead camera by comparing the latest RMSE results from the windowed optimization. Since the narrow-angle lens camera is more likely to converge to local minimum due to its limited FOV, the latter can become the lead only if the following condition

RM SE narrow < f c RM SE wide (4)
is respected 3 keyframes in a row to avoid local minima and maxima results from the optimization. In (4), f c is a factor to decide which camera should be more trusted. We set f c = 0.8 in our experiment to let the small FOV camera leads the large FOV only when the result of the back-end optimization is 20% lower than the one from the other camera.

Asynchronous Initialization

Bootstrapping methods for stereo setup based on an initial depth map from static stereo matching as employed in [START_REF] Mur-Artal | Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras[END_REF][START_REF] Wang | Stereo dso: Large-scale direct sparse visual odometry with stereo cameras[END_REF] will not work efficiently for the wide-angle lens camera. In fact, the estimated depth from static stereo is only limited to the FOV of the narrow-angle lens camera, i.e. the common FOV between the two cameras. We propose an asynchronous method to initialize both cameras in the same coordinate system considering a small overlapping FOV.

Similarly to [START_REF] Wang | Stereo dso: Large-scale direct sparse visual odometry with stereo cameras[END_REF], a semi-dense depth map for the first frame of the small FOV camera can be estimated from static stereo matching to initialize the system. Once the small FOV system has created N i keyframes, the corresponding poses for the large FOV system are computed using the relative pose. Then, the point candidates of its first frame can be tracked and their depth values are refined in the subsequent N i -1 frames by minimizing the photometric error. Moreover, to constraint this discrete search along the epipolar line, all the active points from the small FOV system are projected to the first image plane of the large FOV camera to initialize the associated variance of the candidate points. Finally, the large FOV camera is initialized using the computed depth map and the poses inferred from its counterpart so that both cameras are in the same coordinate system.

Evaluation

For the evaluation of our method, we first demonstrate the ability to reconstruct higher resolution point clouds with two indoor datasets, then we evaluate the odometry on a synthetic dataset to be able to compare its accuracy with ORB-SLAM2 and DSO. Since the aim of this paper is to give a proof of concept about stereo systems with different focal lengths, a runtime analysis will not be detailed. However, with an unoptimized implementation, it runs about twice as slow as DSO considering it has to compute a second time the back-end optimization and estimate stereo matches each time a keyframe is generated.

Point Cloud Reconstruction

The stereo configuration used to evaluate the point cloud accuracy is a 77 • and 32 • FOV camera with a ∼ 20cm baseline. The two datasets contain 800 frames representing around 15 sec of video of an indoor environment. Some examples of input images are illustrated in Figure 3. It also shows the estimated trajectory of the camera pair as well as the point cloud generated using our semi-independent visual odometry method. The accuracy of the registration of both system can be observed by comparing the 3D points from the large and small FOV camera represented in red and green respectively. The color mapping of the trajectory shows that for these two datasets, the camera lead was the narrow-angle lens one. It can be explained by the fact that the motion was relatively slow. However, for both datasets, the lead switches to the big FOV because of the sudden change of direction. The last row illustrates the difference of density between the 3D reconstruction of each camera. In particular, it shows that using a medium telephoto camera, the point selection of DSO can focus more on specific details of the scene. The point cloud is thus more detailed even if the camera pair is at a reasonable distance from the scene.

Odometry Accuracy

We evaluate our method on the Urban Canyon model [START_REF] Zhang | Benefit of large field-of-view cameras for visual odometry[END_REF], where photorealistic synthetic images were generated for a stereo pinhole camera setup with different FOV (40 • , 60 • , and 80 • ) by using cycle raytracing engine implemented in Blender (Figure 4). This way, many stereo configurations for the same trajectory can be proposed to study the impact of the field of view. We compare the accuracy of the visual odometry with ORB-SLAM2 and monocular DSO (since the stereo version is not available publicly). The trajectories are aligned to the ground truth using a rigid-body transform (6DoF) for the stereo methods and a similarity transform (7DoF) for DSO. It can be noted that to allow a fair comparison between all methods, loop closure and relocalization were disabled for ORB-SLAM2, we also disable real-time forcing and we use the default parameters for DSO and ORB-SLAM2. We use three different metrics proposed in [START_REF] Sturm | A benchmark for the evaluation of rgb-d slam systems[END_REF] for our evaluation: the absolute translation RMSE t abs , the relative translation RMSE t rel , and the average relative rotation error r rel .

In the remaining part of this paper, we denote the different results as follows:

-Ours40 : our semi-independent stereo VO using 40 • -80 • FOV stereo camera -Ours60 : our semi-independent stereo VO using 60 • -80 • FOV stereo camera -DSO60 : DSO using 60 • monocular camera -DSO80 : DSO using 80 • monocular camera -ORB40 : ORB-SLAM2 using 40 • stereo camera -ORB80 : ORB-SLAM2 using 80 • stereo camera Table 1 summarizes the visual odometry results for the different evaluated configurations. Their trajectory can be observed on Figure 5. While it does not prove the versatility of our method, Ours40 and Ours60 have the best results for the absolute trajectory error and relative rotation error. The reason is that it is able to exploit the wide FOV camera and it slightly outperforms DSO80 for the absolute trajectory error since information from the narrow FOV is also exploited. While ORB-SLAM2 manages to estimate correctly the relative rotation, the translational error is higher than the two other methods. A reason could be that ORB features are not suitable for this synthetic data since increasing the number of feature points resulted in higher errors in our experiment.

Discussion and Conclusion

In most of stereo VO and SLAM methods, homogeneous camera are considered to take advantage of their overlapping fields of view and the ability to efficiently estimate matches. As a result, temporal information from the second camera is often omitted since it does not provide additional meaningful data to the stereo system when compared to the first camera. In this paper, we suggest that, by using heterogeneous stereo camera with different focal lengths, performing temporal multi-view stereo optimization for both cameras can lead to better 3D reconstruction while having a robust pose estimation. This proof of concept is illustrated by our semi-independent stereo visual odometry for large FOV and small FOV cameras. Some preliminary results show the ability to reconstruct high detailed 3D point clouds while standing at a reasonable distance and to estimate with accuracy the camera pose when compared to DSO and ORB-SLAM2 for the proposed synthetic dataset. While it does not prove that our method is constantly better, it exposes the ability to choose different focal lengths for a multiple cameras setup. For example, this stereo configuration is already present in many smartphones to allow depth of field rendering. Nevertheless, because of the limited range of the static stereo depth estimation due to the small baseline and the difference of FOV, most of common stereo VO and SLAM methods are not suitable. In this case, employing a semi-independent approach could result in a better performance.

Fig. 1 .

 1 Fig. 1. Example of stereo image input with different fields of view: (a) 32 • and (b) 77 • . (c)-(d) Their respective color-coded depth map generated from static stereo matching.

Fig. 3 .

 3 Fig. 3. Qualitative examples on two indoor datasets. (First row ) Example of input images. (Second and third row ) 3D reconstruction of the large and small FOV camera are shown in red and green respectively. The camera trajectory is also represented on the top row, and the same color mapping represents which camera was the lead. (Last row ) A zoomed view of the 3D reconstruction of the large FOV camera displayed on the left and the small one on the right

Fig. 4 .Fig. 5 .

 45 Fig. 4. (a) Top view of the Urban Canyon 3D model. Examples of synthetic images with different fields of view: (b) 80 • (c) 60 • (d) 40 • , each row corresponds to the same camera position.

Table 1 .

 1 Comparison of accuracy in the Urban Canyon dataset. t abs absolute translation RMSE (m), t rel relative translation RMSE (%), r rel average relative rotation error (deg/10 m). Best results are shown as bold numbers

	t abs	t rel	r rel
	Ours40 0.275 1.236 0.896
	Ours60 0.428 3.918 0.486
	DSO60 0.906 1.352 0.612
	DSO80 0.292 0.439 0.622
	ORB40 1.599 2.120 0.533
	ORB80 0.929 1.856 0.543