Nazim Haouchine
email: nhaouchine@bwh.harvard.edu

Parikshit Juvekar

Michael Nercessian

William Wells

Alexandra Golby

Sarah Frisken

Pose Estimation and Non-rigid Registration for Augmented Reality during Neurosurgery

Keywords: Augmented Reality, Image-guided Intervention, Pose Estimation, Registration, Neurosurgery I

causing new neurological deficits. Image-guided neurosurgical systems help register preoperative images to intraoperative patient coordinates so surgeons can view the locations of their surgical instruments relative to their surgical plan [START_REF] Fraser | BrainLab Image Guided System[END_REF] [START_REF] Bucholz | The History, Current Status, and Future of the StealthStation Treatment Guidance System[END_REF]. They also optimize the extent of resection while avoiding damage to surrounding tissues structures. However, commercial systems rely on registration techniques based on patients' skin surface which leads to initialisation errors [START_REF] Frisken | A comparison of thin-plate spline deformation and finite element modeling to compensate for brain shift during tumor resection[END_REF]. Indeed, using the skin surface [START_REF] Kersten-Oertel | Augmented reality in neurovascular surgery: feasibility and first uses in the operating room[END_REF] instead of the brain surface makes it difficult to find strong and reliable features to drive the registration given the shape of the head. In addition the skin surface may deform (particularly in older and obese patients) and registration may thus be innacurate. On the other hand, the brain surface has more structures, such as cortical vessels and sulcal folds that can be used to drive both rigid and nonrigid registration more effectively. In this paper, we present a novel Augmented Reality neurosurgical system to superimpose pre-operative 3D meshes onto a view of the brain surface acquired during surgery. Our method has two main advantages, first, it uses only a single-view image of the brain surface, thus does not need additional acquisition devices and 2) it automatically aligns the 3D models of the cortical vessels generated from preoperative imaging on the intra-operative view of the craniotomy in a non-rigid manner, taking into account initial brain deformation that occurs as a result of the craniotomy as illustrated in Figure 1.

II. RELATED WORK

During neurosurgery the brain moves and deforms [START_REF] Bayer | Intraoperative imaging modalities and compensation for brain shift in tumor resection surgery[END_REF] [START_REF] Miga | Clinical evaluation of a model-updated image-guidance approach to brain shift compensation: experience in 16 cases[END_REF], this deformation, also called brain shift, may considerably impact the surgical outcome since neurosurgical procedures are often based on pre-operative planning where brain shift is not considered. Many methods have been proposed to compensate for brain shift, either by using additional intraoperative imaging data [START_REF] Kuhnt | Brain shift compensation and neurosurgical image fusion using intraoperative mri: Current status and future challenges[END_REF] [START_REF] Reinertsen | Intra-operative correction of brain-shift[END_REF] [10] [START_REF] Rivaz | Deformable registration of preoperative mr, pre-resection ultrasound, and post-resection ultrasound images of neurosurgery[END_REF] or advanced brain models to predict intra-operative outcome pre-operatively [START_REF] Bilger | Biomechanical simulation of electrode migration for deep brain stimulation[END_REF] [13] [START_REF] Essert | Automatic computation of electrode trajectories for deep brain stimulation: A hybrid symbolic and numerical approach[END_REF]. In order to improve trajectory planning during deep brain stimulation, Essert et al. [START_REF] Essert | Automatic computation of electrode trajectories for deep brain stimulation: A hybrid symbolic and numerical approach[END_REF] proposed to learn from a database of brain shift cases to build a geometric template of possible pathways. In the same context, Bilger et al. [START_REF] Bilger | Biomechanical simulation of electrode migration for deep brain stimulation[END_REF] proposed a method to estimate a brain shift risk map using a simulation that gathers statistics on the displacement of anatomical landmarks from different clinical studies. A colorcoded scheme based on vessel proximity is used to visualize this risk map pre-operatively. This method was later improved by Hamze et al. [START_REF] Hamzé | Anticipation of brain shift in deep brain stimulation automatic planning[END_REF] using more advanced biomechanical models. Pre-operative simulation approaches have the benefit of being hardware-independent and do not involve any additional clinical step in the operating room. Nevertheless, they are built using case databases, or generic models that do not depict the actual events that occur during the procedure. Intraoperative data acquisition may be necessary to obtain a precise brain shift estimation.

In Bayer et al. [START_REF] Bayer | Intraoperative brain shift compensation using a hybrid mixture model[END_REF], a non-rigid registration appraoch is used based on vessel centerliens extracted from preoperative and intreoperative C-Arm cone beam CT images. An interpolation warping function is then used as dense deformation field to compensate for brain shift. In Machado et al. [START_REF] Machado | Non-rigid registration of 3d ultrasound for neurosurgery using automatic feature detection and matching[END_REF], intra-operative ultra-sound images and pre-operative MRI are registered using a feature-based method. The extracted features are used to find correspondences between both image modalities and estimate dense mappings throughout the image. An alternative to the features extractor is to rely on user-selected landmarks in order to improve robustness in a clinical routine [START_REF] Luo | A feature-driven active framework for ultrasoundbased brain shift compensation[END_REF]. More advanced deformation models have been proposed to compute deformations. Using intraoperative doppler-based ultrasound, Morin et al. [START_REF] Morin | Brain-shift compensation using intraoperative ultrasound and constraint-based biomechanical simulation[END_REF] measured brain shift at the brain surface and sub-surface vessels thanks to the use of physicsbased models to propagate the deformation to regions of the brain where arteries are not present.

A craniotomy exposes the cortical brain surface and provides an additional source of information to capture intraoperative brain shift. Marreiros et al. [START_REF] Marreiros | Non-rigid deformation pipeline for compensation of superficial brain shift[END_REF] used three Near-Infrared cameras to capture brain surface displacement to register the cortical surface to MRI scans using a point-topoint registration method by considering vessel centerlines as strong matching features. A new deformed MRI volume is generated using a geometric interpolations. The approach presented by Luo et al. [START_REF] Luo | An integrated multi-physics finite element modeling framework for deep brain stimulation: Preliminary study on impact of brain shift on neuronal pathways[END_REF] uses an optically tracked stylus to identify cortical surface vessel features; a model-based workflow is then used to estimate brain deformation correction from these features after a dural opening. Jiang et al. [START_REF] Jiang | Marker-less tracking of brain surface deformations by nonrigid registration integrating surface and vessel/sulci features[END_REF] use phase-shifted 3D measurement to capture 3D brain surface deformations. This method highlights the importance of using cortical vessels and sulci to obtain robust results. The presence of a microscope in the operating room makes it very convenient to deploy such methods clinically. Sun et al. [START_REF] Sun | Near real-time computer assisted surgery for brain shift correction using biomechanical models[END_REF] proposed to pre-compute brain deformations to build an atlas from a sparse set of image-points extracted from the cortical brain surface. These image-points are extracted using an optically tracked portable laser range scanner. Haouchine et al. [START_REF] Haouchine | Deformation aware augmented reality for craniotomy using 3d/2d nonrigid registration of cortical vessels[END_REF] proposed to use a finite element model of the brain to propagate cortical deformation captured from the microscope to substructures. This method uses cortical vessels to drive a non-rigid point-to-point registration. However, a manual rigid alignment is needed to initialize the system and obtain the point-to-point correspondences. Mohammadi et al. [START_REF] Mohammadi | Estimation of intraoperative brain shift by combination of stereovision and doppler ultrasound: phantom and animal model study[END_REF] proposed a projection-based stereovision process to map brain surfaces with a pre-operative finite element model. A predefined pattern is used to recover the 3D brain surface. In order to build a dense brain surface and gather more precise positions, Ji et al. [START_REF] Ji | Cortical surface shift estimation using stereovision and optical flow motion tracking via projection image registration[END_REF] proposed a stereo-based optical flow shape reconstruction. The 3D shapes are recovered at different surgical stages to obtain undeformed and deformed surfaces. These surfaces can be registered to determine the deformation of the exposed brain surface during surgery. Nevertheless, most of the surgical microscopes need tedious pre-calibration which limits the scope of use of 3D stereo reconstruction.

Contributions and Motivations

Previous approaches have shown that it is possible to exploit the brain surface instead of the patient's skin surface to perform preop-to-intraop registration. However, these methods need manual registration to initialize the system. This cumbersome initialization is an obstacle to the adoption and deployment of such approaches in the clinical routine. We propose a novel approach to tackle this problem using an efficient approach to infer the 3D pose of the pre-operative mesh from its projection on a single-view image. In comparison to previous methods, we rely solely on a monocular single-view image acquired from the surgical microscope (or possibly a ceiling mounted camera) , to avoid tedious calibration of the stereo camera, laser range finder or optical stylus. Instead of using the raw image of the brain surface, we rely on probability maps that extract features that outline the vessels from its background. These features are fed into a deep neural network to infer a 6-DoF rigid pose to permit a coherent mesh-to-image superimposition. To train this deep neural network, we synthesize a set of projected probability maps from patients' 3D preoperative meshes, using various rigid transformations. Following rigid registration, we perform a 3D/2D non-rigid registration by satisfying a combination of physical and projective constraints using a shape-fromtemplate formulation. The results demonstrate that our solution is fast, easy-to-use and achieves low errors, thus demonstrating the potential for improving surgical guidance.

III. METHOD

A. Overview

As illustrated in Figure 1, given a 3D mesh of the cortical vessels M and an input image of the brain surface I, our solution recovers the 3D pose y corresponding to the camera rotation and translation required to align the mesh M onto the image I, and a displacement field δu that updates the vertices of M and describes the brain shift deformation. Our method is a three-stage solution composed of: 1) a feature extractor neural network S that estimates, from the input image I, a probability map that extracts the cortical vessels from the parenchyma; 2) a pose estimator network P, that given the previously extracted probability map, estimates the rigid 3D pose (rotation and translation) and 3) a non-rigid refinement stage, that given the camera pose, will deform M in a physically-realistic manner. In order to train the network S, images of the brain surface are synthesized using a styleenforced neural image analogy, while a mapping network is used to synthetize probability maps from random rigid projections of M. Composing the networks S and P will result in a predicted pose y that is used to align M on I. Once aligned, a force-based Shape-from-Template problem is solved to estimate δu and deform the mesh at cortical and sub-cortical levels.

B. Segmentation of Cortical Vessels from Brain Surface

Let us define the training set T S = {(I i , L j )} i composed of images I i of the brain surface and their corresponding binary labels L i , with i ∈ |T S | We first start by training the feature extractor S(I; θ S ) with θ S being the learned parameters for network S. We use the feature extractor to generate a perpixel probability map P that will associate with each pixel of an image I the probability of that pixel being part of the vessels or the parenchyma.

The network S is a deep CNN that follows a typical U-Net architecture [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] with a Softmax final layer. We optimize a categorical cross-entropy loss function using gradient descent over the parameters θ S following:

L(θ S , T S ) = i∈|T S | S(I i ; θ S ) -L i 2 (1)
The model was trained over 94 epochs using mini-batches of size 8 with Adam optimizer and a learning rate of 0.001. In order to train S so that it generalizes well to new data, we needed to increase the amount of available training data T S through data augmentation. We used neural image analogy [START_REF] Nercessian | Deep cortical vessel segmentation driven by data augmentation with neural image analogy[END_REF] to generate an arbitrarily large and variable dataset of artificial images resembling real craniotomy images. This technique combines the style and texture of a source image with the high-level content representations of a target image. In our case we enforced the style on the source image over other features leading to generating many unique images that replicate the style of the craniotomy images. We randomly generated vessel-like binary images to mimic real craniotomy image labels using a procedural algorithm (see Figure 5). Then, using a limited set of real craniotomy images, we could generate a larger dataset composed of the pairs {(I, L)}, which was then used to train S. Fig. 2: Generating mesh projections. Using an input mesh M, we generate binary images by randomly varying the 3D pose of a virtual camera (black dots) on the upper hemisphere of the 3D mesh.

C. 3D Pose Estimator

In order to align a 3D mesh model with its projection in a 2D image, we need to solve for its 3D pose (translation and rotation). In the general case, assuming a set of 2D coordinates v = {v i ∈ R 2 } extracted from the image and their corresponding 3D coordinates u = {u i ∈ R 3 }, with i ∈ |M|, solving for the 3D poses amounts at minimizing the reprojection error:

i∈|M| A[R|t]u -v 2 2 (2)
where R ∈ R 3×3 and t ∈ R 3 represent the 3D rotation and 3D translation, respectively, and A is the camera intrinsic matrix composed of the focal length and the principal points corresponding to the center of the image. Therefore, assuming a known focal length and using the quaternion representation for the rotation, there are altogether seven parameters to be estimated: rotations q x , q y , q z , q w and translations t x , t y , t z .

In order to solve Eq. 2, we propose to train a pose estimator network P(P; θ P ) over the parameters θ P . We use the synthetic training set T P = {(P i ; y i )} i , where P i is the input probability map and y k = [q x , q y , q z , q w , t x , t y , t z ] T i is the output 3D pose; with i ∈ |T P |. This set is generated by creating binary projections of the 3D model M by randomly varying the 3D pose of a virtual camera (see Figure 2). Because the brain surface may deform due to brain shift [START_REF] Frisken | A comparison of thin-plate spline deformation and finite element modeling to compensate for brain shift during tumor resection[END_REF], the preoperative model may differ from its expected image projection intraoperatively. We thus define a function D : R 2 → R 2 that deforms the previously generated binary image. We can now apply, in addition to the rigid transformation, a set of non-rigid transformations to generate a set of binary images representing the plausible projections of the model M during surgery.

As mentioned above, we use probability maps instead of binary images since in practice segmenting the cortical vessels will produce erroneous vessels/background classification that will lead to mis-alignments. In addition, binary images may discard meaningful information especially at the frontiers of classes that are present in the probability maps. We therefore train a mapper network M(L; θ M ) with parameters θ T to translate binary images to probability maps. In order to train this network, we use the training set T M = {(L i ; P i )} i . This set makes use of the already available binary images used for the style-enforced analogy L i and their corresponding inferences P i ; with i ∈ |T M |. Putting all together, we can synthesize a new probability map following:

P i = D M(Π proj • M; θ M ) (3)
where Π proj is a virtual projection matrix that assembles the varying rotation and translation, in addition to pre-defined camera parameters. In practice, we only vary the rigid pose on the upper hemisphere of the 3D mesh to remain consistent with the potentiel camera positions w.r.t. patient's head during the craniotomy.

The network architecture of P consists of 3 hierarchies composed of two convolutional and one ReLU layers. To decrease the spatial dimension, an average pooling layer with a stride of 2 follows each hierarchy except the last one. At the end of the last hierarchy we add one fully-connected layer with 128 neurons and ReLU activation followed by one fullyconnected with 7 neurons with a linear activation. We finally optimize the following loss function over the parameters θ P of the network P : L(θ P , T P ) = i∈|T P | P(P; θ P ) -y i 2

(4)

The model was trained over 142 epochs using mini-batches of size 8 with Adam optimizer and a learning rate of 0.001. Finally, at run-time, given an image I we predict the corresponding pose y by composing the networks S and P together:

y ← P(S(I; θ S ); θ P ) (5)
where θ S and θ P are the resulting parameters from the training.

D. 3D Vascular Network Modelling

In order to simulate brain deformation, we build from the pre-operative vessels geometry a mechanical model capable of handling 3D non-linear deformations in a consistent manner. We rely on a wire-like model to represent the behaviour of cortical vessels. This model has previously been used to simulate guide wires and catheters [START_REF] Cotin | New approaches to catheter navigation for interventional radiology simulation[END_REF] and vessels [START_REF] Haouchine | Deformation aware augmented reality for craniotomy using 3d/2d nonrigid registration of cortical vessels[END_REF]. In order to build this model, we first extract vessel centerlines from the 3D mesh M, and build the model by connecting the 3D nodes u along the centerline in sequence. Each beam element is delimited by two nodes each having 3 spatial and angular degrees of freedom (see Figure 3-a). A stiffness matrix K e is then defined to relate the node positions to the forces applied to them accounting for rotational degrees of freedom through a rotational matrix Q e . At each node i, the internal forces f i generated by the deformation of the structures is formulated as:

f i = i e=i-1 Q e (u)K e (Q e (u) T (u -u e ) -u rest ) (6)
where e is the index of the two beams connected to this i th node. u i-1 , u i and u i+1 are the degrees of freedom vectors of the three nodes (respectively i -1, i, i + 1) and belong to the two beams in the global frame. u ej denotes the middle frame of the j th beam that is computed as an intermediate between the two nodes of the beam, and u rest corresponds to the degrees of freedom at rest in the local frame.

Furthermore, in order to propagate the deformation from cortical to sub-cortical level, and more particularly to tumors, we rely on a linear geometrical barycentric function φ(•) that can act as a mapping between the surface and the underlying tissue (see Figure 3-b;c).

Formally speaking, if we denote r the vector of vertices of the tumor model, we can express each vertex r i using barycentric coordinates of facet vertices u, such that

r i = 3 j=1 φ j (x i , y i , z i )u j (7)
where φ(x, y, z) = a + bx + cy with (a, b, c) being the barycentric coordinates of the triangle composed of nodal points u j , with 1 ≤ j ≤ 3. Since craniotomy are performed to access structures beneath the opening region, this mapping is sufficient to propagate deformations to the immediate subcortical structures.

E. Force-based Shape-from-Template

Estimating a 3D deformed shape from a single projection knowing the 3D shape in rest configuration is in an ill-posed problem also known as Shape-from-Template [START_REF] Bartoli | Shapefrom-template[END_REF]. Its illposedness comes from its under constrained nature, where many 3D shapes may have identical 2D projections. In order to solve this problem, we can add constraints to reduce the space of plausible solutions. We chose to rely on physical constraints to force the 3D estimate shape to follow the laws of mechanics according to the beams model introduced in Eq. 6. In our case, the 3D shape at rest configuration consists of the pre-operative 3D centerlines u while its 2D projection consists of a set of points v extracted from the probability map along the vessel centerlines. The unknown 3D deformed shape δu is the displacement field induced by a potential brain shift. Using the previously estimates pose and assuming a known camera parameters A from pre-calibration, we can build the projection matrix Π = A[R|t] . Using Π we can establish a set of correspondences c so that if a 2D point v i corresponds to a 3D point u j then c i = j for each point of the two sets. Deforming the 3D vessels so they fit their corresponding 2D projections amounts to minimize the re-projection error Πu ci -v i for i ∈ n c while ensuring minimal internal forces f = nc i f i emanating from the vascular mechanical model. Combining these two constraints leads to the following force minimization expression:

argmin δu nc i f ci -κ Πu ci -v i 2 (8)
where κ is the stiffness coefficient that permits translation of the image re-projection error to an image bending force. Note the subscript c i that denotes the correspondence indices between the two point sets. This minimization can be seen as enforcing the 3D vessel centerlines to fit the sightlines that come from the camera position to the 2D vessels centerlines while maintaining a coherent 3D shape. It is computationally inexpensive to solve when formulated as linear complementary problem and solved using a Gauss-Seidel algorithm [START_REF] Haouchine | Deformation aware augmented reality for craniotomy using 3d/2d nonrigid registration of cortical vessels[END_REF].

Fig. 4: Force-based Shape-from-Template, problem formulation: we aim at recovering the deformed 3D vessels shape δu from its known reprojection in the image v, the known pre-operative 3D vessels at rest u and known rigid alignment, satisfying physical and reprojective constraints.

IV. RESULTS

A. Dataset

To build the training set T S we generated 1856 images using neural image analogy from 10 craniotomy style source images and 71 annotated inputs. These images were randomly split and 25% of the images were kept for validation. The model achieved an 0.71 IoU and 0.82 Dice accuracy on the validation set. Examples of probability maps generated using S are shown in Fig. 7. We tested our method retrospectively on six clinical datasets (see Fig. 7). These consisted of T1 contrast MRI scans and images of the brain surface after dura opening. The cortical vessels and the tumors were segmented and triangulated to generate 3D meshes using 3D Slicer (www.slicer.org). Only a subpart of the cortical vessels is used. The microscopic images (left image of the stereoscopic camera) were acquired with a Carl Zeiss surgical microscope. The ground-truth pose is obtained by manually aligning the 3D meshes on their corresponding craniotomy images.

To build the training set T P we generated around 1400 synthetic probability maps per case. We randomly sampled rotation and translation on the upper hemisphere of the 3D mesh. These images and poses were randomly split where 25% were kept for validation. We chose the average distance metric (ADM) as proposed in [START_REF] Hinterstoisser | Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes[END_REF] for evaluation. Given a set of object's 3D vertices, the ground-truth and estimated rotations and translations, the ADM computes the mean of the pairwise distances between the 3D model points transformed. Using this metric, a pose is considered correct if the average distance is lower than a distance threshold (usually 10% of the object's diameter).

B. Pose Estimation

We first vary the distance threshold on the validation sets of the dataset in order to reveal how our method performs w.r.t that threshold. We plotted an accuracy-threshold curve showing the ADM variation with a threshold in a range of 0 mm to 20 mm. We can see in Figure 6 that for most of the sets, a 80% pose accuracy is reached with a threshold below 5 mm, except for case 2 and 3.

We then conducted experiments to analyze the effect of using probability maps. We compared our method (Our) with a baseline silhouette-based approach (SIL). The later approach estimates the pose using a binary image that corresponds to the silhouette of the 3D object. We report in Figure 8 the distance between the ground-truth and estimated pose where we can notice that our method outperforms the silhouettebased approach with average ADM errors of 4.13 ± 0.70 mm compared to 8.67±2.84 mm. These errors should be compared to those of the study in [START_REF] Frisken | A comparison of thin-plate spline deformation and finite element modeling to compensate for brain shift during tumor resection[END_REF], which reports a 5.26 ± 0.75 mm initialization error measured on 15 craniotomy cases when using commercially-available systems. Figure 7 shows our results as bounding boxes and overlaid meshes. Our methods perform well with case 3, 4, 5 and 6 with visually consistent alignements. Nevertheless, it should be noted that our method may produce misalignment errors as in case 1 and 2. This suggests that our model does not properly handle occlusions (case 1) and that the pose accuracy may decrease if the cortical vessels are only partially visible (case 2). However, in all cases, our predictions remain always close to the ground-truth.

From the computation time standpoint, it takes 2.07 seconds to extract the probability map and 1.23 seconds to predict the pose. We used the Tensorflow/Keras framework (www.tensorflow.org) on an NVidia GeForce GTX 1070 without any specific optimization.

C. Non-rigid Refinement

To evaluate the non-rigid refinement we first used simulated data of a 3D synthetic human brain. Using a mean curvature skeletonization technique we extract 3D vessel centerlines from the vessels mesh surface. The wire-like model is built upon the centerlines nodes with a subset nodes in range between 12 and 30 nodes. The mechanical parameters of the vessels are taken from a textbook [START_REF] Ebrahimi | Mechanical properties of normal and diseased cerebrovascular system[END_REF] with a Young's modulus set to 0.6 MPa and Poisson's ratio to 0.45 to simulate a quasi-elastic and incomprehensible behaviour where vessels thickness is set to 0.5 mm These parameters are used to assemble the stiffness matrix of the vessels that will describe its elasticity. We used the Sofa framework (www.sofaframework.org) to build the 3D beam-elements model.

To measure how the cortical deformation is propagated to sub-cortical levels, we measured the target registration error (TRE) on different locations. We also quantify the percentage of brain-shift compensation by measuring the difference between the initial and the corrected error (normalized by the initial error). This measure is important since the TRE depends on the amount of brain shift. We randomly placed 4 landmarks inside the simulated brain at different depths from 0 mm to 45 mm, 0 mm being the cortical level. The simulation of brain shift consisted of a protrusion deformation of 4 increasing amplitudes to mimic brain swelling after the craniotomy, Def1 = 1.7 mm, Def2 = 4 mm, Def3 = 6 mm and Def4 = 10 mm. We projected the deformed 3D centerlines w.r.t a known virtual camera to obtain the images and added Fig. 9: The first row shows qualitative results obtained on our dataset retrospectively. The contours correspond to the preoperative 3D meshes projected using the non-rigid refinement (red), and the estimated poses (green). Our predictions remain always close to the ground-truth. The second row shows a color-coded estimated stress on the vessels. Gaussian noise with a standard deviation of 5 mm and a 5 clustering decimation on the set of 2D points extracted from the images. We can notice in Figure 10 that using our method achieves small TRE ranging from 0.53 mm to 1.93 mm, on the cortical level and the immediate sub-cortical structures (≤ 15 mm). Brain shift compensation of up to 68.2 % is achieved, and at least a 24.6 % compensation is reported for the worst configuration. We can also observe that errors and compensation at the cortical level is very close to the immediate subcortical location (≤ 15 mm). It increases when the targets are locate deeper in the brain (≥ 30 mm) or the amount of deformation increases (≥ 6 mm). Finally, we show in Figure 9 the measured deformation amplitude on each vessel after non-rigid registration with color-coded stress visualization as well as the projection of the deformed vessels on the image.

V. LIMITATIONS

a) Robustness to Occlusions: As reported in the results, our method is limited when the 3D mesh is partially visible, as seen in case 1 and 2. Expanding the training set with occluded cases would permit handling of occlusions properly. Moreover, a minimal amount of visible vessels could naturally be considered as a hard condition for the success of our solution. Another possibility is to combine the vessels with the sulci which are more likely to be visible during a craniotomy and can also be segmented from the MRI scans.

b) Microscopic Camera Focal Changes: Although a camera calibration is a well-established technique to build the camera parameters A, during a craniotomy, the focal length of the microscope may change and the pose estimation may be inaccurate. Including the camera focal length in the prediction can be done by adding a new element to the prediction vector y and training the model to take into account these focal length changes. This future work is aligned with our objective to make such a technique easy to use in the operating room.

c) Error Propagation: Although we reported errors below actual clinical practice, neurosurgery requires very accurate registration. The proposed solution does integrate potential brain deformations but solely as a refinement step. Handling deformations can create ambiguities when estimating the pose thus leading to registration errors. In addition, an inaccurate rigid registration will undoubtedly lead to an inaccurate nonrigid refinement. Our way to reduce error propagation consists of building a full model capable of predicting an n-DoF pose, accounting for rigid and non-rigid components simultaneously.

VI. CONCLUSION

We proposed a fully automated brain-shift aware Augmented Reality method for craniotomy guidance. Our method first performs a rigid alignment of the 3D pre-operative mesh onto a single-view image of the brain surface, then refines the alignment by deforming the 3D mesh to compensate for brain shift. The rigid registration removes the curbesome step of manually aligning a 3D mesh onto its projection on an image in the context of craniotomy, while the nonrigid refinement step permits updating the underlying tumor's location. Restricting our method to a single-view image makes it more adequate to operating rooms but turned the registration process into an ill-posed problem that we solved using an elegant end-to-end solution.

The proposed method was evaluated both quantitatively and qualitatively on a clinical dataset where we demonstrated that our method is fast and can achieve ADM errors below current clinical margins. In addition, we showed that low TRE can be obtained at cortical and sub-cortical levels and large brain shift compensation can be achieved. We strongly believe that this method is adequate for the use in the operating room, thanks to its high degree of automation and low level of surgeon interaction.

Fig. 1 :

 1 Fig. 1: Our pose estimation model is composed of a feature extractor S and a pose estimator P. The feature extractor generates from the input image I a per-pixel probability map P to highlight the cortical vessels. The pose estimator uses the probability map to predict the 3D pose y. The resulting pose y is used to overlay the pre-operative model M on the surgeon's view I. A non-rigid refinement follows to estimate brain shift and updates tumor's position.

Fig. 3 :

 3 Fig. 3: 3D vascular model and mechanical mapping. (a) Serially-linked beams on the mesh centerlines. Each beam element (represented by the frames) is delimited by two nodes that relate to the forces applied to them w.r.t a stiffness matrix. The wireframe mesh represent the vascular network. In (b), a visualization of the mapping mechanism with blue lines represents the attachments.

Fig. 5 :

 5 Fig. 5: Samples from dataset T S used for training the segmentor network S.

Fig. 6

 6 Fig. 6: Accuracythreshold curves with average distance error on all validation sets.

Fig. 7 :Fig. 8 :

 78 Fig. 7: Retrospective results on six cases. For each case the probability map (left) and the estimated pose are shown (right). The input images are displayed in the bottom left corner of the probability maps. Cases 1 to 6 are ordered from top to bottom, left to right.

Fig. 10 :

 10 Fig. 10: Quantitative evaluation on simulated data: Charts (a) and (b) show the percentage of compensation and TRE respectively w.r.t to depth and brain shift amplitude. 0 mm in the "depth" axis represents the cortical level.

ACKNOWLEDGEMENTS

This work was supported in part by the National of Health under Grant R01 EB027134-01, R01 NS049251 and BWH Radiology Department Research Pilot Grant Award. Human datasets were used in this work. Approved, recognized procedures were followed for human subject inclusion.


Christine Chambris

CHANGES IN THE TEACHING OF NUMBERS AND RATIOS IN THE PRIMARY CURRICULUM

Keywords: Numbers, ratios, New Math, reference knowledge, didactic transposition

INTRODUCTION

In the 1960s and 1970s, in many countries throughout the world, the New Math Reform aimed at renewing the teaching of mathematics from the primary school up to the university, and at increasing mathematics proficiency in the general public in the context of the cold war [START_REF] Furinghetti | ICMI Renaissance: The emergence of new issues in mathematics education[END_REF]. One of the major concerns of the New Math was the teaching of some "new" math. This enables to make the assumption that this period showed major changes in the mathematical foundations of basic arithmetic. The case of the teaching of set theory in primary schools in these years is a famous example. Another concern of the reform was to take into account some psychological features related to the learning or to the child development. The famous psychologists Piaget and Bruner contributed at least indirectly but significantly to the implementation of this reform. It has often been said that New Math failed, however "In no country did school mathematics return to where it had been before the new math movement began: The pendulum is not a suitable metaphor for curriculum change." (Kilpatrick, 2012, p.569).

The aim of this paper is thus to identify some of the changes promoted by the New Math, and their long term effects. It will focus on the teaching of numbers, operations, and ratio in the primary curriculum in the French curriculum 1 . Hence, I aim at identifying possible changes in mathematical foundations for numbers and ratios surrounding the New Math, and possible effects of these changes on the curriculum up to nowadays. 1 In this note I indicate some specific uses that may be helpful to better understand some of the data presented after. In France, at present time, there is no use of comma to write great numbers, a space is required. It seems a century ago, at least some people were using a point. Moreover, for writing decimal numbers -with the positional notation-a comma is required to indicate the unit. Hence, 30 000,4 means thirty thousand and four tenth. With regard to multiplication, the sign x is used between two numerals: 13 x 15. In algebra, "x", "." or no sign can be used to indicate multiplication between two letters or between a numeral and a letter.

METHOD

Theoretical frame

The Theory of Didactic Transposition (TDT) [START_REF] Chevallard | La transposition didactique[END_REF] considers school mathematics as a reconstruction by the educational institutions from the mathematical knowledge produced by academic scholars. Four steps are distinguished (fig. 1): scholarly knowledge, syllabi and textbooks, teachers, then students. The Anthropological Theory of Didactics (ATD) (e.g. Bosch and Gascon, 2006 for a reference in English) extends the TDT. It postulates that practicing mathematics, as any human practice, can be described with the model of praxeology. It is constituted by four components: a type of tasks -a set of similar problems-, a technique -a "way of doing" for all the tasks of the type-, a technology justifies the technique, and a theory legitimates the technology. In this model, the reference knowledge is the scholarly knowledge. The ATD introduces the "ecology of knowledge" metaphor to describe the evolution of teaching objects. For a given object at a given moment, it defines: its habitats (where it lives) and ecological niches (its roles in each of its habitats). It also considers "trophic chains": A needs (to eat) B to live. Thus, the ATD asserts the following paradox: To be eaten is a reason to live in the teaching system.

Academic knowledge

In the history of mathematics, the development of numbers has been closely connected to the measurement of quantities, especially that of continuous ones (e.g. Book V by Euclid). But, since 1870 approximately, in academic mathematics, numbers have no longer been elaborated from quantities but from whole numbers, then from sets [START_REF] Bourbaki | Étude didactique des nombres réels. Idécimalité et racine carrée[END_REF]. Did such a change impact the primary curriculum? To what extent has present academic knowledge impacted the primary curriculum during the New Math, and still does? Background Bronner (1997) highlights five main periods in the teaching of numbers in the French system: the stable classical period 1870-1950, the New Math highly turbulent period 1970-1980, the continuously evolving contemporary period 1995-2010, and two transitional ones before and after the New Math. Treatises by [START_REF] Bezout | Cours de mathématiques à l'usage des gardes du pavillon et de la marine, Eléments d'arithmétique[END_REF], then Reynaud (1821) appear as reference books for primary teacher education, and textbooks by 1940.

Research questions

What are the habitat and niche of quantities in the different praxeologies during the different periods? Are there traces of changes of "reference knowledge" for numbers, operations, ratios and quantities during the different periods, especially around the New Math?

Data

The studied period is huge. In order to get an overview of the curricula, data have been first restricted to the national French syllabi (1882, 1923, 1945, 1970, 1977-1980, 1985, 1995, 2002, 2008) for grades 1 to 5 and to previous works studying teaching of arithmetic throughout the 19 th or 20 th century. Several 2 ndand-3 rd -grade textbooks of have been included after. Mathematical treatises and books for teacher education inform us about scholarly or reference knowledge.

Methodology

First, an extensive analysis of syllabi is conducted in order to identify each occurrence of discrete and continuous quantity. This enables to locate their habitats, and then the role of quantity is interpreted in term of praxeology (niche). Similar analysis is conducted with treatises, textbooks analysis enables to refine, infirm, or confirm the previous analyses on specific topics (addition and multiplication). According to the collected data -i.e. syllabi and textbooks-, only the two first steps of the didactic transposition (Fig. 1) is considered.

FINDINGS

Before the New Math The syllabi from 2 nd grade (1923) even 1 st grade (1945) gather, in the same paragraphs, discrete and continuous quantities: indeed both types of measurement are involved for whole numbers, operations, and place valuewhich is clearly connected with the metric system-. Moreover, in the 4 th grade, in 1945, fractions are fractions of quantities and decimals are introduced using the metric system. Before 1940, the teaching of arithmetic is much inspired by the treatises by [START_REF] Bezout | Cours de mathématiques à l'usage des gardes du pavillon et de la marine, Eléments d'arithmétique[END_REF] then [START_REF] Reynaud | Notes sur l'arithmétique, à l'usage de l'école polytechnique et de l'école spéciale militaire[END_REF]. This influence remains up to 1970. Both treatises present an arithmetical theory relying on measurement of quantities for numbers and ratios.

In order to highlight the didactic transposition, I present excerpts of the beginning of Reynaud's treatise and a page of a textbook (fig. 2) of the beginning of the 2 nd grade.

First chapter. Numeration, Addition, Subtraction, Multiplication, and Division of whole numbers. 1. Whatever is capable of increase or diminution is called quantity. When one thinks about the nature of quantities, one feels it would be impossible to have an exact idea of magnitude of quantities of the same kind without considering one of these magnitudes as a term of comparison; this magnitude is called unit; combining several units of the same magnitude forms a number. The manner of forming, expressing, and writing numbers is the object of numeration; and the science which teaches how to perform operations upon numbers is called Arithmetic.

Numeration (…)

3. A number is abstract or concrete when no particular denomination is mentioned to which its units belong or not. Thus, 3 and 5 times are abstract numbers; 3 toises and 5 leagues are concrete numbers (…). Within whole Arithmetic, as the nature of the units is known in advance, one only has to find their number; this leads to operate on abstract numbers. (Reynaud 1821, p.1 and p.6), my translation. In the treatise, numbers emerge from the measurement of quantities. Within the textbook's page, the words "concrete" and "abstract" are not used. Whatever, three cases are taken for talking about addition. Putting together firstly discrete quantities (1. collections of pears), secondly continuous quantities (2. adding length through two pieces of a fishing rod) provide two cases with concrete numbers. A third case using abstract numbers is elaborated from the two first cases (fig. 2).

On figure 3 -a page of the beginning of the 3 rd -grade textbook, published in 1932-, the themes of the word problems may seem old-fashioned. I don't know whether they were familiar or not to the children of the 1930s. The structure of the lesson is common for the period: the rule is established using "solved problems", and then it is stated. Exercises -not shown-are proposed after: simple and complex word problems involving multiplication the rule. This excerpt reveals several clues related to the underlying reference knowledge. They may not be easily visible for an unaware reader.

Multiplying by several tens or hundreds

Multiplying an amount of tens. Problem A box of quills is 60 g heavy. How heavy are 72 boxes? The weight is 72 times 6 tens of g.; 6 tens x 72 = 432 tens; 432 tens= 4.320 g. Solution. The total weight of 72 boxes is: 60 g x 72 = 4.320 g. Multiplying by an amount of tens. Problem How much do 60 barrels of 72 l. each contain? A ten of barrels contains: 72 l. x 10 = 720 l. 6 tens of barrels contains: 720 l. x 6 = 4.320 l. Solution. The total capacity of the barrels is: 72 l. x 60 = 4.320 l. Rule and practical disposition. To multiply an amount of tens 60 by 72, or to multiply 72 by the amount of tens 60, one multiplies 72 by 6 and one put a zero at the right side of the product. [column computation] Practically, one writes first the zero, then the product one the left. First, I go back to the treatise:

6. The purpose of the multiplication is to calculate a number which is called product. The latter is formed with a known number called multiplicand, similarly as a given number -called multiplier-is formed with the unit. So, to obtain the product, one just has to operate on the multiplicand the same operations one would have operated on the unit to form the multiplier. Multiplicand and multiplier are the factors of the product. For instance, for multiplying 5 by 3, one can observe that the multiplier is formed with three times the unit. So the product must be formed with three times the multiplicand 5; this product is thus 5 plus 5 plus 5 or 15. Generally, when the multiplier is a whole number, multiplication is simply repeating the multiplicand, as many times as there are units in the multiplier.

The multiplier is always an abstract number because it indicates how many times the multiplicand has to be taken. The product is of the same nature as the multiplicand; indeed it expresses the addition of several numbers that are equal to the multiplicand. (…) (Reynaud 1821, p.10, my translation) The definition given by Reynaud may seem very complex. Actually, the formulation allows to gather in a same approach multiplication by whole numbers, and by fractions. With whole numbers, it goes back to repeated addition of the multiplicand. [START_REF] Bezout | Cours de mathématiques à l'usage des gardes du pavillon et de la marine, Eléments d'arithmétique[END_REF] gives a shorter definition, though in the same vein: "Multiplying a number by another; this means: taking the first one as many times as there are units in the other one. Multiplying 4 by 3; this means: taking three times the number 4" (p.28, my translation). The reference knowledge is thus related to the external multiplication of quantities by a number. For instance, this implies that multiplication is not commutative. Within the textbook's page, a major consequence of the definition is the distinction between multiplicand and multiplier. It has several implications: e.g. 1) 60 g. x 72 is read 72 times 60 g or 60 g multiplied by 72; 2) multiplier is written on the right, multiplicand on the left; 3) two cases are needed for elaborating the rule: multiplying tens and being multiplied par tens. Some textbooks elaborate the possibility of changing the order of abstract numbers within a multiplication before, so they refer to it and develop only one case; 4) actually this definition constraints but also allows to refer to quantities for elaborating the rule. Last, another point is the use properties of numeration, especially linked with units [START_REF] Chambris | Mathematical foundations for place value throughout one century of teaching in France[END_REF]. This hides the use of associativity.

The New Math reform curriculum (1970)

The introduction of the reformed syllabus asserts the curriculum is not a new one but a "different writing" of the previous one from 1945, a huge ecological reorganization I add. The main visible change is the creation of a new domain: "measurement: practical exercises". Previously, there were only two domains: arithmetic and geometry. Now, they are three: numeric, geometry, measurement.

An acute look at the text shows that: 1) The new measurement domain gathers the study of continuous quantities which was previously in the arithmetical part -measurement and computation with length, capacity, mass, time and corresponding metric units-, and that of the geometrical quantities which was previously in the geometrical part -computation of area and volume, and the corresponding metric units-. 2) The "numeric" domain is devoted to measurement (i.e. counting) and computation of discrete quantities, with many quotations of the set theory.

3) The only exception to this is that of scales problems which moved from geometry to "numeric", and joined there the old "rule of three" problems under the new "proportionality" label. Last, computation involving units' names are forbidden: "Sentences like "8 apples + 7 apples = 15 apples" do not belong to mathematical language neither to ordinary language" (Instructions for syllabus 1970, my translation).

Decimals and fractions are studied in both domains: numeric and measurement with the same techniques and technologies. It is noteworthy that they do not involve fractioning of quantities. I interpret this as the transposition of the academic construction of the real numbers with whole numbers. The "numeric" domain contains many tables of numbers with operators. They support the new way to study ratios: with linear properties and coefficient, that is a trace of the transposition of the theory of linear application. These are traces of new reference knowledge. All computations are computations between numbers; and proportionality is no longer a relation between two quantities: it became a linear numerical relation.

To sum up: The reorganization of the New Math syllabus is the sign of the implementation of new reference knowledge for numbers and ratios, of academic knowledge. Paradoxically, the birth of the measurement domain is the visible side of the disappearance of quantities as the roots of arithmetic.

From the 1980s

From 1970, the measurement domain has remained with several changes in each new syllabus, but more and more continuous quantities get into the "numeric" domain (fractioning of quantities for fractions and decimals from 1980, number line for whole numbers and fractions from 1980). This can come from the influence of several academic works in mathematics education: e.g. [START_REF] Perrin-Glorian | Aires de surfaces planes et nombres décimaux[END_REF]). Yet, despite the assertion that fractions are measurement of quantities, there are no fractions of quantities -such as: 3/4 of a length or of 300 g-in the syllabus. Moreover, most of mathematical definitions of quantities (e.g. Euclid Book I, [START_REF] Rouche | Le sens de la mesure[END_REF][START_REF] Griesel | Reform of the construction of the number system with reference to Gottlob Frege[END_REF] involve a basic relation between order and addition: a<b iff it exists c, a+c=b; yet, addition is never linked with order relation in the curriculum. Related to proportionality, in 2002, the syllabus indicates examples of reasoning (technologies) with quantities in the form of computation with units:

One needs 400 g of fruit with 80 g of sugar to prepare a fruit salad. How much sugar is required with 1000 g of fruit? Reasoning can be of the following types:

-For 800 g of fruit (twice more than 400), 160 g of sugar (twice more than 80) are required […]. For 1000 g (800g + 200 g) of fruit, 200 g (160 g + 40 g) of sugar are required; -The mass of sugar required is five times less than the mass of fruit; 200 g of sugar (1 000 : 5 = 200) are required. (Instruction for syllabus 2002, 3rd-to-5th grade, my translation)

Looking closely at these rationales, a phenomenon becomes visible: operations with units are used in discursive or in uncomplete arithmetical ways (the only equal sign is used between numbers). Operations with units seem not to belong to mathematics, but seem to fulfil pedagogical needs. My interpretation is that quantities come back for didactical needs, but the underlying mathematical reference knowledge has not changed from the reform. This brings to the fore the need for new reference knowledge: an adequate theorization of quantities for the teaching of numbers and ratios.

Figure 4 displays an excerpt of a present 3 rd -grade textbook: the topic is roughly the same as in 1932. According to the teacher's guide, through the "discovery" activities, the students attempt first to compute 8 x 40 then 60 x 30 by their own means, then the focus -within the bubble-is on the implicit use of the associativity, mixed with commutativity of multiplication. It is supposed to be drawn on students' attempts -and, if need be, on a "solved problem": Leila's method-. There is no unit, no quantities. I specify that in the previous lesson -multiplying by ten-, no link had been established between multiplying tens and converting ones into tens (numeration). Last, current curricula do not propose a specific way for writing the numbers within multiplication (which number on the right?). Some examples of commutativity for multiplication are often given at the very beginning of the curricula. Then 3 x 5 or 5 x 3 are used indistinctly. There seems to be very little further investigation about this property.

Multiplying by multiples of 10, of 100, of 1000 Goals: understanding the rule of multiplication by multiples of 10, of 100, of 1000. A "learning" theory rules the design of the activity: building on students' knowledge. The mathematical reference knowledge seems to be that of operations within the set of whole numbers -N-. Yet, within the ancient texts, quantities provide rationales for the multiplicative properties. It is not the case within the present excerpt. The rationale seems rather to be a matter of writing. Does this involve learning difficulties? Whatever, depending on the topic and on teachers' choices, quantities can be solicited or not: e.g. some textbooks refer to "word problems" for this given topic.

Discovery

This fosters my previous interpretation: quantities have come back for didactical needs, especially when continuous quantities or ratio are involved. This come back is less clear for whole numbers.

DISCUSSION

To begin with, the grain of the current study is rough and further investigation is needed to better characterize even the written curriculum. However, it is clear that the measurement domain cut many old trophic chains in the 1970s. Some new trophic chains were planned -e.g. the reorganization of proportionality-but it is not sure they "compensate" the broken links with quantities. Indeed, more and more continuous quantities have been added in the "numeric" domain after the reform, as if it were a pedagogical need. There is still a measurement domain which role is poorly defined, especially its contribution to the understanding of operations on quantities -e.g. addition-, involved for instance in the learning of fractions.

The New Math is an international phenomenon (Kilpatrick 2012) that impacted several western countries in the 1960s and 1970s, among them the US and Germany. Ma ( 2013) is very critic on the current written math curricula of the US, and connects its "disorganization" to the New Math and to the disappearance of a classical theory of arithmetic. Singaporean and Chinese curricula are built on such theory. [START_REF] Griesel | Reform of the construction of the number system with reference to Gottlob Frege[END_REF] identifies different "New reference knowledge" for fractions in West and East Germany during the New Math: with quantities, with sets or whole numbers.

CONCLUSION

This work helps to better understand some effects, even long term effects of the New Math in France. Quantities disappeared in reference knowledge for numbers and ratios surrounding the New Math. Yet, they seem to have partly come back -more and more in recent years-for pedagogical needs. It shows the lack of adequate reference scholar knowledge for numbers and ratios, connected to quantities despite relevant scholar works (e.g. [START_REF] Griesel | Reform of the construction of the number system with reference to Gottlob Frege[END_REF][START_REF] Rouche | Le sens de la mesure[END_REF]. Such theory notably enables to formulate basic arithmetical rules in basic terms. The increasing role of learning theory based on students' activity that has been promoted from the New Math may hide -or compensate-this lack. Present study leads to ask whether the results could be similar to other countries that were close (in term of curriculum) or not to France. Before the New Math, whole numbers, place value; and operations were taught with both discrete and continuous quantities. The creation of the "measurement" domain is the visible side of the transposition of the set theory and the elimination of continuous quantities in the reference knowledge for numbers, operations and ratios. Yet, continuous quantities seem to be a key input for conceptualization of numbers and ratios [START_REF] Barrett | Children's unit concepts in measurement: a teaching experiment spanning grades 2 through 5[END_REF]; and the epistemology [START_REF] Artigue | Epistémologie et didactique[END_REF] of numbers also fosters the desirable approaches on continuous quantities for numbers and ratios. Another question is: to which extent the roles of quantities in basic arithmetic impact teachers' perceptions of Mathematics, especially the idea of rigor, and that of the relation between math and everyday life.

Fig. 1 :

 1 Fig.1: The process of didactic transposition (op. cite)

  3. Addition of numbers. Be they pears or meters, three and two are five. Counting 3 and 2 are 5: this is doing an addition. Instead of 3 and 2 are 5, one writes: 3 + 2 = 5, what one reads: 3 plus 2 equals 5.

Fig. 2 :

 2 Fig. 2: (Marijon et al., grade 2, lesson 4, 1947, p.14), my translation.

Fig. 3 :

 3 (Châtelet et al. 1932, p.160), my translation.

Fig. 4 :

 4 Fig. 4: (Peltier et al. 2010 p.89), 3 rd grade, middle of the year. My translation