F Démé

E Peuvrel-Disdier

B Vergnes

Rheology and morphology of polyester/thermoplastic flour blends

Keywords: Flour, Polyester, Glycerol, Blend, Rheology, Morphology

Two maize flours (standard and waxy grades) were plasticized in an internal mixer with a constant amount of water and two glycerol contents. Resulting thermoplastic flours (TPF) were characterized in dynamic oscillatory shear and creep/recovery rheometry. They displayed two different behaviors: the viscoelastic behavior of a high molecular weight polymer for the first one and a gel-like behavior for the second one.

TPF were then mixed with a co-polyester (poly(butylene adipate-terephtalate), PBAT). All blends contained the same volume fractions and were prepared using the same mixing conditions. Morphology and rheological behavior of each blend were characterized. Different morphologies, ranging from cocontinuous to nodular, were observed. In fixed mixing conditions, blend morphology was shown to be governed by the rheological behavior of the starchy phase and the plasticizer content. The gel-like behavior of the second TPF seems to prevent droplets coalescence, leading to a very fine dispersion. The rheological behavior of each blend appears to be linked to both morphology and rheological behavior of the two phases.

Introduction

The development of polymer/thermoplastic starch blends has recently widely increased, mainly for packaging applications [1][2][3][4]. A part of these studies is focused on synthetic polymer matrices such as polyethylene (PE) [5][6][7][8][9][10] or polyamide (PA) [11,12]. However, the main part is devoted to bioplastics, like poly(lactic acid) (PLA) [13][14][15][16][17], polycaprolactone (PCL) [18][19][20][21][22], polybutylene adipate terephthalate (PBAT) [23][24][25][26], or other aliphatic polyesters [27][28][29][30]. In some cases, ternary systems were also studied [16,25]. Applications in areas such as films and bags require a fine dispersion of the starchy phase in the polymer matrix with sizes much smaller than the initial size of starch granules (5 to 20 microns). Indeed, the usual thickness of plastic films is around ten micrometers. To reach this level of dispersion, it is necessary to destroy the native starch structure, either by fragmentation or by melting and dispersion. The second method is the most used. In this case, thermomechanical processing and plasticization of the starch phase is mandatory before mixing it with the polymer matrix.

The morphology of a polymer blend depends on the rheological properties of components (viscosity ratio and elasticity ratio), volume fractions and interfacial tension [START_REF] Utracki | Polymer Blends Handbook[END_REF][START_REF] Isayev | Encyclopedia of Polymer Blends[END_REF]. To reduce the interfacial tension, it is usual to add compatibilizers [7,11,12,14,22,25].

Starch is composed of two polymer chains (amylose and amylopectin), both based on D-glucose units, and minor components such as lipids and traces of minerals [START_REF] Tester | Starch in biopolymers[END_REF]. Amylose is a linear chain whereas amylopectin has a highly branched structure. The rheological behavior of thermoplastic starches is more complex than the one of a classical polymer [START_REF] Valle | [END_REF][START_REF] Valle | [END_REF][START_REF] Valle | [END_REF][37][38]. It depends on the starch botanical origin (more precisely the ratio amylose/amylopectin), eventual chemical modifications like esterification, amount and type of plasticizers, and thermomechanical treatment undergone by the starchy phase during processing.

Starch thermal transitions and phase transformations are also more complex than those of synthetic polymers. There are two main types of crystallinity for native starch. Three others, formed by amylose/lipid or amylose/glycerol complexes, appear during cooling, after the destruction of the native crystallinity during the process [START_REF] Van Soest | Crystallinity in starch bioplastics[END_REF]40] and remain stable at high temperature. Therefore, to avoid parasitic crystallization problems, it is usual to plasticize starch in a first processing step and then to mix it with the polymer phase, without intermediate cooling.

Flour contains proteins in addition to starch. Its use instead of starch for preparing bioplastic blends is more seldom. Therefore, the aim of the present work is to characterize the behavior of two different thermoplastic flours (TPF) during their mixing with a PBAT matrix. TPF and blends were prepared in an internal batch mixer. The effect of plasticizer amount was also studied through the comparison of two formulations. For a better understanding of the morphology and the rheological behavior of the blends, a special attention was paid to the rheological characterization of TPF.

Materials and methods

Materials

Two maize flour types were kindly provided by the company ULICE (Riom, France): a waxy maize flour (called flour A in the following) and a standard maize flour (called flour B). They differ essentially by their amylose/amylopectin ratio. The standard grade (flour B) contains 70% of amylose and 30% of amylopectin whereas the waxy grade (flour A) only contains amylopectin. Beside starch, flours also contain proteins, lipids and some minerals in small quantities (less than 1%).

Glycerol was chosen as a plasticizer. Two contents were selected: 15 wt% and 30 wt%. These contents were based on the total weight of the TPF formula: glycerol + water + flour. Indeed, to facilitate processing, water was also added to reach, for all formulations, a total water content of 20 wt% (including water initially present in the flour).

The polybutylene adipate terephthalate (PBAT) (kindly provided by ULICE) is a semi-crystalline random copolymer, with a crystallinity around 10%. Comonomer units are butylene adipate (BA) and butylene terephthalate (BT).

Processing

TPF and blends were processed using an internal mixer (Haake Rheocord 600P, ThermoFischer, Karlsruhe, Germany). Flour, water and glycerol were initially mixed by hand and let at rest under controlled humidity conditions for 24 hours prior introduction in the internal mixer. Mixing parameters were kept constant for all experiments: chamber temperature = 125°C, filling ratio = 70%, rotation speed = 175 rpm and mixing time = 8 min. After this treatment, part of the flour was removed for further analyses and the remaining part was blended with PBAT, in the same mixing conditions. The weight ratio PBAT/TPF was kept constant, PBAT forming the major phase. The exact PBAT/TPF ratio could not be indicated for confidentiality reasons, but is in the range 60-80 wt% for the PBAT phase and 40-20 wt% for the TPF.

For sake of simplicity, TPF with flour A and 30wt% glycerol content will be denoted TPF A30 and the corresponding PBAT/TPF blend will be called A30 blend. Similar notations will be used for the other products.

Characterization

TPF samples were quenched in liquid nitrogen and cryo-fractures were observed by scanning electron microscopy (SEM) using a Philips XL30 ESEM, with an accelerating voltage of 15 keV in backscattered-electron mode. These pictures, made under a low vacuum (0.1 to 1 mPa), were intended to characterize the starch plasticization state.

Cryo-fractured TPF/PBAT blend surfaces were treated to dissolve either the starchy or the PBAT phase: attack in 1N sulfuric acid for 3 h under magnetic stirring to dissolve the starchy phase, or dissolution of the PBAT phase in a 50/50 acetonitrile/chloroform mixture for 3h at 50°C followed by centrifugation to collect the starchy phase. Samples were then metallized by vacuum plasma deposition of Au-Pd alloys before SEM observations. Dynamic rheological behavior of PBAT, TPF and blends was characterized on a strain controlled rheometer (ARES, TA Instruments). Measurements were carried out with parallel plates of 25 mm diameter and a 1.5 mm gap. Samples were obtained by compression molding for 10 min at 20 MPa and 150°C, followed by cooling under pressure. Sample edges were coated with a thin film of silicone oil to limit water losses during the measurements. Striated plates were used to prevent wall slippage.

Creep/recovery measurements with a stress controlled rheometer (Stresstech, Reologica) were also performed with the same protocol.

Experimental results

TPF characterization

The specific mechanical energy (SME) provided during the TPF plasticization process was calculated from the time integration of the mixer torque. SME values and final product temperatures are reported in Table 1 for the different TPF (same mixing conditions). SME values are different for the two flours and mainly depend on plasticizer amount. SME value for TPF with the lowest glycerol content (15 wt%) is indeed three times larger than for the one with the highest content (30 wt%). This is valid for both flour types. SME values for the standard flour grade TPF B are in all cases higher than for the waxy flour grade TPF A, indicating a more viscous material. This higher viscosity also leads to a higher overheating: compared to the chamber temperature (125°C), product temperatures up to 152°C were reached in TPF B15 after 8 minutes compared to 130-132°C for all other samples. SEM pictures (not shown here) indicate that the presence of residual granules (or fragments) is fairly low for both flours, even though TPF B shows a little more residual particles.

Before characterizing TPF rheological behavior, their thermal stability was evaluated using time sweep tests at 1 rad/s and 150°C in the linear viscoelastic domain (1% strain). We can see in Fig. 1.a that TPF storage modulus G' increased with time, even though TPF B was more stable than TPF A. As expected from torque measurements in the mixer, we can also remark that TPF B (standard grade) is more viscous than TPF A (waxy grade). Time evolution can be due to a loss of plasticizer during the measurements. To check this point, mass losses were measured after 48 hours at 130 °C, on samples taken just after mixing and then after one hour at 150°C in the rheometer. We can see in Table 2 that mass losses vary from 8 to 23 % after mixing and from 3 to 18 % after rheological measurements. It thus confirms that, despite the silicon oil, part of the volatiles is evaporated during these measurements. Complementary measurements of glycerol content (performed at ENSCCF, Clermont-Ferrand, France) showed that mass losses are mainly due to water for temperatures below 200°C. Consequently, we consider that the measured losses correspond to the amount of water present in the samples at the beginning and at the end of the thermal stability tests. It means that around 3 to 4 wt% of water was evaporated for TPF B in comparison with 12 to 15 wt% for TPF A. The greater stability of the TPF B15 is thus probably due to its low water content after processing (Table 2). Even though an equivalent water amount was initially introduced in flours A and B, TPF B15 was subjected to a much larger overheating than TPF A15 (see Table 1), leading to higher water loss. In the case of 30 wt% glycerol content, the overheating was low and similar for both flours, resulting in similar water losses. However, TPF A30 lost much more water during the rheological test, indicating that water appears to be less bonded to the standard flour grade A than to the waxy flour grade B at the same temperature.

The linear viscoelastic domain was determined for the different samples (Fig. 1.b). It is different for each TPF. The linear domain is much smaller for TPF B than for TPF A. Moreover, whatever the flour, it decreases when increasing the glycerol content. The critical strain is 1.6, 4.1, 22.2 and more than 100%, for TPF B30, TPF B15, TPF A30, and TPF A15, respectively. This difference can be explained by the difference in rheological behavior between the two TPF. Fig. 2 presents the viscoelastic data (G', G" and complex viscosity η* versus frequency) for TPF A30 and TPF B30 at 150°C (similar results were obtained for TPF with 15 wt% glycerol but with higher values). TPF A30 shows a quasi-superposition of both modules over several decades, which is characteristic of a polymer melt of very high molecular weight with a branched structure [41,42]. This behavior is consistent with the composition of the waxy flour (100% amylopectin). The complex viscosity obeys a power law with an index of 0.46. Conversely, TPF B30 exhibits a gel-like behavior, with G' higher than G" over the whole range of frequencies, both modules being quite independent of frequency.

The complex viscosity also obeys a power law, but with a very low index, close to 0.06, confirming the plastic-like behavior of TPF B30. Such rheological behaviors have already been reported for TPS with similar amylose/amylopectin ratios [START_REF] Valle | [END_REF][START_REF] Valle | [END_REF].

To complete the rheological characterization, creep experiments were carried out at 150°C (at 100 Pa during 480 s), followed by recovery (Fig. 3.a). TPF A30 shows a certain deformation during creep, proportional to time like for a viscoelastic fluid. However, the recovery seems non-existent. Conversely, TPF B30 is much more difficult to deform: under the same stress, the strain after 480 s is only 0.017 compared to 265 for TPF A30. However, by imposing a stress ramp between 0.1 and 50 Pa, we can see in Fig. 3.b that TPF B30 can effectively be deformed, but to a smaller extent and only above a yield stress of around 1 Pa.

The complex viscosities of the different formulations are compared on Fig. 4. As already seen, flour B leads to more viscous TPF than flour A and, for both materials, the viscosity decreases with the increase of glycerol content. All curves can be fitted by power laws, with indices of 0.41, 0.46, 0.16 and 0.06 for TPF A15, TPF A30, TPF B15 and TPF B30, respectively. The low power law index values for TPF B are coherent with the existence of a yield stress, as observed previously. The viscosity of the PBAT which will be used in the next section to prepare blends is also indicated on the same figure for sake of comparison. PBAT viscosity is lower than the ones of both TPF with 15wt% glycerol. In the case of 30wt% glycerol, depending on the frequency, the viscosity of PBAT can be higher than the ones of both TPF, or in between TPF B30 and TPF A30. We therefore expect to obtain different morphologies for the blends, depending on the glycerol content.

TPF/PBAT blends characterization

As for TPF, the specific mechanical energy (SME) provided to the blends during mixing and the final product temperatures were recorded. Results are presented in Table 3. PBAT having a melting temperature extending from 110 to 130°C, the starch phase undergoes significant shear stresses at the beginning of mixing and overheating is important: for a chamber temperature of 125°C, the final temperature of the blend can reach 142°C to 173°C. This temperature increase is slightly higher for blends with TPF B, and more limited at high glycerol content. SME is generally high, between 170 and 563 kWh/t. Values are more important for blends based on TPF B and on TPF with low glycerol content.

TPF/PBAT blends morphology

PBAT/TPF blend morphologies were analyzed by scanning electron microscopy. Pictures for A15, A30, B15 and B30 blends are presented in Figs. 5 and6. A great diversity of both morphology and starchy phase size is observed. Blends realized with TPF A show a relatively coarse morphology, with a nodular morphology for A15 blend (spheres of several tens of microns, Fig. 5.a and 5b) and a co-continuous one for A30 blend (Fig. 5.c and 5.d). The co-continuity of A30 blend is more apparent in Fig. 5c, after dissolution of the starchy phase. Indeed, the starchy phase after PBAT extraction does not appear as continuous, because it is very brittle. Nevertheless, the very large size of the starchy domains tends to confirm this assumption. Contrary to what could be expected from the rheological behavior, blends made with TPF B show a much finer morphology, with a nodular starchy phase (Fig. 6). The nodules have a nice spherical shape. The average values of nodule dimensions measured by image analysis are summarized in Table 4.

TPF/PBAT blends rheological behavior

The thermal stability of the different TPF/PBAT blends was evaluated using time sweep tests at 1 rad/s and 150°C in the linear domain (1% strain) (Fig. 7.a). The presence of the PBAT matrix does improve the thermal stability of the blend comparatively to TPF alone (Fig. 1.a). However, it does not fully prevent water evaporation during test. Blends based on TPF B are more stable as already noticed for the TPF.

The linear viscoelastic domain was determined for all the blends (Fig. 7.b). It is almost the same for the different blends: the critical strain is around 10% and does not depend on TPF type or glycerol content.

The linear viscoelastic behavior (G', G" and complex viscosity η* versus frequency) of the different blends is shown in Fig. 8. Blend B15 presents a classical viscoelastic behavior, with G" larger than G' on the investigated range, a Newtonian plateau at low frequency and a shear thinning behavior at higher frequency. Blends A15 and A30 behave similarly with G" larger than G' over the entire frequency range and a more pronounced shear thinning behavior. A15 blend tends to show at low frequency an onset of G' plateau, associated to a low yield stress. B30 blend depicts a different behavior with G' quite superimposed with G" over the investigated frequency range. The interpretation of these results, according to the behavior of the components and the blend morphology, is discussed in the next paragraph.

Discussion

TPF/PBAT blends morphology

For a nodular morphology, the particle size is usually defined by the critical capillary number, which is a function of the viscosity ratio between the dispersed phase and the matrix η d /η m [43][44][45]. Different expressions can be found in the literature, for simple (shear, elongation) or complex (extrusion) flow situations. For example, Wu [43] proposed the following empirical equation in case of blends prepared by extrusion:

0.84 2 d m m D η σ η γ η ±   =      ( 1
)
where D is the droplet diameter, σ the interfacial tension and γ the shear rate. The exponent is positive for η d /η m > 1 and negative for η d /η m < 1. The ratio η d /η m must be calculated at the shear rate and product melt temperature corresponding to the processing conditions. This empirical model was established for blends of immiscible thermoplastic polymers. It considers that the droplet size results from an equilibrium between the interfacial stress and the shear stress and takes into account the effect of the viscosity ratio. It does not consider any effect of components elasticity on the blend morphology. Therefore, our purpose in the following was just to check whether the simple consideration of the viscosity ratio and the shear stress was enough to explain the variation in droplet size of the TPF/PBAT blends. In the internal mixer, the shear rate can be estimated from the rotor speed, according to the methodology originally proposed by Bousmina et al. [46,47]. At 175 rpm, the average shear rate is around 100 s -1 . Accordingly, the viscosity ratio is equal to 0.2 and 0.4 for A30 and B30 blends, and 1.5 and 3.0 for A15 and B15 blends. If we assume that the nodule size follows Eq. ( 1), the smallest size should be obtained for A15 blend, the largest one for A30 blend and similar values for B15 and B30 ones. We can see in Table 4 that it is not really the case. It means that the viscosity ratio alone cannot explain the size variations of the starchy phase. Beside possible differences in the interfacial tension, the difference in rheological behavior of TPF phase is also at the origin of this size difference. Indeed, unlike TPF A, TPF B shows a gel-like behavior with a very short linear domain. This TPF is more elastic and can be deformed only under high shear stress (Fig. 3.b). Similar studies with modified highly elastic starches, mixed with PE [6] or PCL [21],

indicated that the deformed starch particles form very stable fibrils due to their high elasticity. The gellike behavior of TPF B could also lead to a greater difficulty for particles to coalesce. Thus, this absence of coalescence could explain the small nodule size observed in these blends, deformation and rupture being the only possible mechanisms. In the case of TPF A, the coalescence of the droplets can be at the origin of the coarser observed morphologies . The evolution from a nodular morphology to a fibrillar one when increasing glycerol content has already been demonstrated in PE/starch blends by Rodriguez-Gonzales et al. [6] and was explained by the viscosity ratio. In the case of TPF A, the drop in viscosity ratio when increasing the glycerol content could explain the formation of a more elongated dispersed phase which is favorable for the formation of a co-continuous morphology as observed in Fig. 7 [48].

TPF/PBAT blends rheological behavior

The rheological behavior of each blend at 150°C is compared to the one of its respective components (PBAT and TPF) in Fig. 9 (complex viscosity η*) and Fig. 10 (storage modulus G').

Considering a blend of two immiscible polymers and applying a simple mixing rule without interfacial tension (for example, a model of Kerner [49]), one expects that the modulus or the viscosity should lie in between the component ones. However, the droplet interface (characterized by a specific time relaxation)

gives rise to an additional viscoelastic contribution, characterized by a shoulder on G' curve at intermediate frequency, which can be predicted by the model of Palierne [50]).

B15 blend storage modulus and viscosity curves tend to follow the tendency of a mixing rule: B15 blend curves follow the PBAT ones but shifted to higher values. B30 blend follows also the matrix behavior but with a pronounced departure at low frequency. A15 blend follows more closely the behavior of TPF than the one of PBAT, but with values inferior to the ones of both components. It is similar for A30 blend, which exhibits a co-continuous morphology.

An attempt was made to interpret the B15 blend data using classical blend models (Kerner and Palierne).

Droplet size was taken as 1 µm radius (as observed by SEM) and, for Palierne model, interfacial tension was varied in the usual range, i.e. between 1 and 15 mN/m. The experimental curves for the blend (Fig. interactions between particles, eventually favored compared to B15 by the smaller particle size. This also confirms that these blends behave more like suspensions with a "solid" TPF phase than like classical blends.

A15 and A30 blends are more close to the behavior of TPF. For these materials, TPF viscosity is closer or even lower than the PBAT one. The morphology is nodular with large nodules (A15 blend) or cocontinuous (A30 blend). Consequently, the influence of the TPF phase is more pronounced.

Anyway, it seems very difficult to propose clear explanations for these various results: the behavior of the components, together with the blend morphology, is unable to allow a correct interpretation. Moreover, it is worth pointing out that an evolution of the volume fraction of the TPS phase, via the evaporation of plasticizer during the mixing process, cannot be neglected.

Conclusion

We studied the behavior of two flours plasticized with two glycerol contents in an internal mixer and then blended with a PBAT matrix under identical conditions. The rheological study of thermoplastic flours (TPF) highlights two different behaviors. In the case of waxy grade, we observe the behavior of a high molecular weight viscoelastic polymer, and in the case of standard grade a gel-like behavior. In all cases, glycerol content only modifies the viscosity level. Water content, which decreases considerably during mixing, also influences TPF viscosity.

The study shows that TPF rheological behavior governs the blends morphology and their rheological behavior. The TPF rheological behavior is governed by the flour nature, i.e. the amylose/amylopectin ratio and the plasticizer content, in a similar way as for TPS. According to the amylose/amylopectin ratio and plasticizer content, different morphologies and sizes of the starchy phase could be obtained. The glycerol content modifies the viscosity ratio between TPF and PBAT matrix, and thus the ability of TPF particles (or droplet) to deform and break, or to coalesce to form a co-continuous phase. The development of a co-continuous structure appears to be due to the behavior, size and shape of particles rather than the viscosity ratio of the two phases. Indeed, a gel-like behavior of the dispersed phase seems to prevent the coalescence of droplets or fibrils, allowing for much finer structures. By selecting flour type, plasticizer amount and mixing conditions (not presented here), it is thus possible to obtain blends with a wide variety of morphologies and rheological properties, leading to final products with various possible applications.

Using TPF in bioplastic blends may constitute interesting alternatives to TPS with an economic interest.

Flour is less expensive than starch due to its simpler process of flour. Using TPF in bioplastic blends may constitute interesting alternatives to TPS with an economic interest. Flour is less expensive than starch due to its simpler process of flour.

Aknowledgements

This study was conducted as part of the French project CEREMAT, supported by the "Cereales Vallée" cluster. We gratefully thank FUI (Fond Unique Interministériel) for financial support, project members for giving permission to publish this work and ENSCCF (Clermont-Ferrand, France) for measuring glycerol content on TPF and blends.

  11.a) are globally fitted by the two models. However, calculations do not show any difference between the simple mixing rule and the Palierne model, whatever the range of droplet size and interfacial tension. Due to the high viscosity and elasticity of TPF B15 relatively to PBAT, inclusions do not seem to be deformed during the solicitation. For this reason, B15 blend can be considered as a suspension. A suspension model, such as a Krieger-Dougherty model for example, predicts that the suspension Newtonian viscosity is equal to the matrix one multiplied by a constant factor depending on volume fraction and close packing fraction. The viscosity curve of the suspension thus should be parallel to the matrix one in a log-log representation. This effectively corresponds to the behavior of B15 blend as shown in Fig 9.b. In Fig. 11.b, we can see that this behavior can be nicely described by the simple Einstein equation. B30 blend (Fig. 9.d and 10.d) shows a similar rheological response, except at low frequency with the presence of an onset of G' plateau, indicating the existence of a melt yield stress. This indicates

Figure captions Figure 1

 captions1 Figure captions

Figure 2 .

 2 Figure 2. Viscoelastic modules (G', G") and complex viscosity (η*) of TPF at 30wt% glycerol content as function of frequency at 150°C 1% strain . a) TPF A30, b) TPF B30.

Figure 3 .

 3 Figure 3. a) Creep-recovery tests and b) creep in stress steps for TPF A30 and TPF B30 at 150°C.

Figure 4 .

 4 Figure 4. Complex viscosity η* as function of frequency at 150°C, 1% strain : ○ : TPF A15, □ : TPF B15, • : TPF A30, ■ : TPF B30, ∆ : PBAT.

Figure 5 .

 5 Figure 5. SEM micrographs of A15 blend after extraction of a) the starchy phase, b) the PBAT and of A30 blend after extraction of c) the starchy phase, d) the PBAT.

Figure 6 .

 6 Figure 6. SEM micrographs of B15 blend after extraction of a) the starchy phase, b) the PBAT and B30 blend after extraction of c) the starchy phase, d) the PBAT.

Figure 7 .

 7 Figure 7. a) Thermal stability (storage modulus G' as function of time, 1 rad/s, 1% strain) and b) linear domain (storage modulus G' as function of strain, 1 rad/s) for the TPF/PBAT blends at 150°C.

Figure 8 :

 8 Figure 8: Rheological response of the TPF / PBAT blends (storage and loss modules G' and G", complex viscosity η* versus frequency) : a) A15 blend, b) B15 blend, c) A30 blend, d) B30 blend.

Figure 9 .

 9 Figure 9. Complex viscosity η* as function of frequency at 150°C, 1% strain for TPF (○), PBAT (•) and TPF/PBAT blends (□): a) A15, b) B15, c) A30, d) B30.

Figure 10 .

 10 Figure 10. Storage modulus G' as function of frequency at 150°C, 1% strain for TPF (○), PBAT (•) and TPF/PBAT blends (□): a) A15, b) B15, c) A30, d) B30.

Figure 11 .Figure 1 .

 111 Figure 11. Comparison between model predictions and experimental measurements for B15 blend: a) storage modulus, b) viscosity. Symbols are experimental values. Lines are obtained with Palierne (), Kerner (----) and Einstein (…..) models

22

 22

Figure 2 .

 2 Figure 2. Démé et al.

Figure 3 .

 3 Figure 3. Démé et al.

Figure 4 .

 4 Figure 4. Démé et al.

Figure 5 .

 5 Figure 5. Démé et al.

Figure 6 .

 6 Figure 6. Démé et al.

Figure 7 .

 7 Figure 7. Démé et al.

Figure 8 .Figure 9 .

 89 Figure 8. Démé et al.

Figure 10 .

 10 Figure 10. Démé et al.

Figure 11 .

 11 Figure 11. Démé et al.

Table 1 .

 1 Specific mechanical energy (SME) during TPF processing and final product temperatures.

		15 wt%	30 wt%
		glycerol	glycerol
	Flour A	111 kWh/t	38 kWh/t
		132°C	130°C
	Flour B	228 kWh/t	67 kWh/t
		152°C	132°C

Table 3 .

 3 Specific mechanical energy (SME) during TPF/PBAT mixing and final product temperatures.

			Length	Width
	Blend	Morphology	(µm)	(µm)
	A15	Nodular	21	12
	A30	Co-continuous	(39)	(17)
	B15	Nodular	3.4	1.8
	B30	Nodular	1.5	0.8

Table 4 .

 4 Blend morphology and medium size (length and width) of TPF nodules.


Serge Mazauric
email: serge.mazauric@univ-st-etienne.fr

Thierry Fournel

Mathieu Hébert

Fast-calibration reflectance-transmittance model to compute multiview recto-verso prints

Keywords: Computational printing, flux transfer models, duplex halftone print, multiview images

Predicting simultaneously the spectral reflectance and transmittance of recto-verso prints is made easier thanks to a flux transfer matrix model. In the case where the printing support is symmetrical, i.e., its two sides are similar, the model can be calibrated from 44 halftone colors printed on the recto side, whose spectral reflectances and transmittance are measured. Color predictions are then allowed for any recto-verso halftone print illuminated on either side, and the inverse approach can be addressed: we can compute the digital layout for the recto and verso side that, once printed, can display different images according to the illuminated side.

Introduction

Spectral prediction models are key tools for fast and accurate color management in printing. They are also indispensable for designing advanced printing features such as those where the print displays different images according to the viewing conditions. Such multi-view effects can be obtained by using metallic inks [START_REF] Babaei | Yule-Nielsen based multi-angle reflectance prediction of metallic halftones[END_REF] or a specular support [START_REF] Pjanic | Specular color imaging on a metallic substrate[END_REF] and observing the print in or out of the specular reflection direction. They can also be obtained by double-side printing with classical inks and supports, by observing one face in reflection mode or through both faces in transmission mode [START_REF] Hébert | Spectral mixing approach for modeling the coloration of surfaces[END_REF]. For these printing configurations, color management based on digital methods like ICC profile is almost impossible because the number of needed sample measurements, often beyond one thousand for single-mode observation, exponentially increases with the number of observation modes. The calibration time of the printing system thus becomes inacceptable, whereas a few tens of measurement often suffice to calibrate a prediction model, thus able to predict all reproducible colors in the different modes.

Among the most accurate models for the spectral reflectance of halftone prints, the physically-based models such as the Clapper-Yule model have high potential for extension to multilayer surfaces as they explicitly describe flux transfers between the layers. Some examples have been investigated in previous studies though stacks of printed films [START_REF] Machizaud | Spectral transmittance model for stacks of transparencies printed with halftone colors[END_REF] and double-side printed papers [START_REF] Hébert | Reflectance and transmittance model for recto-verso halftone prints: spectral predictions with multi-ink halftones[END_REF][START_REF] Hébert | Yule-Nielsen based recto-verso color halftone transmittance prediction model[END_REF], configurations for which good prediction accuracy was achieve by means of acceptable number of calibration samples.

The main issues with these models where both reflectance and transmittance of prints are combined, it that they need a calibration for the reflectance mode, and a separate calibration for the transmittance mode: one parameter may have different values in the two modes. This issue was recently solved thanks to the so-called Duplex Primary Reflectance-Transmittance (DPRT) model, a two-flux transfer matrix model predicting simultaneously the reflectance and transmittance of duplex halftone prints on both faces, whose parameters related to the primaries are deduced from both reflectance and transmittance measurements and therefore optimal for the two modes [START_REF] Mazauric | Two-flux transfer matrix model for predicting the reflectance and transmittance of duplex halftone prints[END_REF]. In the general case where the two faces of the support are different or printed with different inks, the DPRT model needs 328 spectral measurements. In the present paper, we show that in the special case where the support is symmetric, i.e., its two faces are identical and printed with the same ink set, the number of spectral measurements for calibration is reduced to 104, while keeping the optimal predictive performances in reflectance and transmittance modes.

The corresponding model, called Double-Layer Reflectance-Transmittance model, relies on the idea of separating the printed support into two sublayers, in which spectral flux transfers are described. We propose to present it and to show a few example of multiview images that were designed by using the model in inversed mode.

Two-flux transfer matrices

A flux transfer model is particularly adapted to the layered structure of printed supports. When the printing support is strongly scattering, like paper or white polymer, we can use a two-flux model. The model presented in Ref. [START_REF] Mazauric | Two-flux transfer matrix model for predicting the reflectance and transmittance of duplex halftone prints[END_REF] applies with a stack of layers of strongly materials and the interfaces between them. They are characterized by four transfer factors: the front-side reflectance r, the back-side reflectance r', the forward transmittance t and the backward transmittance t'. The component is said to by symmetrical when r = r', and t = t'. The angular distribution of the incident and transferred lights is to be considered in the definition of every transfer factor, which can be accordingly bi-hemispherical, directional-hemispherical, bi-directional and so on [START_REF] Nicodemus | Geometrical considerations and nomenclature for reflectance[END_REF]. All fluxes and transfer factors may also depend upon wavelength.

1 r 2 t 1 t 2 r 1 ′ r 2 ′ t 1 ′ t 2 ′ Component 1
Front side Back side

Component 2 I 0 I 2 I 1 J 0 J 2 J 1
When two components are on top of each other, interreflections of light occur between them, thus producing mutual exchanges between the fluxes propagating forwards (denoted as I k in Fig. 1) and backwards (denoted as J k ). These exchanges can be easily described by using flux transfer matrices. For each component k = 1 or 2, assuming 0 k t  , the relations between incoming and outgoing fluxes is described by the following matrix equation:

1 1 1 1 k k k k k k k k k k k I r I J r t t r r J t                        , (1)
where the matrix is the transfer matrix attached to the component k, denoted as

M k 1 1 k k k k k k k k r r t t r r t             M (2)
Grouping components 1 and 2 together, Eq. ( 1) can be repeated twice. One obtains:

0 1 2 2 1 1 2 0 1 2 2 I I I I J J J J                            M MM M (3)
where M, product of the transfer matrices of the individual components, is the transfer matrix representing the two layers together, similarly defined as Eq. ( 2

R m m T m R m m T m m m m         , (4)
3

Double-layer Reflectance-Transmittance (DLRT) model

We will use flux transfer matrices to describe the reflection and transmission of light by recto-verso halftone prints. Let us first consider the support alone, by assuming it symmetrical and strongly scattering. Its structure is shown in Fig. 2: an effective diffusing medium of refractive index 1.5 is bordered by interfaces with the surrounding air. Since it is symmetrical, it has similar transfer factors R(λ) and T(λ) on its two sides, and is therefore represented by a transfer matrix, defined for each wavelength as: The diffusing layer (considered without the interfaces) also has similar transfer factors ρ(λ) and τ(λ) on its two sides. It is represented by the following transfer matrix:

2 2 1 1 R T R T R            P , (5)
2 2 1 ρ 1 τ ρ τ ρ            M . ( 6
)
The reflectances and transmittances of the interfaces depend on the measuring geometry. In the present study, measurements were done with the X-rite Color i7 spectrophotometer relying on the di:8° geometry in reflection mode (illumination with perfectly diffuse light over the hemisphere, and observation at 8° from the normal of the sample by including the specular reflection component), and the d:0° geometry in transmission mode (illumination with perfectly diffuse light over the hemisphere, and observation in the normal of the sample). The corresponding transfer factors are given in Fig. 2. For these geometries and the refractive index value 1.5, the transfer matrices representing the interfaces at the recto and verso sides are given by:

                                   F F , (7)
In practice, the measurable quantities are the spectral reflectance R(λ) and transmittance T(λ) of the support with interfaces, therefore matrix P. The transfer matrix M for the diffusing layer without interfaces is then given by:

1 1       M F P F . (8)
Later, it will be convenient to decompose this diffusing layer into two similar sublayers (which justifies the appellation "double-layer reflectance-transmittance' model), thus represented by the transfer matrix:

  1/2 1/2 1 1 1        A M F P F . ( 9
)
r s ρ ρ t in τ τ r d t out r d r s ′ Diffusing layer Air Air Interface Interface ′ t in ′ t out ′ F M A 1 A 1 P F′ Front side Back side
We can extend the model to the Neugebauer primaries, obtained when superposing cyan, magenta and yellow halftone screens: white (surface with no ink, labelled 1 i  ) cyan, magenta, yellow, red (magenta + yellow), green (cyan + yellow), blue (cyan + magenta) and black (cyan + magenta + yellow). It is classical after Clapper and Yule [START_REF] Clapper | The Effect of Multiple Internal Reflections on the Densities of Halftone Prints on Paper[END_REF] to consider the inked support as a four-component structure: the two interfaces, the ink layer and the diffusing layer. We proposed in [START_REF] Mazauric | Two-flux transfer matrix model for predicting the reflectance and transmittance of duplex halftone prints[END_REF] a different approach where the inked support without interfaces is one non-symmetrical layer of effective medium. According to the double-layer approach, we can then split it into two sublayers: one on the front side containing the inks represented by a matrix A i , one with the matrix A 1 given by ( 9) for the non-inked back side. Thus, the transfer matrix representing the support covered by primary i on the recto side only is written:

1 1 i i      P F A A F
We thus have, for each primary i = 2, …,8:

   1/2 1 1 1 1 1 1 1 1 11 1 1 i i i i i i i i i i                                   A M A F P F F P F (10)
The transfer factors of the front sublayer, denoted i  , i   , i  , and i   , are obtained from [START_REF] Nicodemus | Geometrical considerations and nomenclature for reflectance[END_REF] by applying formulas (4). Since the support is symmetrical, and the verso side is printed with the same inks as the recto side, we can create similar matrix as i A for the verso side, by permuting i  and i   on the one hand, and i  and i   on the other hand, and finally replacing i with j:

1 1 j j j j j j j j                    A (11)
When a primary i is on the recto side and a primary j on the verso side, the matrix representing the whole inked support (interfaces excluded) is simply given by:

ij i j    M A A (12)
The model can be extended to halftone colors (Fig. 3). We denote as  

1, ,8 i a i   and   1, ,8 j a j    the surface coverages of the primaries on the recto side, respectively on the verso side. They are obtained from the surface coverages of the cyan, magenta and yellow inks using Demichel's equations, valable in case of typical stochastic or cluster dot halftoning techniques [START_REF] Hébert | Review of spectral reflectance prediction models for halftone prints: calibration, prediction and performance[END_REF]. We use the transfer factors i  , i   , i  , and i   issued from ( 10) and (4), and combine them according to the ink surface coverages i a and j a present on each side of the support. By using the generic letter  for either ρ, ρ', τ, or τ', we have, for the recto side:

8 { } 1 1 i a ii i a      (13)
and for the verso side:

8 { } 1 1 . i a ii i a        (14)
Fig. 3. Flux transfers within a homogeneous support printed with halftone on both sides: the inked support is split into two sublayers.

From these transfer factors given by ( 13) and ( 14), we create the transfer matrices representing the sublayers on the recto, respectively on the verso:

{a } { } {a } {a } {a } {a } {a } {a } { } { } { } { } { } { } { } { } 1 1 , 1 1 i i i i i i i i j j j j j j j j a a a a a a a a a                                                A A (15)
Finally, the duplex print with interfaces is represented by the transfer matrix:

{ } { } i j H a a        M F A A F ( 16
)
and its reflectances and transmittances are deduced from this matrix by using [START_REF] Machizaud | Spectral transmittance model for stacks of transparencies printed with halftone colors[END_REF]. Note that we can also predict, without further calibration effort, the spectral reflectances and transmittances of a doubled printing support. The matrix M H is simply computed as follows:

{ } 1 1 { } i j H a a             M F A A F F A A F (17)

Calibration and verification of the model

The DPRT model introduced in [START_REF] Mazauric | Two-flux transfer matrix model for predicting the reflectance and transmittance of duplex halftone prints[END_REF], valid for any non-symmetrical diffusing support, needed 8 8 64   CMY patches for its calibration, corresponding to the different combinations of the eight primaries on the front side and the eight primaries on the back side. The present version, valid for homogenous diffusing supports, needs only 8 rectoonly samples if the support is symmetrical and printed with the same set of inks on both sides. Since the four transfer factors of each sample must be measured, this makes an appreciable gain of time for the calibration of the DPRT model. The calibration of the model is made in two steps.

′ ′ { } i a { } i a { } i a { } i a A { } j a A { } i a ρ ρ τ τ′ ′ ′ ′ ′ ′ ′ { } j a { } j a { } j a { } j a i a j a′ r s t in r d t out r d r s ′ ′ ′ t in ′ t out ′

Back side (verso) Front side (recto)

The first step is the determination of the matrices A i , given by ( 10), for the eight Neugebauer primaries. For primary 1, the reflectance and transmittance of the unprinted support are measured on the recto side; for the other seven primaries, we print each primary with a coverage of 100% on the recto side, and measure the reflectances and transmittances of the printed support on its both sides. The reflectances and transmittances i

 , i   , i
 , and i   for the front-side sublayer printed with each primary i are computed, then the matrices A j representing the back-side sublayer printed with each primary j.

The second step is establishing the correspondence between the nominal ink surface coverages (those specified in the digital layout) and the effective ones, observed on the printed paper through the spectral reflectance and transmittance. We follow the very efficient method introduced by Hersch et al. [START_REF] Hersch | Improving the Yule-Nielsen modified spectral Neugebauer model by dot surface coverages depending on the ink superposition conditions[END_REF], also detailed in [START_REF] Hébert | Review of spectral reflectance prediction models for halftone prints: calibration, prediction and performance[END_REF], which yields 12 curves computed from 36 single-ink halftones printed on the recto side. For each halftone, the effective ink surface coverages is estimated by minimizing the difference between the measured and predicted spectra, according to a metric which may be the rms deviation between the spectral reflectances and/or transmittances, or the equivalent color distance (e.g. CIELAB ΔE 94 computed from the spectra for a given illuminant and a given white reference).

Finally, for any recto-verso halftone defined by its nominal ink surfaces coverages (CMY values) on the recto and verso sides, the primary surface coverages i a and i a are computed according to the Demichel equations, the transfer matrix M H of the rectoverso print is obtained by using equations ( 13) to [START_REF] Hébert | Review of spectral reflectance prediction models for halftone prints: calibration, prediction and performance[END_REF], and the reflectances and transmittances by using formulas (4).

The model was verified over tens of recto-verso samples with CMY values randomly selected over the color gamut on both sides of the support. They were printed with both inkjet and electrophotography (laserjet) printer, each time on both high-quality supercalendered paper and usual office paper. The number of patches in each set is specified in Table 1. The prediction accuracy, in reflectance or transmittance mode, is assessed by the average CIELAB ΔE 94 color distance between the predicted and measured spectra (see computation details in [START_REF] Hébert | Review of spectral reflectance prediction models for halftone prints: calibration, prediction and performance[END_REF]). The values presented in Table 1 show that the prediction accuracy is rather good, even though slightly poorer with the laserjet printer due to a low printing homogeneity, and with the office paper which is also less homogeneous than the supercalendered one. In most cases, the DLRT model is more accurate, and even much more accurate in transmittance mode, than previously proposed models: the Duplex Clapper-Yule model (DCY) [START_REF] Mazauric | Two-flux transfer matrix model for predicting the reflectance and transmittance of duplex halftone prints[END_REF], and the Yule-Nielsen model ( [START_REF] Hersch | Improving the Yule-Nielsen modified spectral Neugebauer model by dot surface coverages depending on the ink superposition conditions[END_REF], and [START_REF] Hébert | Yule-Nielsen based recto-verso color halftone transmittance prediction model[END_REF] for the extension to transmittance). Recall that the model is calibrated from only 44 recto-only color patches measured in reflectance and transmittance mode, and then that billions of recto-verso colors can be predicted.

Multiview images

Once the model is calibrated and its prediction accuracy has been verified, we can use it in an inversed mode to produce multiview prints, i.e., prints displaying different images according to the illumination and observation configurations. For example, we can illuminate the print on the observer's side (reflection mode) or the opposite side (transmission mode). We can also consider a transmission mode where ambient light slightly illuminates the observer's side (with similar SPDs on both sides), which corresponds to an addition of α times the transmittance and   1   times the reflectance, a configuration referred to as "α-transmission mode".

The ink quantities to print on the recto and verso sides are obtained by performing a brute-force optimization in order to obtain the targeted CIE 1976 L*a*b* values, or the targeted spectral radiance, in the two selected illumination modes. We generally compute first the ink surface coverages on the recto side yielding the reflection-mode image, then the ones on the verso side so as to obtain the targeted image in α-transmission mode. The selection of the targeted colors for the two modes is critical since color matching is not always possible, a question addressed in [START_REF] Dalloz | How to design a recto-verso print displaying different images in various everyday-life lighting conditions[END_REF]. Fig. 4 shows an example of multi-view image printed by laserjet printer on doubled office paper. It displays (flipped) "10" patterns on the verso side is reflectance mode, and only part of these patterns remain visible in 0.9-transmittance mode, because the color of the other part matches the color of the background of the image. A second print is shown in Fig. 5, where randomly distributed squares of eight different colors are printed on the recto side, eight colors are correspondingly printed on the verso side in order to display, in 0.9-transmission mode, two patterns of homogenous color and a background with more contrasted texture. Registration of the recto and verso images, and perfect parallelism of the two paper sheet is crucial for a good rendering of the targeted visual effect.

Conclusions

The DLRT model is a two-flux transfer matrix model that allows the prediction of the reflectance and transmittance of duplex halftone prints with a better accuracy and no more halftone patches than previously proposed models such as the Duplex Clapper-Yule model and the Yule-Nielsen model extended to transmission. The model relies on the separation of the printing support into two sublayers, each one containing the inks of the recto halftone color, or accordingly the ones of the verso halftone color. The model should be limited to printing supports where the inks do not penetrate too much deeply, but a more exhaustive testing is needed to see the precise limitations in practice. The model showed its performance when multiview images are produced from layouts computed with it. Since these prints rely on color matching between different rectoverso halftones, the quality of the displayed images directly depends on the accuracy of the model.

Fig. 1 .

 1 Fig. 1. Flux transfers between two flat components (arrows do not render orientation of light).

  r

  ) in terms of its transfer factors R, T, R and T. The multiplicative property of transfer matrices is true for any number of components, and the left-to-right position of the matrices in the product reproduces the front-to-back position of the corresponding components. Every transfer matrix has the structure displayed in Eq. (2) and from a given transfer matrix m  , one retrieves the transfer factors in the following way:

Fig. 2 .

 2 Fig.2. Flux transfers between a diffusing layer (split into two identical sublayers) and its bordering interfaces, and corresponding flux transfer matrices.

Fig. 4 .

 4 Fig. 4. Multiview recto-verso halftone print produced by laserjet printing on doubled office paper, displaying different images (a) on the verso side in reflection mode, and (b) on the recto side in transmission mode, under sunlight.

Fig. 5 .

 5 Fig. 5. Multiview recto-verso halftone print produced by laserjet printing on doubled office paper, displaying a mosaic of randomly distributed color squares on the recto side (a) and verso side (b), and homogenously colored patterns surrounded by a contrasted color mosaic in 0.9 transmission mode (c, with increased lightness).

Table 1 .

 1 Average CIELAB ΔE 94 value between predicted and measured spectra

	Support	Printer Set	Mode		Model
					DLRT DCT	YN
	Supercalen-	Inkjet	36 samples Reflectance	0,76	0,92	0,7
	dered paper			Transmittance 0,75	1,01	0,94
		Laserjet 25 samples Reflectance	1,43	1,52	1,5
				Transmittance 0,87	1,5	1,18
	Office paper	Inkjet	78 samples Reflectance	1,31	1,06	1,1
				Transmittance 1,17	1,58	1,28
		Laserjet 25 samples Reflectance	1,39	1,27	1,35
				Transmittance 0,89	2,31	1,38


Joan Sol Roo
email: joan-sol.roo@inria.fr

Martin Hachet
email: martin.hachet@inria.fr

Towards a Hybrid Space Combining Spatial Augmented Reality and Virtual Reality

Keywords: Mixed Reality, Augmented Reality, HMD, 3D Interaction Index Terms: H.5.1 [Multimedia Information Systems]: Artificial, augmented, and virtual realities-; H.5.2 [User Interfaces]: Input devices and strategies-

: A coherent interaction with an augmented space, combining Spatial Augmented Reality and an immersive head mounted display (HMD). Left: a user interacts with an augmented mock-up. Center: wearing the HMD's enables to see virtual elements. Right: two users can interact and collaborate each using a different modality.

INTRODUCTION

Spatial Augmented Reality (SAR) mixes physical and digital information directly onto the real world by the mean of screens or projectors [START_REF] Raskar | Shader lamps: Animating real objects with image-based illumination[END_REF]. SAR leverages our already available skills to interact with the physical world. One or more users can interact with the same coherent illusion, without the obstruction of instrumentation (Figure 1, left).

Despite its huge potential, SAR has also limitations. The used technology may penalize the immersion of the user in the hybrid world (e.g., the light path of the projectors may generate occlusions, and the resulting image will depend on the projector brightness and the surface material). Moreover, the need of a physical support to display digital information is a strong constraint of SAR. Indeed, it is not possible to project mid air information with the current technology, unless using anamorphic illusions [START_REF] Benko | Dyadic projected spatial augmented reality[END_REF]. Even when this limitation is partially overcome by see-through augmented reality (ST-AR) devices such as Microsoft Hololens, they are still constrained by the physical world, with the additional requirement to instrument the user. Alternatively, immersive head mounted displays for Virtual Reality (HMD-VR) allow us to override completely the perceived reality; with HMD-VR it is possible to freely interact with the perceived scene, moving and scaling objects or changing the point of view independently from the physical space. There is then a clear trade-off between instrumenting the user to increase the quality of the illusion, and keeping the interaction unencumbered.

An example that benefits from mixed reality are mock-ups, physical replicas used to preview artifacts or buildings. Thanks to SAR, these physical objects can be augmented with dynamic content, albeit with limited resolution and without mid air information (Figure 1). This is particularly interesting for prototyping [START_REF] Marner | Physical-virtual tools for spatial augmented reality user interfaces[END_REF], or for a group of persons to perceive how the final project would look like, and to discuss the possible improvements. The problem with this approach is that the scene cannot be scaled or navigated. On the other hand, HMD-VR allow users to perform these interactive tasks [START_REF] Kato | Virtual object manipulation on a table-top ar environment[END_REF], at the cost of losing the physical properties. Combining multiple mixed reality modalities would complement their individual weaknesses.

Our motivation for this work is to explore how SAR and HMD-VR technologies can complement each other. We aim at bridging the gap between these two approaches that have tended to evolve in separate tracks. More precisely, we explore how to favor transitions between SAR and VR by proposing unified interaction and several awareness helpers.

RELATED WORK

The notion that the physical and digital worlds can be combined in varying degrees is not new, first formalized in 1994 by Milgram et al. with the virtuality-reality continuum [START_REF] Milgram | Augmented reality: A class of displays on the reality-virtuality continuum[END_REF]. This section briefly describes previous work that focused on enabling the combination and transition between mixed realities.

See-through displays and SAR have been combined in the past, notably in order to complement the HMD's high resolution with the projectors' scalable FOV (field of view) [START_REF] Ilie | Combining head-mounted and projector-based displays for surgical training[END_REF][START_REF] Benko | Fovear: Combining an optically see-through near-eye display with projector-based spatial augmented reality[END_REF]. In the context of multi-display environments, the combination of screens and projection has been studied, both with [START_REF] Butz | Enveloping users and computers in a collaborative 3d augmented reality[END_REF] and without see-through technologies [START_REF] Rekimoto | Augmented surfaces: a spatially continuous work space for hybrid computing environments[END_REF][START_REF] Gervais | Tangible viewports: Getting out of flatland in desktop environments[END_REF]. Other hybrid mixed reality systems have also been explored. For example, Magicbook [START_REF] Billinghurst | The magicbook-moving seamlessly between reality and virtuality[END_REF] combines a physical non-augmented book with video see-through, enabling the transition from non augmented interaction, through ST-AR to immersive VR. Dedual et al. [START_REF] Dedual | Creating hybrid user interfaces with a 2d multi-touch tabletop and a 3d see-through head-worn display[END_REF] proposed a system that combines interactive tabletops with video see-through AR. Smarter Objects [START_REF] Heun | Smarter objects: using ar technology to program physical objects and their interactions[END_REF] and ex-Touch [START_REF] Kasahara | extouch: Spatiallyaware embodied manipulation of actuated objects mediated by augmented reality[END_REF] use video ST-AR to control embedded systems; even when the physical artifact was the focus of attention, no spatial augmentation was presented, except the electronic behaviour itself. Closer to our work is Metadesk [START_REF] Ullmer | The metadesk: models and prototypes for tangible user interfaces[END_REF] by Ullmer and Ishii, which combines tabletop interaction, tangible tokens and a see-through AR window.

The aforementioned projects are based on complementary technologies, combined in a unified frame of reference, where the scene is observed from a unique physical location. Compared to these works, we are interested in interaction metaphors where the virtual component is used to explore different scales and points of view, without being restricted to the user's location. On this direction, Ibayashi et al. created Dollhouse VR [START_REF] Ibayashi | Dollhouse vr: a multi-view, multi-user collaborative design workspace with vr technology[END_REF], where users can interact with a virtual space, either with a top view (displayed on a tabletop), or from inside (using a HMD-VR). Our work extends Dollhouse VR by supporting a non-flat physical support (using SAR instead of a tabletop), while also providing a see-through modality between physical and immersed.

Our work complements the state of the art by supporting transitions between SAR and HMD-VR, using a physical artifact as starting point while also enabling free navigation of the virtual scene.

SAR/VR HYBRID SPACE

We propose a hybrid SAR/VR hybrid space that enables both 1) the interaction of one or various users with an augmented object, and 2) immersive experiences. A unified view provides traceability of what happens in VR while in SAR, and vice versa. This can be used for symmetric and asymmetric collaboration between users, or to increase the perceived information for one user. This section describes the interaction techniques for one user, yet multiple users can use the system at the same time, their activities visible to one another thanks to the the considerations presented on Section 3.7.

Scenario

As an example, let us consider two users interacting with the hybrid mock-up. They can explore it by simply physically moving around, and interact with their hands (touching or pointing with their finger). They can also use an augmented tool to do annotations directly onto the mock-up. Now, one user wants to push the frontiers of physicality. By wearing a VR helmet, she can observe additional information (e.g. the virtual landscape surrounding the mock-up). She can also decide to navigate inside the scene to observe the mock-up from an egocentric point of view. Navigating virtually around the mock-up with an exocentric point of view, the user can change her perception of the scene by observing it from a location that would not be accessible with a standard AR approach. A scanned version of both users can be displayed to reinforce the link with the physical world. The navigation can involve both a change in position and scale. At all times, any of the users can annotate the mock-up, and the annotation will be visible to the other user. For awareness reasons, if one of the users is immersed in the mock-up, the position is indicated on the surface.

While in VR, it is also possible to use traditional virtual interaction, complementing classical limitations of the physical world. Virtual objects can be translated, rotated and scaled, and the internal components exposed. These changes can be reflected on the augmented mock-up to different extents, depending on the available physical geometry.

Scene Elements

In order to understand the similarities and differences on how a user interacts with the mock-up, it is necessary to mention that an element on the scene can be either: physical, virtual, or augmented (both physical and virtual).

• Physical elements are already available in reality and do not need to be displayed in SAR

• Virtual elements can be seen in VR, but they are not readily available in SAR. Nevertheless, they can be displayed as simplified augmented surfaces (for example, generating a 2d projection of a 3d object)

• Augmented elements have dual physical-virtual characteristics, and they can be interacted with differently in SAR and VR

Unified Interaction

In order to support a unified interaction with both SAR and VR, we decided to implement all the interaction techniques using a 6DoF wand controller.

Pointing and Annotation

Most interactions with the scene require to be able to point. This is solved via ray-casting, using the controller's forward direction, with an inclination of 55 downwards to ease the interaction with nearby objects. While in VR, the cursor is represented with a ray; the SAR modality only displays the target cursor, since it is not possible to show the ray in mid air. In order to perform free hand annotations on a surface, the user can simply point at the target location and press the controller's trigger to start drawing (Figure 5).

Manipulation

Physical objects can be manipulated with the bare hands, and augmented versions can be manipulated when tracked [START_REF] Laviole | Papart: interactive 3d graphics and multitouch augmented paper for artistic creation[END_REF]. Virtual objects can also be manipulated using the controller's grab button. This action is best suited for VR, but it is possible to still perform it in SAR. If the virtual object has a physical counterpart (i.e., they form an augmented object), they will leave behind a ghost when grabbed, and the virtual model will return to the default physical position when dropped, in order to reinforce the illusion. Purely virtual objects can be released in a new position freely.

Bimanual operation is also possible, using one controller to change the position and orientation of the virtual object, and a second one to create annotations. While in SAR, both augmented and virtual objects will reflect the annotations, yet augmented objects will not change position.

Navigation

The physical mock-up can be explored by naturally navigating the physical space, or by virtually changing the point of view (Figure 3).

• Jump in: A novel point of view can be selected by pointing at the desired surface, either on SAR or VR, and then pressing the controller's trackpad. This will teleport the user to the selected position on the surface, while also changing the user's scale to a fixed factor (depending on the mock-up characteristics). While on the surface, the trackpad can be used to adjust the orientation. The user can move by selecting a new jump position. Note that teleportation is the standard navigation technique for HTC Vive, selected to prevent cyber-sickness.

• Orbit: While in VR and outside the mock-up, it is possible to orbit around the table, using the trackpad to indicate the angle.

• Reset navigation: At any time, the user can point the controller up and press the trackpad to return to the natural POV.

Resetting the navigation before removing the HMD eases the transition back to reality.

Awareness

In order to prevent the users to feel lost once they change their point of view, the physical surroundings of the mock-up -including the users-are reconstructed in 3D (Figure 4). To ease the understanding of where the user is located when they "jump in" the mock-up, an arrow is displayed on the SAR version of the system, indicating their position and orientation (Figure 5). This, in combination of the visible cursor enables non-immersed users to communicate with the immersed ones. Navigation creates a disconnection between the physical environment and what the user sees, which can lead to accidents. To prevent this, real objects are shown as ghosts when getting close to them (Figure 6), similar to the HTC Vive approach safety cage.

IMPLEMENTATION

The main application was developed using Unity3D 5.4 and Di-rectX 11. The software runs on an Alienware i7-3820 with 8GiB of RAM and a GeForce GTX 660. The setup is comprised of: a Microsoft Kinect v1, a LG 1500G projector and a HTC Vive with its controllers. The overall system runs at 60 frames per second, without particular optimization considerations, except by the processing of the Kinect data using the GPU.

The calibration of the optical devices was performed offline, setting the center of the table as origin. The projector calibration uses OpenCV's camera calibration functionality (2D to 3D points), while the depth camera was calibrated using Kabsch algorithm (3D to 3D points, rotation and translation). The alignment of HTC Vive with our system's frame of reference is performed online. The transformation between coordinate systems is computed as the inverse of the controller's origin at a known table position (i.e., resting position over the table's origin).

Limitations

The current implementation could be greatly improved using the advances of avatar generation in the area of tele-presence [START_REF] Orts-Escolano | Holoportation: Virtual 3d teleportation in real-time[END_REF][START_REF] Beck | Immersive groupto-group telepresence[END_REF]. Object tracking is not currently supported, yet it can be implemented using off the shelf solutions such as Optitrack (taking into account infrared interference from the current sensors), or markerless tracking [START_REF] Newcombe | Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time[END_REF]. Free hand interaction while in VR could also be supported, using consumer grade sensors such as Leap Motion. The main objective of this project was not to exhaustively explore these alternatives, but instead to take the first steps towards the use of immersive displays to support spatially augmented scenes.

INTERVIEWS

We conducted informal interviews with 3 colleagues from related fields to know more about the acceptability and viability of the hybrid space. We presented the system features incrementally, and discussed their perspective on the positive aspects as well as the limitations of the system. The collected comments are briefly summarized in this section.

• Participant 1 (tangible interfaces and SAR): he was interested on the flexibility of the system in contrast with pure tangible interfaces, specially the change in perspective, while not completely leaving the physical world. He suggested to include support for direct interaction.

• Participant 2 (collaborative VR): he was particularly pleased with the feeling of presence once inside the mock-up. He suggested to support mid air annotations while in VR, and also recommended to track the user' hands, using for example skeleton tracking.

• Participant 3 (Spatial navigation and maps): she found that the mock-up was a good example of semantic zoom, with the physical props working as low resolution landmarks. Nevertheless, she mentioned that the translation action was too abrupt, and that it would benefit from a smoother transitioning.

Even when these interviews did not validate the system, they showed the system can be promising for different fields, yet further work would be required.

CONCLUSIONS

In this work we presented a hybrid space comprised of Spatial augmented reality augmentation and immersive displays. Interaction techniques were presented to annotate and interact with the scene, along with the navigation of the virtual counterpart. The discussions with practitioners in related fields show that merging VR and SAR is promising, yet there is space for improvement by adopting additional interaction techniques, such as direct interaction or skeleton tracking.

In this work we took the example of mock-ups in the field of architecture. Of course, this approach extends to many scenarios, including object design, physical visualization [START_REF] Jansen | Opportunities and challenges for data physicalization[END_REF], or education.

The perception of this hybrid space as a unique world is a key requirement for its success. Future work will involve perceptual user studies, which will help us to improve this hybrid space and to make the link between the two worlds stronger; then, it would be of interest to study the interaction between multiple users in collaborative scenarios. These studies will build towards the creation of novel hybrid applications.

Figure 2 :

 2 Figure 2: Pointing on the mock-up using a wand controller. Left: pointing in SAR shows a target directly onto the surface. Right: pointing in VR shows a ray.

Figure 3 :

 3 Figure 3: Detaching the point of view from the physical location. The user in gray represents the physical location, while the blue user indicates the perceived location. Left: traveling inside the mock-up. Right: orbiting around the table.

Figure 4 :

 4 Figure 4: The user orbited left, so he can see a 3D reconstruction of himself and the chairs when looking to his right.

Figure 5 :

 5 Figure5: Left: the user is inside the virtual room looking at a building. Right: the user's position and orientation are indicated with a yellow arrow in SAR. The annotation over the augmented object is visible in SAR, while the annotation over the virtual wall is presented as a top-down flat projection.

Figure 6 :

 6 Figure 6: If the controller approaches physical objects (in this case, the table and the right building), their bounding boxes are displayed. Left: since the user did not change the point of view, then the bounding box matches the perceived virtual element. Right: the user performed an orbit operation to the left, yet the bounding box is still displayed in the same physical location.

ACKNOWLEDGEMENTS

This work was supported by the ANR-ISAR project, ANR-14-CE24-0013.


Augmented Human Experience: Spatial Augmented Reality and Physiological Computing

Keywords: Spatial Augmented Reality, Calm Technologies, Physiological Computing, Tangible Interaction. ACM Classification H.5.1. Multimedia Information Systems: Artificial, augmented, and virtual realities, H.5.2 User Interfaces: Prototyping

Human Computer Interfaces are in constant advance. Current research focuses on moving the interaction out of the limited reach of computers into the environment, but the focus is still on performance. The maturity of physiological sensors enables interaction that takes into account the user's inner state, but users are not really aware of such states. It is possible to take a step further and expose those inner states to the user, which have been shown to increase well-being. This talk will include an introduction to the joint field and a set of projects that work towards building an augmented human experience.

Introduction

Twenty-five years ago, computer interfaces were composed mostly of text based applications or simple

Copyright is held by the author/owner(s). This article was discussed at the Second European Tangible Interaction Studio, held in Fribourg, Switzerland, on January 25-29, 2016. More info at: http://www.etis.estia.fr/ Joan Sol Roo Inria Bordeaux, France joan-sol.roo@inria.fr

Renaud Gervais

Inria

Bordeaux, France renaud.gervais@inria.fr

Jeremy Frey

Bordeaux university

Bordeaux, France jeremy.frey@inria.fr

Martin Hachet

Inria, Labri

Bordeaux, France martin.hachet@inria.fr Graphical User Interfaces (GUI), restricted to desktop computers. Nowadays, computational devices such as notebooks, smartphones and tablets are ubiquitous. These devices allow us to access powerful and flexible applications whenever required, but these applications still live on a different -virtual -realm, and can only be reached through special windows -screens -that require an interaction approach that greatly differs from our experience on the real world.

Several research areas look to reduce the gap that separates real and virtual realms, such as Spatial Augmented Reality (SAR, complementing the real world with computer generated information, placing the information directly onto the environment) [START_REF] Raskar | Interacting with spatially augmented reality[END_REF], Tangible User Interfaces (physical handles coupled with virtual information) [START_REF] Shaer | Tangible user interfaces: past, present, and future directions[END_REF] and Ubiquitous Computing (computational power available whenever required, anytime and everywhere) [START_REF] Weiser | Ubiquitous computing[END_REF]. These research areas share a view that takes into account the integral role of the human on the interaction: which of the already available capabilities of the user can we leverage, while taking into account the user's experience.

The maturation of physiological computing on recent years have opened yet another realm on interaction: interfaces that are aware of the user's inner-state [START_REF] Gateau | Real-Time State Estimation in a Flight Simulator Using fNIRS[END_REF]. While promising, they are transparent for users that are not really aware of what is happening inside of them [START_REF] Nisbett | Telling more than we can know: Verbal reports on mental processes[END_REF]. Using the same sensing capabilities, we can create interfaces that expose the user's inner state in different ways. Instead of focusing on performing tasks, we can foster awareness of our own humanity.

Computer interfaces should not be only measured in performance. One of the cornerstones of the Ubiquitous Computing vision was about creating calm, but we seem to have used productivity tools to increase our overall pace of life instead of using the extra time to relax and make our lives easier. As knowledge work is getting more prevalent, this is becoming a problem, especially knowing the increasing body of research showing the negative impact of stress on health and productivity [START_REF] Lupien | Effects of stress throughout the lifespan on the brain, behaviour and cognition[END_REF].

Mindfulness, contemplation and overall interoceptionthe ability to sense the origin of one's bodily signalshave been shown to have a positive impact on the ability to cope with stress and increase well-being [START_REF] Farb | Interoception, Contemplative Practice, and Health[END_REF].

Calm computing [START_REF] Weiser | Designing calm technology[END_REF] and slow technology [START_REF] Hallnäs | Slow technology--designing for reflection[END_REF] have the potential to be a great medium to foster self-reflection [START_REF] Grosse-Hering | Slow design for meaningful interactions[END_REF][START_REF] Odom | Designing for slowness, anticipation and re-visitation: a long term field study of the photobox[END_REF].

Augmented Human Experience

The same way that Augmented Reality provides complementary virtual information to the real world, we can work towards providing complementary external world information about the user's internal world; we will call this "augmented human experience". This talk will include the presentation of three works that build on an augmented humane experience: Teegi [START_REF] Frey | Teegi: Tangible EEG Interface[END_REF] (Figure 1), Tobe [START_REF] Gervais | TOBE: Tangible Out-of-Body Experience[END_REF] (Figure 2) and Inner Garden [START_REF] Sol Roo | Inner Garden: an Augmented Sandbox Designed for Self-Reflection[END_REF] (Figure 3). These works focus on exposing inner states at different levels (from brain signal lectures to high level metrics such as workload or attention, from lung inflation to breathing variability), using a form factor that foster manipulation and playful learning (tangible avatars and a sandbox). For this end, Tangible User Interfaces offer a perfect support, providing a physical representation, which we can explore with our hands; fast prototyping is performed using SAR, once validated it is possible to move towards instrumented objects instead of SAR.

The objective of this talk is to foster dialog about the potential of these interfaces, and how we can move towards a more humane experience, instead of restricted by computers, empowered by them.

Figure 1 :

 1 Figure 1: Teegi (tangible EEG interface) enables to explore our brain activity in real time.

Figure 2 :

 2 Figure 2: Tobe (Tangible Out of Body Experience) enables users to customize representation of both low level signals and high level metrics.

Figure 3 :

 3 Figure 3: Inner garden, a world in miniature that evolves according to the user's inner state.