T Ablekim

J N Duenow

C L Perkins

J Moseley

X Zheng

T Bidaud

B Frouin

S Collin

M O Reese

M Amarasinghe

E Colegrove

S Johnston

W K Metzger
email: wyatt.metzger@nrel.gov

Exceeding 200-ns Lifetimes in Polycrystalline CdTe Solar Cells

CdTe photovoltaics has achieved one of the lowest levelized costs of electricity among all energy sources. However, for decades, carrier lifetimes have been inferior to those of other prevalent solar cell materials. This quality has inhibited common methods to improve solar cell efficiency such as back-surface fields, electron reflectors, or bifacial solar cells. In this work, we demonstrate a significant increase in carrier lifetime to values exceeding 200 ns in fully functional CdTe solar cells. The increased lifetime is achieved in these complex polycrystalline cells by realizing multiple characteristics simultaneously-the formation of oxidized tellurium species at the transparent conducting oxide interface, large CdSeTe grains at the absorber/emitter interface, and intragrain passivation in the absorber layer. The carrier lifetime is correlated to the open-circuit voltage and enables paths for back-surface manipulation and novel cell architectures to further improve CdTe photovoltaic performance.

Introduction

The quality and performance of photovoltaic materials such as Si, GaAs, perovskites, Cu(In,Ga)Se2, and CdTe depend on the average time that carriers excited by sunlight are able to drift and diffuse prior to recombination. [START_REF]Handbook of Photovoltaic Science and Engineering[END_REF] Typical CdTe carrier lifetimes-several ns to tens of ns-are relatively short compared to the hundreds of ns achieved in CIGS and perovskites, [START_REF] Gloeckler | CdTe solar cells at the threshold to 20% efficiency[END_REF][START_REF] Metzger | Time-resolved photoluminescence studies of CdTe solar cells[END_REF][START_REF] Guo | Effect of selenium and chlorine co-passivation in polycrystalline CdSeTe devices[END_REF][START_REF] Metzger | Exceeding 20% efficiency with in situ group V doping in polycrystalline CdTe solar cells[END_REF][START_REF] Metzger | Long Lifetimes in high-efficiency Cu(In,Ga)Se 2 solar cells[END_REF][START_REF] Dequilettes | Impact of microstructure on local carrier lifetime in perovskite solar cells[END_REF] and microseconds and milliseconds achieved in GaAs and Si, respectively. [START_REF] Casey | Concentration-dependent absorption and spontaneous emission of heavily doped GaAs[END_REF][START_REF] Lush | Microsecond lifetimes and low interface recombination velocities in moderately doped n-GaAs thin films[END_REF][START_REF] Niewelt | Taking monocrystalline silicon to the ultimate lifetime limit[END_REF] This is one reason why CdTe record solar cell efficiency is less than in these other material systems. [START_REF]Best Research-Cell Efficiency Chart[END_REF] Furthermore, the resulting diffusion length limits the ability to implement back-surface passivation and carrier reflection, which are critical to existing high-efficiency Si and III-V solar cell technologies. [START_REF]Handbook of Photovoltaic Science and Engineering[END_REF] It also restricts semiconductor device designs such as bifacial and interdigitated back-contact solar cells. Increased lifetime in and of itself can improve performance. Recent modeling indicates that for typical CdTe solar cell thickness and properties, carrier lifetime exceeding about 100 ns can also enable backsurface passivation and designs to significantly increase efficiency to values as high as 28%, provided other properties are also optimized. [START_REF] Duenow | Back-surface recombination, electron reflectors, and paths to 28% efficiency for thin-film photovoltaics: A CdTe case study[END_REF] GaAs solar cells, which have a similar bandgap, have demonstrated >28% efficiency in part due to long carrier lifetime and back-surface fields; [START_REF] Green | Solar cell efficiency tables[END_REF] but applications are limited by the low throughput, and costs/W are currently about 500-1000 times greater than those of commercial modules for residential and utility applications. [START_REF] Wilson | Photovoltaics Technologies Roadmap[END_REF] The high throughput of module production has positioned CdTe as a cost leader in utility photovoltaics; substantive lifetime gains can enable higher efficiency at levelized costs of electricity well below those of conventional fuels. [START_REF]Lazard's Levelized Cost of Energy Analysis -Version 13[END_REF] Figure 1 illustrates a schematic of the superstrate CdTe solar cell configuration. From about 1992 to 2012, the most common n-type layers were either SnO 2 /i-SnO 2 or CdSnO x /ZnSnO x bilayers followed by CdS. [START_REF] Britt | Thin-film CdS/CdTe solar cell with 15.8% efficiency[END_REF][START_REF] Wu | High-efficiency polycrystalline CdTe thin-film solar cells[END_REF] Carrier lifetime measured by time-resolved photoluminescence through the glass increased substantively to about 5-30 ns in typical devices when CdS was replaced with CdSeTe bandgrading; this was a key component of the increase in efficiency from 16.7% to 22.1%. [START_REF] Metzger | Time-resolved photoluminescence studies of CdTe solar cells[END_REF][START_REF] Guo | Effect of selenium and chlorine co-passivation in polycrystalline CdSeTe devices[END_REF][START_REF] Metzger | Exceeding 20% efficiency with in situ group V doping in polycrystalline CdTe solar cells[END_REF][START_REF] Wu | Phase control of CuxTe film and its effects on CdS/CdTe solar cell[END_REF][19][START_REF] Munshi | Polycrystalline CdSeTe/CdTe absorber cells with 28 mA/cm 2 short-circuit current[END_REF][START_REF] Zheng | Recombination and bandgap engineering in CdSeTe/CdTe solar cells[END_REF][START_REF] Fiducia | Understanding the role of selenium in defect passivation for highly efficient selenium-alloyed cadmium telluride solar cells[END_REF] Eliminating CdS substantively reduced absorption of sunlight with wavelengths less than 520 nm, thereby increasing photocurrent in the blue end of the solar spectrum. At the same time, the CdSeTe alloy formed by Se diffusion during the post-deposition CdCl 2 anneal decreased the bandgap to ~1.4 eV, thereby increasing photocurrent in the red region of the solar spectrum. Critically, open-circuit voltage (V oc ) did not decrease commensurately, in large part because CdSeTe demonstrated greater lifetime than CdTe. 23 A fundamental challenge is that the fast deposition (few to >10 µm/min) of CdTe-based absorber material on a nanocrystalline heterointerface results in as-deposited lifetimes on the order of tens of picoseconds, 3 a poor starting point remedied only partially by recrystallization and passivation during post-deposition CdCl 2 or MgCl 2 processing. [START_REF] Metzger | Time-resolved photoluminescence studies of CdTe solar cells[END_REF][START_REF] Major | A low-cost non-toxic post-growth activation step for CdTe solar cells[END_REF][START_REF] Poplawsky | Structural and compositional dependence of the CdTe x Se 1-x alloy layer photoactivity in CdTe-based solar cells[END_REF][START_REF] Moseley | Recombination by grain-boundary type in CdTe[END_REF] Nonetheless, recent developments indicate that long lifetimes in CdTe-based material are possible. Single crystals doped with P demonstrated that bulk CdTe is capable of radiatively limited lifetimes reaching hundreds of nanoseconds, commensurate with GaAs over the ideal hole density range for solar cells. [START_REF] Burst | CdTe solar cells with open-circuit voltage breaking the 1 V barrier[END_REF] In parallel, depositing thick large-grain films with standard polycrystalline equipment, growth rates, and source material revealed that CdTe lifetimes of tens of ns were possible without the CdCl 2 treatment, indicating that relatively low-impurity source material does not necessarily introduce lifetime-limiting impurities. [START_REF] Jensen | Long carrier lifetimes in large-grain polycrystalline CdTe without CdCl 2[END_REF] More recently, decay times of hundreds of nanoseconds have been observed by growing polycrystalline glass/Al 2 O 3 /CdSeTe/Al 2 O 3 or glass/MgZnO/CdSeTe/MgZnO double heterostructures and subjecting them to a CdCl 2 anneal. [START_REF] Kephart | Sputter-deposited oxides for interface passivation of CdTe photovoltaics[END_REF] The results are promising, although multiple diffusion length measurements have yet to corroborate lifetimes in these idealized test structures, and the structures are distinct from high-efficiency polycrystalline CdTe solar cells.

Here, we examine glass/SnO 2 :F/i-SnO 2 /MgZnO/CdSeTe/CdTe cells as a function of Se composition in the CdSeTe layer. The chemistry at the MgZnO/CdSeTe interface is examined by producing atomically smooth planar cleaves in a controlled glovebox environment followed by X-ray photoelectron spectroscopy (XPS) and other measurements. The measured recombination lifetime is strongly correlated to device performance and reaches values exceeding 200 ns. The carrier lifetime is not driven by one mechanism, but instead, by a confluence of interfacial chemistry, recrystallization of large CdSeTe grains at the MgZnO/CdSeTe interface, and intragrain and grain-boundary passivation in the absorber layer.

Compositional dependence and recombination

MgZnO was deposited by sputtering on TEC12D commercial glass with SnO 2 :F and nominally undoped SnO 2 . [START_REF] Ablekim | Tailoring MgZnO/CdSeTe interfaces for photovoltaics[END_REF] About 1 µm of CdSeTe was deposited by thermal evaporation from a ternary CdSe x Te 1-x source. The Se content was varied from 0% to 40% in increments of 10%. Afterwards, CdTe was thermally evaporated from a separate source, and the samples were subjected to a CdCl 2 anneal of 420ºC. Devices were completed by immersing the samples in a CuCl 2 solution followed by Au metallization. Figure 2 illustrates the external quantum efficiency (EQE) measured as a function of alloy composition and a representative current densityvoltage (J-V) curve for Se composition of 30% with a V oc of 843 mV, short-circuit current density (J sc ) of 29.4 mA/cm 2 , and efficiency of 17%-18%. The lifetimes were measured by time-correlated single-photon counting using a laser pulse at 640 nm incident through the glass and absorbed primarily in the CdSeTe absorber region. [START_REF] O'connor | Time-Correlated Single Photon Counting[END_REF] Figure 3a illustrates a representative curve. The latter section of the decay curve is fit as has become customary in the field, and corresponds to a decay time exceeding 250 ns. Long decay times can be associated with traps or carrier kinetics. For example, if the electron-hole excitation is low relative to the equilibrium hole density, the p-n junction electric field separates holes and electrons to the edge of the p-and n-type regions, respectively. These carriers can then gradually leak back, recombine, and luminesce, thereby creating relatively long decay tails that are not correlated to device performance. [START_REF] Metzger | Analysis of charge separation dynamics in a semiconductor junction[END_REF] Here, the injection levels (10 16 -10 17 cm -3 ) are high relative to the equilibrium carrier density, which serves to screen the electric field. [START_REF] Metzger | Analysis of charge separation dynamics in a semiconductor junction[END_REF][START_REF] Metzger | Characterizing recombination in CdTe solar cells with time-resolved photoluminescence[END_REF] Long decay curves can also be generated by trap capture and reemission, in which case the measured lifetime is not directly related to the diffusion length and photovoltage. [START_REF] Ahrenkiel | Minority Carriers in III-V Semiconductors: Physics and Applications[END_REF] Figure 3b illustrates that the V oc is correlated to the logarithm of the measured carrier lifetime. This relation is consistent with basic device physics, [START_REF] Metzger | Time-resolved photoluminescence studies of CdTe solar cells[END_REF][START_REF] Fahrenbruch | Fundamentals of Solar Cells: Photovoltaic Solar Energy Conversion[END_REF] and it indicates that the lifetimes are not produced by an artifact unrelated to solar cell performance. Figure 3b also illustrates that the lifetime generally increases with increasing Se concentration until reaching a value of 30%. On average, the measured lifetime decreases as the Se level increases from 30% to 40%. By conducting different characterization experiments, it is possible to gain insight into the underlying mechanisms adjusting recombination.

Physical Mechanisms

To assess the physical mechanisms creating these very long lifetimes, we used a thermomechanical cleaving method to cleanly separate a number of devices at the MgZnO/CdSeTe interface. [START_REF] Mcgott | Thermomechanical lift-off and recontacting of CdTe solar cells[END_REF] This method exposes the critical p-n junction interface that changes during subsequent deposition, CdCl 2 annealing, and back-contact processing for examination by XPS and other surface-sensitive probes. Cleaving occurs in an argon-filled glovebox that is connected to a surface analysis cluster tool. Thus, the samples are not exposed to air, oxidation, polishing, focused-ion-beam milling, or other damaging sample preparation, enabling a unique examination of these poorly understood interfaces.

Figure 4a illustrates XPS spectra of the Te 3d 5/2 region taken from the CdSeTe side of cleaved devices. The XPS is sensitive to depths of just several nanometers and reveals two peaks associated with Te 2-and Te 4+ . The spectra in Fig. 4a are normalized in height to the Te 2-peak. The Te 2-stems from tellurium in the CdSeTe alloy. The Te 4+ is oxidized tellurium that had been directly in contact with the transparent conducting oxide. [START_REF] Perkins | 2 -catalyzed oxidation in highefficiency CdTe solar cells[END_REF] Such oxidized tellurium is typically found at the air-exposed surfaces of CdTe and has been associated with native oxides such as CdTeO 3 , the thermodynamically most stable CdTe oxide. [START_REF] Carmona-Rodriguez | CdTeO x to CdTeO 3 structural phase transition in asgrown polycrystalline films by reactive sputtering[END_REF] Figure 4a demonstrates that the extent of tellurium oxidation depends on the Se content of the CdSeTe source material. In fact, adding Se, but not too much, can actually promote Te oxidation. Figure 4b illustrates that the ratio of [Te 4+ ]/[Te 2-+ Te 4+ ] summed from both sides of the interface is strongly correlated to carrier lifetime. Figure 4c illustrates that V oc progressively climbs with the lifetime and [Te 4+ ]/[Te 2-+ Te 4+ ] ratio, and the V oc then saturates, as predicted for absorbers with low hole density (~10 14 cm -3 ). Interestingly, the [Te 4+ ]/[Te 2-+ Te 4+ ] ratio is a better predictor of lifetime than the absorber Se composition. To image electro-optical properties across the CdSeTe and CdTe absorber, an electron beam was scanned across a beveled surface to generate electron-hole pairs with ~100-nm spatial resolution (Fig. 5a). Cathodoluminescence (CL) spectra were collected at each position, and the peak photon energy and intensity were averaged parallel to the junction as a function of depth into the film. Figure 5b illustrates that the peak photon energy successively drops with increasing Se source concentration at the CdSeTe/MgZnO interface (distance = 0 µm); for all source concentrations, the peak energy gradually climbs to values consistent with CdTe toward the back of the film. The injection levels correspond to electron-hole generation on the order of 10 16 -10 17 cm -3 , which screens the field created by an equilibrium hole density of 1×10 14 -5×10 14 cm -3 and leads to an intensity that is driven by the electron and hole carrier lifetimes. [START_REF] Moseley | Physics, Simulation, and Photonic Engineering of Photovoltaic Devices[END_REF] Once the beam is fully within the absorber, where there is more Se, there is generally more CL intensity (Fig. 5c). Supplemental Fig. 1 illustrates areal CL intensity and bandgap images that corroborate these findings. The correlation of the CL intensity with the absorber grain structure provides more insight into the mechanisms controlling carrier lifetime. The first row of Fig. 6 illustrates CL intensity as a function of depth and Se composition using a common scale. The Attolight CL-scanning electron microscope described in the Methods section is specially designed to allow quantitative CL intensity comparisons between different samples. The CdTe (x = 0) samples appear dark due to less intensity throughout the film. For increased Se concentration at the front interface, the CL intensity similarly increases, whereas the CL intensity in the CdTe region near the back of the film remains dim. This is consistent with Se content being correlated to CL intensity and reducing recombination. The CL data in the Supplementary section reproduce these features in different sample locations on a standard parabolic mirror system, and they demonstrate that the bright bands correspond to regions where Se content shifts the bandgap. The second row in Fig. 6 shows the CL intensity scaled individually to enhance the contrast for each image. The Se bandgrading at the front interface overlaps with enhanced luminescence, and the grain-boundary intensity relative to the grain interiors is consistently dark. Because grain-boundary potentials are easily screened by the injected carriers, [START_REF] Metzger | The impact of charged grain boundaries on thin-film solar cells and characterization[END_REF] the data indicate that the grain boundaries are defective and not fully passivated, regardless of the Se composition. The data illustrate another physical mechanism that can contribute to very long lifetimes. The grain structure varies as a function of Se content. Initially, as the Se composition increases, the grain size at the front interface becomes progressively larger, peaking at source compositions of 30% and then decreasing for 40%. The initial climb in grain size is correlated with carrier lifetime; however, the 40% composition shows longer lifetimes than compositions from 0% to 20%, yet fails to show significantly larger grains. The results illustrate that there are several physical mechanisms, rather than just one, that contribute to long carrier lifetime. The Se composition progressively increases intragrain CL intensity. At the same time, interfacial oxidized tellurium and grain size peak along with measured lifetime for Se contents of 30%. The optimal confluence of these multiple effects contributes to measured lifetimes in excess of 200 ns.

Conclusions

For decades, the measured carrier lifetime in high-quality CdTe solar cells has been inferior to those of current CIGS, perovskite, and GaAs solar cells. This has been one limitation to CdTe reaching the record cell efficiencies of these other technologies. This also has impeded the success of implementing back-surface fields, electron reflectors, and back-surface passivation to further increase efficiency, as well as the ability to establish new cell configurations such as bifacial or interdigitated back-contact cells. Here, lifetime values reaching 250 ns are achieved in fully functional CdTe solar cells. The long lifetimes are not easily attributed to any one physical mechanism, but instead, to several simultaneous features: the formation of a thin layer of oxidized species at the p-n junction interface, recrystallization of large CdSeTe grains at the absorber/emitter interface, and grain-boundary and intragrain recombination mitigation in the absorber layer. The carrier lifetime is correlated to the open-circuit voltage and enables paths for back-surface manipulation and novel cell architectures to further improve CdTe photovoltaic performance and reduce the costs of solar electricity.

Methods

Materials synthesis. The cell layers were deposited on NSG TEC12D soda-lime glass commercially coated with fluorine-doped tin oxide (SnO 2 :F) and intrinsic tin oxide (i-SnO 2 ). MgZnO was deposited on the i-SnO 2 by RF sputtering from a compressed target of ZnO and MgO. The weight percent (wt%) of MgO was 4%; the remainder was ZnO. The x value in Mg x Zn 1-x O films deposited from this target was about 0.08. The CdSeTe and CdTe layers were then thermally evaporated sequentially without breaking vacuum. The CdSe x Te 1-x source material had Se compositions ranging from x=0 to 0.4. The chamber base pressure was mid-10 -6 Torr. The source and substrate temperatures for CdSeTe and CdTe were about 680°C and 400°C, respectively. The CdSeTe layers were nominally 1 µm thick, and the CdTe layers were 3-4 µm. After CdSeTe and CdTe deposition, the samples were CdCl 2 -treated at 420°C for 10 minutes. Residual CdCl 2 was rinsed off with deionized water. The samples were then dipped in a 0.1 mMol CuCl 2 solution in water and annealed in a tube furnace at 200°C for 30 minutes in air. Finally, a 100-nm-thick Au contact was deposited by evaporation. The cells were isolated with mechanical scribing with a cell area of about 0.25 cm 2 .

JV measurements were measured in the dark and under simulated 1-sun illumination using a Class AAA solar simulator with a Xe-arc lamp and AM1.5G filter from PV Measurements. The total cell area was measured with a system installed by the NREL Cell and Module Characterization group that uses image capture, a calibrated CNC x-y stage, and area-measurement software. The illumination intensity was calibrated using an encapsulated Si cell certified by the NREL Cell and Module Characterization group. Spectral mismatch errors based on reference and test cells are estimated to be less than 4%. The solar cells were measured in the laboratory ambient (air) and were scanned starting in the forward direction. We did not use preconditioning or observe hysteresis. Integrated absolute quantum efficiency response was measured with a Newport Oriel IQE 200 system.

Characterization. One-photon excitation time-correlated single-photon counting was performed by firing laser pulses with a wavelength of 640 nm at a rate of 1.1 MHz with a CW power of 200 µW through the glass onto the absorber layer. [START_REF] O'connor | Time-Correlated Single Photon Counting[END_REF] The beam diameter was 60 µm, corresponding to an initial injection level near the absorber interface of 10 16 -10 17 cm -3 . A silicon avalanche photodiode detector was used for single-photon detection of photons passed through interference filters to separate photoluminescence from scattered laser light. After deconvolution with the instrument system response, 20-ps lifetimes can be resolved. Bevels for the CL measurements were prepared by focused-ion-beam milling in a xT-Nova NanoLab focused ion beam/scanning electron microscope (FIB/SEM) made by FEI Company. An ~1-µm-thick layer of protective Pt was deposited above the eventual beveled surface, which also helps to reduce "curtaining." [START_REF] Munroe | The application of focused ion beam microscopy in the material sciences[END_REF] Samples were tilted 20° to the ion beam, a 30-kV beam was used to shape the bevels, and a 5-kV beam was used to clean the surfaces for CL. Surfaces were cleaned further by milling at a glancing angle with Ar + ions using a JEOL cross-section polisher tool operating at 3 kV. CL spectrum imaging measurements [START_REF] Abou-Ras | Advanced Characterization Techniques for Thin Film Solar Cells[END_REF] were performed at room temperature in an Attolight Chronos CL-SEM. 43 Electron-beam conditions were 6-kV voltage/8-nA current. Luminescence was collected by an achromatic reflective objective (numerical aperture 0.72) that provides constant collection efficiency over a field of view of about 150 µm in diameter. Luminescence spectra were dispersed with a Horiba diffraction grating (150 grooves/mm) and recorded with an Andor Newton charge-coupled device (CCD) camera (1024×256 pixels, pixel width 26 µm). The corresponding spectral dispersion is 0.53 nm per pixel. Luminescence spectra were corrected for the diffraction efficiency of the grating and the sensitivity of the CCD camera. For Supplemental Fig. 1, CL measurements were collected on different beveled areas of the same samples in Figs. 5 and6 at room temperature in a JEOL JSM 7600F field-emission SEM equipped with a Horiba H-CLUE CL system. The H-CLUE includes a parabolic mirror light collector, an iHR320 imaging spectrometer, and a thermoelectrically cooled Syncerity CCD camera. A grating with 300 grooves/mm and blaze at 600 nm was used. Electron-beam conditions were 7.5-kV voltage/~14-nA current.

XPS and Auger electron spectroscopy experiments were conducted in a home-built cluster tool described in Ref. 43. Using an ultra-high-vacuum (UHV)-compatible epoxy (H2OE EPO-TEK), devices were affixed to clean pieces of aluminum foil and allowed to cure overnight at 65°C. In a glovebox attached to our cluster tool, epoxied samples were immersed in a bath of liquid nitrogen where they spontaneously cleaved. Each half was then withdrawn from the liquid nitrogen bath into a stream of dry nitrogen gas until it reached room temperature. Samples were loaded into the UHV transport system and the XPS system such that air exposure was avoided. Data were processed with PHI MultiPak v9.6 using standard elemental sensitivity factors. High-resolution XPS spectra were taken at a normal takeoff angle using monochromatic Al Kα radiation, with a pass energy of 1.75 eV, and in angle-integrating lens mode (±7°).

Figure 1 .

 1 Figure 1. Schematic of a CdTe solar cell. Transparent conducting oxide (TCO) is deposited on glass, followed by different potential buffer layers as described in the text, CdSeTe, CdTe, and back-contact formation. The layers and grain size are not drawn to scale.

Figure 2 .

 2 Figure 2. EQE and JV characteristics. a, EQE for solar cells with Se composition ranging from 0% to 40% indicates successively more photocurrent in the red region as the Se alloying decreases the bandgap near the MgZnO/CdSeTe interface. b, Typical J-V curves for Se composition corresponding to 30%.

Figure 3 .

 3 Figure 3. The correlation of lifetime and open-circuit voltage. a, Characteristic single-photon counting curve measured on a solar cell with 30% Se composition. The green line illustrates a single exponential decay fit. b, The correlation of lifetime vs V oc for CdSeTe/CdTe solar cells with Se composition ranging from 0% to 40%. The straight dashed orange line serves as a guide to the eye in the linear-log plot.

Figure 4 .

 4 Figure 4. The relationship of Te 4+ at the front interface with carrier lifetime and V oc . a, XPS spectra at the front interface b, lifetime, and c, V oc vs [Te 4+ ]/[Te 2-+ Te 4+ ] ratio. The red dashed line is a guide to the eye.

Figure 5 .

 5 Figure 5. The correlation of peak photon energy and CL intensity with film depth. a, Schematic of CL experiment on a beveled sample. b, The peak photon energy and c, CL intensity, averaged parallel to the junction, as a function of distance from the CdSeTe/MgZnO interface.

Figure 6 .

 6 Figure 6. CL intensity contrast versus Se composition. CL intensity of beveled CdSe x Te 1-x samples with compositional x values ranging from 0% to 40%. The top row indicates the intensity for common scaling shown in the legend on the right. The middle row uses individual scaling for each composition to enhance the contrast for each composition. The bottom row indicates the results from the second row in color to enhance visualization throughout the film, particularly for higher Se concentration. The right-hand legend provides the scale. The white bar corresponds to 5 microns along the beveled surface.

This work was authored in part by the National Renewable Energy Laboratory, operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. Funding provided by the U.S. Department of Energy Office of Energy Efficiency and Renewable Energy Solar Energy Technologies Office and CRD-13-507. The views expressed in the article do not necessarily represent the views of the DOE or the U.S. Government. funding listed in the acknowledgments section, the authors declare no competing interests. .

The data that support the plots within this article are available from the authors upon reasonable request.


Jules Françoise

Téo Sanchez

Ml

Marcelle: Composing Interactive Machine Learning Workflows and Interfaces

Fig. 1. Marcelle is a toolkit for interactive machine learning addressing the composition of custom workflows. It implements a component-based architecture using reactive programming for pipeline specification. Components provide views that can be composed to form custom interfaces. Marcelle's architecture facilitates collaboration between machine learning experts, designers and end-users.

Human-centered approaches to machine learning have established theoretical foundations, design principles and interaction techniques to facilitate end-user interaction with machine learning systems. Yet, general-purpose toolkits supporting the design of interactive machine learning systems are still missing, despite their potential to foster reuse, appropriation and collaboration between different stakeholders including developers, machine learning experts, designers and end users. In this paper, we present an architectural model for toolkits dedicated to the design of human interactions with machine learning. The architecture is built upon a modular collection of interactive components that can be composed to build interactive machine learning workflows, using reactive pipelines and composable user interfaces. We introduce Marcelle, a toolkit for the design of human interactions with machine learning that implements this model. We illustrate Marcelle with two implemented case studies: (1) a HCI researcher conducts user studies to understand novice interaction with machine learning, and (2) a machine learning expert and a clinician collaborate to develop a skin cancer diagnosis system. Finally, we discuss our experience with the toolkit, along with its limitation and perspectives.

CCS Concepts: • Human-centered computing → Interactive systems and tools; HCI theory, concepts and models; • Computing methodologies → Machine learning.

Additional Key Words and Phrases: Interactive Machine Learning, Machine Teaching, Architectural Model, Toolkit.

INTRODUCTION

Machine Learning (ML) system development is still a matter for specialists. App designers struggle to include ML in their design practice. Domain experts, such as clinicians, have little control over the development of the systems Françoise, Caramiaux, Sanchez providing them assistance in decision making. Small businesses developing ML solutions may find it difficult to provide their clients with adaptable tools for calibrating and evaluating models on real-world data. With the growing use of machine-learned models in user-facing applications, it is essential to include a broad spectrum of stakeholders in the process of building and assessing such systems.

Interactive Machine Learning (IML) is a paradigm engaging people in a tight interaction loop with a learning algorithm [START_REF] Amershi | Power to the people: The role of humans in interactive machine learning[END_REF][START_REF] Dudley | A Review of User Interface Design for Interactive Machine Learning[END_REF][START_REF] Fails | Interactive Machine Learning[END_REF][START_REF] Gillies | Human-Centred Machine Learning[END_REF]. In this approach, users refine a model through an iterative process involving several activities, from data acquisition to model training, testing and deployment [START_REF] Dudley | A Review of User Interface Design for Interactive Machine Learning[END_REF]. Some of the benefits of IML include helping users to efficiently convey expert knowledge to the system [START_REF] Holzinger | Interactive machine learning for health informatics: when do we need the human-in-the-loop?[END_REF], allowing system adaptation to a user's specific needs [START_REF] Fiebrink | Human Model Evaluation in Interactive Supervised Learning[END_REF],

enabling ML as design material [START_REF] Dove | UX Design Innovation: Challenges for Working with Machine Learning as a Design Material[END_REF], or helping users to better understand caveats and strengths of a ML pipeline [START_REF] Hohman | Understanding and Visualizing Data Iteration in Machine Learning[END_REF].

IML research is burgeoning: it has given rise to novel interaction techniques, original visualisation strategies, as well as important guidelines for designing IML systems and Human-AI Interactions (HAI) [START_REF] Amershi | Guidelines for Human-AI Interaction[END_REF][START_REF] Corbett | Interactive Machine Learning Heuristics[END_REF][START_REF] Nodalo | On Building Design Guidelines for An Interactive Machine Learning Sandbox Application[END_REF][START_REF] Wall | Using Expert Patterns in Assisted Interactive Machine Learning: A Study in Machine Teaching[END_REF][START_REF] Yang | Re-Examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design[END_REF]. Systems and studies explicitly adopting a human-centered perspective on ML illustrate the need to support diverse workflows in ML practice. Yet, IML software is often specialized for particular domains [START_REF]Runway | Make the Impossible[END_REF][START_REF] Berg | Ilastik: Interactive Machine Learning for (Bio)Image Analysis[END_REF][START_REF] Fiebrink | Human Model Evaluation in Interactive Supervised Learning[END_REF] or tasks [START_REF]Lobe | Machine Learning Made Easy[END_REF][START_REF] Carney | Teachable Machine: Approachable Web-Based Tool for Exploring Machine Learning Classification[END_REF][START_REF] Fails | Interactive Machine Learning[END_REF][START_REF] Fogarty | CueFlik: Interactive Concept Learning in Image Search[END_REF] and cannot readily be adapted to different practices. To our knowledge, IML is still missing general-purpose toolkits supporting the design of such systems, that would foster reuse and appropriation as well as collaboration between different stakeholders.

In this paper, we propose a software architecture for toolkits dedicated to the design of human interactions with ML systems. Our goal is to facilitate the development of such interactions for a broad range of users such as designers, domain experts, creatives, or ML researchers. In the proposed architecture, developers can compose interactive machine learning workflows by assembling components. Components provide graphical user interfaces that can be easily composed into custom layouts, and their reactive data streams can be composed into reactive pipelines that link user interactions with machine learning processes. This approach emphasizes flexibility in the definition of workflows, enabling rapid prototyping of pipelines and interfaces that can be tailored for a large diversity of end users.

Our contribution in this paper is twofold. First, we present an architectural model for composing interactions with ML systems. Second, we present an implementation of the model in the form of a programming toolkit, called Marcelle.

Marcelle provides a high-level JavaScript API allowing developers to quickly prototype and distribute interactive applications. We illustrate Marcelle with two implemented case studies: (1) a HCI researcher conducts user studies to understand how novices interact with ML, and (2) a ML expert and a clinician collaborate to develop a skin cancer diagnosis system.

COMPOSING INTERACTIONS WITH ML

Our vision of composable human-ML interactions relies on toolkits that are flexible, integrable in existing practices and supporting collaboration. Flexibility means that developers should be able to easily customize how they interact with elements of a ML pipeline (datasets, models, metrics, etc.) during development, and how end-users may interact with these elements. The need for flexibility concerns both workflows, that structure activities and their relationships, and specific interaction techniques, where a careful design of interfaces is essential to provide appropriate feedback and interaction mechanisms. Such tools should be seamlessly integrated in the existing practices of developers and end-users. Typically, tools should be easily shared and integrated in diverse practices, from ML experts developing models in Python using dedicated ML libraries to domain experts with context-specific constraints (e.g. using the system in a medical environment). The development process should support collaboration between several stakeholders, for example different developers, designers and end-users. w o r k i n g d r a f t .

N o t f o r d i s t r i b u t i o n .

Marcelle: Composing Interactive Machine Learning Workflows and Interfaces

We propose an architectural model to achieve this objective for the design of human interactions with machine learning. The architecture is built upon a modular collection of interactive machine learning components with a unified interface, that can be composed to form custom processing pipelines and user interfaces. This component-based architecture is extensible and facilitates reuse of interaction techniques across projects. The architecture is built over web technologies to facilitate collaboration, and supports sharing of applications, data and models.

To motivate this approach, we describe two case studies. The first case study originates in our own research experience [START_REF] Sanchez | How Do People Train a Machine? Strategies and (Mis)Understandings[END_REF]. The second case study stems from recent findings on human-ML collaboration in the medical field [START_REF] Tschandl | Human-Computer Collaboration for Skin Cancer Recognition[END_REF].

The scenarios of the two case studies are depicted in Figures 2 and3, respectively.

Scenario 1 Studying Human Interaction with Machine Learning. Suzanne is an associate professor. Her research focuses on democratizing ML systems for the general public. To that end, she runs workshops and studies with playful scenarios where participants can teach concepts to a classifier and she collects data on user interactions. One of these scenarios involves input sketches drawn by people. Due to the sanitary restrictions, she needs to run her study remotely, while collecting data from participants.

She starts by prototyping a simple sketch-based classification workflow using a single JavaScript script. She instantiates a few standard components of the machine learning process: a dataset with a recording interface, a classifier, and a visualization module for its predictions. She reuses a sketchpad component from a previous project, and extends her toolkit by creating a component to visualize the model's uncertainty in order to help people to understand how well the classifier is trained. She composes these components together to form a processing pipeline, and assembles them in a graphical user interface. She shares her application as a simple web page in an online workshop with novices. Their feedback leads her to quickly update her script so that the pipeline and visual layout provides instantaneous feedback after each stroke and supports incremental learning. Her prototype now ready for the study, she sets up a data store to record the participants' sketches and models, along with logs of the interactions with the system.

Her experiments show that her design is successful in supporting ML understanding by novices. Later on, she starts collaborating with Shan, a designer, to deploy a version of the application in a science popularization exhibition. Shan proposes and implements a new visual interface using a web framework, which he later links with Suzanne's machine learning pipeline by removing the default app interface from her script, without affecting its functioning.

RELATED WORK

In this section we outline background work in IML, and we analyze existing systems and toolkits for ML and IML according to our design considerations highlighted in the scenarios of two case studies.

Background work in Interactive Machine Learning

Research in Interactive Machine Learning has highlighted the benefit of bringing human input in the training process of machine learning systems, leading to the development of original iterative workflows and interaction techniques [START_REF] Amershi | Power to the people: The role of humans in interactive machine learning[END_REF][START_REF] Dudley | A Review of User Interface Design for Interactive Machine Learning[END_REF].

IML workflows differ from conventional machine learning workflows in involving shorter cycles of data edition, training and evaluation. Such workflows have been shown to produce more robust models in diverse tasks such as image segmentation [START_REF] Fails | Interactive Machine Learning[END_REF], web image search [START_REF] Fogarty | CueFlik: Interactive Concept Learning in Image Search[END_REF], or text classification debugging [START_REF] Kulesza | Principles of Explanatory Debugging to Personalize Interactive Machine Learning[END_REF]. Aside from improving model performance, IML workflows foster exploration and discovery, which makes it attractive for creative and artistic practices where the learning process may become more important than the learned model itself [START_REF] Scurto | Designing Deep Reinforcement Learning for Human Parameter Exploration[END_REF]. Such an approach has been used in digital musical instrument design [START_REF] Caramiaux | Mapping through listening[END_REF][START_REF] Fiebrink | The Machine Learning Algorithm as Creative Musical Tool[END_REF][START_REF] Fiebrink | Human Model Evaluation in Interactive Supervised Learning[END_REF][START_REF] Françoise | Motion-Sound Mapping through Interaction: An Approach to User-Centered Design of Auditory Feedback Using Machine Learning[END_REF] and movement-based interaction design [START_REF] Gillies | Understanding the Role of Interactive Machine Learning in Movement Interaction Design[END_REF]. IML research also gave rise to original interaction techniques. Iterative and exploratory workflows make user's action history [START_REF] Hohman | Understanding and Visualizing Data Iteration in Machine Learning[END_REF] and model's state tracking [START_REF] Amershi | ModelTracker: Redesigning Performance Analysis Tools for Machine Learning[END_REF] particularly important in ML development. Building ensemble models interactively is faster and as robust as using optimization [START_REF] Talbot | EnsembleMatrix: Interactive Visualization to Support Machine Learning with Multiple Classifiers[END_REF]. Other works have investigated how interpretable feedback on predictions can help users understand and steer a model towards better performance [START_REF] Kulesza | Principles of Explanatory Debugging to Personalize Interactive Machine Learning[END_REF][START_REF] Spinner | explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning[END_REF]. We refer to Dudley and Kristensson [START_REF] Dudley | A Review of User Interface Design for Interactive Machine Learning[END_REF] for a comprehensive survey of interface design in IML.

Finally, previous works proposed guidelines to build better IML systems, where quality can be assessed in terms of model performance, user's understanding of the system, or user experience [START_REF] Amershi | Designing for Effective End-User Interaction with Machine Learning[END_REF]. Dudley and Kristensson [START_REF] Dudley | A Review of User Interface Design for Interactive Machine Learning[END_REF] identified 6 activities that have to be taken into account within an IML workflow from a behavioral perspective: (1) feature selection, (2) model selection, (3) model steering, (4) quality assessment, (5) termination assessment and (6) transfer.

While this list of activities covers the scope of a comprehensive IML workflow, many approaches address a single activity (or a subset of activities), and specific systems are developed in this objective. In the next section, we review a number of IML systems and tools that are publicly available.

Interactive Machine Learning Tools

Several tools following the IML paradigm have been proposed to assist in model development by non-experts. Teachable

Machine [START_REF] Carney | Teachable Machine: Approachable Web-Based Tool for Exploring Machine Learning Classification[END_REF] and Lobe [START_REF]Lobe | Machine Learning Made Easy[END_REF] are standalone applications offering a simple interface to create machine learning classifiers from examples. They require neither programming nor ML knowledge, and can be used as educational ressources [START_REF] Up | Bringing AI into the Classroom[END_REF].

U n p u b l i s h e d w o r k i n g d r a f t .

N o t f o r d i s t r i b u t i o n .

Françoise, Caramiaux, Sanchez

They are mostly dedicated to developers willing to integrate image classification in application. While they facilitate model development, assessment and transfer, they implement a fixed workflow and cannot be customized.

More modular IML tools have been proposed to support creative practices. RunwayML [START_REF]Runway | Make the Impossible[END_REF] presents itself as a Photoshop-like tool that integrates many state of the art deep learning models. To that extent, RunwayML is less of a toolkit than a complete software tool for ML-assisted creative practices. Wekinator [START_REF] Fiebrink | Human Model Evaluation in Interactive Supervised Learning[END_REF] is a standalone application dedicated to the design of new instruments and gesture-based interaction for music performance. It integrates generalpurpose models that can be used with a wide variety of inputs and outputs. However, it provides limited visualization and fixed workflows. There exist libraries integrating ML algorithms into computer music programming platforms such as Cycling'74 Max [START_REF][END_REF], including either general-purpose models as in ml.lib [START_REF] Bullock | Ml.Lib: Robust, Cross-Platform, Open-Source Machine Learning for Max and Pure Data[END_REF] or specialized techniques as in XMM [START_REF] Françoise | Probabilistic Models for Designing Motion and Sound Relationships[END_REF]. InteractML [START_REF] Gonzalez Diaz | Interactive Machine Learning for More Expressive Game Interactions[END_REF] is a visual programming extension for Unity3D dedicated to game developers and designers willing to explore embodied interactions. Its modular architecture facilitates the creation of custom workflows for end-users. Finally, the RapidMix API [START_REF] Bernardo | Designing and Evaluating the Usability of a Machine Learning API for Rapid Prototyping Music Technology[END_REF] is a JS library for rapid prototyping of creative applications embedding machine learning methods. The API allows for creating custom ML pipelines in the browser but it does not allow for composing user interfaces.

Even though there are many examples of tools with diverse workflows integrating specific interaction techniques, to the best of our knowledge no general-purpose toolkit exists for building standalone IML applications. Developers must dedicate significant resource to developing systems using machine learning frameworks and visualization tools.

Machine Learning Frameworks

A multitude of modular frameworks has emerged in the machine learning community to make ML more accessible for developers and data scientists. Available as libraries, these tools provide a high-level API for training and evaluating ML models, either with general-purpose algorithms as in Scikit-Learn [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF], or specialized for deep learning, for instance with Tensorflow [START_REF] Abadi | TensorFlow: A System for Large-Scale Machine Learning[END_REF] or Pytorch [START_REF] Paszke | PyTorch: An Imperative Style, High-Performance Deep Learning Library[END_REF]. These libraries still require ML expertise in addition to programming knowledge, and focus on creating training and evaluation pipelines with limited interactivity.

There has been a recent effort to bring ML to web browsers through JavaScript libraries. ONNX.js [START_REF]ONNX.js: run ONNX models using JavaScript[END_REF] is a JavaScript library for running inference on Open Neural Network Exchange (ONNX) models in web browsers. The Tensorflow.js [START_REF] Smilkov | TensorFlow.js: Machine Learning for the Web and Beyond[END_REF] library enables inference and training of neural networks in web browsers. Its API is close to Keras [START_REF]Keras: The Python Deep Learning API[END_REF], and it provides compatibility with models trained with Python. Other libraries such as ml.js [START_REF]ml.js: Machine learning tools in JavaScript[END_REF] and ml5 [START_REF]ml5js: Friendly Machine Learning For The Web[END_REF] were designed to provide a high-level API for developers and designers. While these tools can power the development of interactive machine learning systems, they do not explicitly support the design of interactions with ML pipelines.

Finally, there also exist toolkits for data science emphasizing the creation of custom workflows for data analysis tasks.

Orange [START_REF] Demšar | Orange: Data Mining Toolbox in Python[END_REF] and RapidMiner [START_REF] Hofmann | RapidMiner: Data Mining Use Cases and Business Analytics Applications[END_REF] provide a visual programming environment that allows users to create data analysis pipelines. However, their focus is on data analysis rather model refinement, and they are not designed to enable custom end-user interaction.

Visualization Toolkits for Machine Learning

With the rising interest for visual analytics as a support for ML practice [70], a number of visualization toolkits have been proposed to support model development, debugging and understanding. Examples include Tensorboard [START_REF] Wongsuphasawat | Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow[END_REF],

ClearML [START_REF] Allegro | ClearML | MLOps for Data Science Teams[END_REF], Neptune.ai [START_REF]Neptune.ai | Metadata Store for MLOps[END_REF], VisualDL [START_REF]VisualDL: Deep Learning Visualization Toolkit[END_REF] or Visdom [START_REF]GitHub -fossasia/visdom: A flexible tool for creating, organizing, and sharing visualizations of live, rich data[END_REF]. Tensorboard [START_REF] Wongsuphasawat | Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow[END_REF] is a visualization toolkit for Tensorflow, running in the browser, dedicated to facilitate machine learning experimentation. Its modular architecture provides flexibility regarding the information to display, that includes metrics, model graphs and assets. Notebook environments w o r k i n g d r a f t .

N o t f o r d i s t r i b u t i o n .

Marcelle: Composing Interactive Machine Learning Workflows and Interfaces such as Jupyter [START_REF] Kluyver | Jupyter Notebooks ? a publishing format for reproducible computational workflows[END_REF] provide literate programming environments emphasizing narratives. Notebooks support a wide range of visualization libraries, are easily shared, and therefore foster reuse and appropriation. Tensorwatch [START_REF] Shah | A System for Real-Time Interactive Analysis of Deep Learning Training[END_REF] emphasizes real-time monitoring in Python notebooks. It allows users to query about the training process and produce real-time visualisation. While these tools bring interactivity to the workflows of ML practitioners, they are dedicated to ML monitoring and assessment. Their focus is on computation, whereas our approach emphasizes direct interaction for a larger range of activities. Moreover, they do not allow the development of standalone applications.

Summary. While there exist toolkits for machine learning and visualization, interactive machine learning systems are often domain-specific. To our knowledge, general-purpose toolkits for designing IML applications are still missing, along with defining principles for their architecture.

DESIGN PRINCIPLES

We present 5 design principles that define an architecture model enabling composing interactions with ML pipelines, as described in Section 2. The proposed architecture is illustrated in Figure 4. The implementation in Marcelle presented in Section 5 follows these principles. Fig. 4. The proposed architectural model relies on loosely coupled components (dark blue boxes) embedding data, processing and interaction. Components expose event streams that serve as interface to build reactive pipelines (green arrows). Stream processing can be performed outside components (outlined boxes). Components optionally provide views that can be displayed on demand (right panel), offering users ways to interact with machine learning pipelines. These views can be visually composed through layout mechanisms (right panel). Data stores offer persistent storage to components and enable bidirectional communication with Python.

Component-based Architecture

Building interactive ML application requires assembling interactions to facilitate the manipulation of machine learning concepts and objects. These objects of interest are highly heterogeneous in nature and relate to various activities, such as the ones reported by Dudley and Kristensson [START_REF] Dudley | A Review of User Interface Design for Interactive Machine Learning[END_REF]: feature selection, model selection, model steering, quality assessment, termination assessment and transfer (i.e. deployment). Depending on the activity, users might need to operate upon various objects: data, algorithms, parameters, models, predictions, explanations, etc. In addition, as the needs of users can be dramatically different, their actions upon the ML pipeline need to be supported by a custom arrangement of interfaces that facilitate the manipulation of these objects.

U n p u b l i s h e d

w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n . Françoise, Caramiaux, Sanchez
We propose a modular architecture where the building blocks for designing interactions are components embedding data, computation, and interaction. Components enable granular interaction with specific elements of the ML pipeline, 1and can be flexibly composed to form higher-level interactions and workflows that support the aforementioned

Composable Interfaces

Workflows encompass two main facets: the specification of reactive pipelines describing the relationships between various objects and actions, as described in the previous section, and the visual arrangement of components in the end-user interface. In their review of user interface design for IML, Dudley and Kristensson [START_REF] Dudley | A Review of User Interface Design for Interactive Machine Learning[END_REF] underline that while there exist common elements, the design of IML interfaces vary considerably according to the data and application. The proposed component specification includes an optional view allowing for visualization and enabling user interactions.

However, the way views are used in an application (e.g. where and when they are displayed) should not affect the logical relationships between components, that are defined using reactive pipelines.

We propose that components should be displayed on demand. In practice, this means that components' computation should not be affected when their view is hidden (besides preventing from user interactions), and that multiple views

Data Persistence and Communication

In a collaborative scenario involving users with diverse levels of expertise in ML, it is essential that the objects of various types contained in the application are shared among collaborators (datasets, annotations, models, predictions, logs, etc.). We propose a design that supports flexible data persistence and sharing, allowing developers to specify where to store and how to share various parts of an application's state, including training or test data, models, logs, etc. We propose the use of data stores, accessible to components, that are generic enough to store heterogeneous data describing the application's state. Data stores should provide a unified interface while allowing data storage in diverse locations, including remote servers and browsers' web storage.

Interoperability with ML Libraries

Machine learning practice now relies on a key set of programming languages and libraries that are widely used among researchers and engineers, as reported in Section 3.3. Among them, Python is particularly popular, with libraries such as Scikit-Learn [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF], Tensorflow [START_REF] Abadi | TensorFlow: A System for Large-Scale Machine Learning[END_REF] or Pytorch [START_REF] Paszke | PyTorch: An Imperative Style, High-Performance Deep Learning Library[END_REF], to name a few. The architecture needs to provide interoperability with machine learning frameworks in order to be used by machine learning experts. It is essential to provide an interface to communicate data and models between Python programs and components. For instance, enabling access to data stores from third-party programs would help creating bridges between programming environments. We suggest that architectures supporting interoperability should seamlessly share data and models, but also computational pipelines, including, for instance, feature extraction and preprocessing.

IMPLEMENTATION

Marcelle is an IML toolkit that implements the design principles defined in the previous section. Marcelle is distributed as open-source software and available online. 2 The toolkit, written in TypeScript, is distributed as a client-side JavaScript library that can be integrated in web applications or used in standalone. The core library (marcellejs/core) is composed of a set of definitions specifying a concrete implementation of the architecture model. It also provides a set of standard components, a flexible data store system, and two mechanisms for interface composition. Marcelle also provides a server package for persistent data storage, a command-line interface for project generation, and a Python package allowing communication with Tensorflow. The overall principle of the library is illustrated in Figure 1 and its architecture is depicted in Figure 4.

IML Components

Components are the building blocks of Marcelle applications. They embed the state, logic and interaction for particular tasks. They implement a minimal interface enabling visualization and communication with other components. A component is essentially a JavaScript object that (1) exposes a set of reactive streams that can be processed by other components (as described in Section 5.2), and (2) provides methods to display the component's graphical user interface in the DOM.

Components are versatile in scope and can address a large variety of tasks. We classified standard components into 5 categories:

Input components address data acquisition from different sources. For images, this includes recording images from a webcam, from file upload, or from a drawing canvas. Françoise, Caramiaux, Sanchez

Data management components concern data storage, processing and visualization. This includes datasets for storing training or test instances, predictions, or arbitrary logged data.

Models are special components implementing specific machine learning models. They implement a specialized interface for training, inference and import/export to files and data stores. Marcelle currently integrates models from the Tensorflow.js [START_REF] Smilkov | TensorFlow.js: Machine Learning for the Web and Beyond[END_REF] library, but other JavaScript libraries can easily be used.

Visualization tools concern interactive visualization of various objects in the pipeline, such as datasets, model parameters, training progress, or predictions.

Widgets are standard GUI widgets (buttons, menus, input fields, etc.) that are necessary to compose custom user interfaces.

Components often provide a graphical user interface, or view, that can be displayed on demand in a web application, using .mount() and .destroy() methods. Examples of views of components are included in the dashboards presented in Figure 1. Views only communicate with the component using streams: they are reactive to changes but can also push events into the component's streams. This mechanism provides a clear separation between the view and the component's processing. In other words, a component remains functional in a given pipeline even if its view is not displayed, since it is possible to imperatively put events into any component's streams.

We use Svelte [START_REF]Svelte: Cybernetically enhanced web apps[END_REF] to power the views in Marcelle's component library. Svelte is a compile-time framework with native support for reactive streams. However, Marcelle does not enforce the use of any framework for creating views of custom components. Creating custom components only requires creating JavaScript objects that expose streams and provide a .mount() method to display the view. A developers is free to use Svelte or any other framework to generate views and can use any pattern for generating components, including object literals, factory functions or classes.

Reactive Pipelines

Reactive pipelines are concerned with the specification of the relationships between user actions and machine learning objects. We propose a distributed model that gives developers explicit control over the information flow, offering flexibility for building applications where user actions must trigger complex sequences of operations. Our approach relies on reactive programming [START_REF] Bainomugisha | A Survey on Reactive Programming[END_REF], a paradigm that is well-suited for the development of IML applications, that are essentially event-driven. It facilitates the creation, filtering, transformation and consumption of asynchronous data streams that propagate changes over the pipeline. Reactive programming enables encapsulated, loosely coupled modules where event streams act as an interface. By avoiding callbacks, it provides more readable and maintainable code.

Components achieve a large variety of tasks, and, from a reactive programming perspective, can broadly be classified into 3 categories:

Sources are components producing data. Examples of source components include GUI widgets and data input tools.

Sinks are components that respond to event streams without necessarily producing new events, and are mostly used to provide users with feedback.

Processors are both sinks and sources: they rely on input events to produce a set of output streams. This is the broadest category of components and includes feature extractors, models, or datasets.

Our implementation of reactive streams relies on Most.js [START_REF]Most.js: Monadic Event Stream[END_REF], a high performance reactive programming library.

Components expose streams as properties whose name is prefixed with a dollar sign by convention. For instance, a U n p u b l i s h e d w o r k i n g d r a f t .

N o t f o r d i s t r i b u t i o n .

Marcelle: Composing Interactive Machine Learning Workflows and Interfaces webcam component exposes several streams, including a boolean stream called $active specifying whether the webcam is turned on, and a periodic stream called $images containing images sampled from the webcam in ImageData format.

Dashboard and Wizard Layouts

Since components provide their own views, creating user interfaces that are tailored for a particular application or user is straightforward. Developers can mount any component to a given element in the DOM. To simplify interface design, Marcelle provides two high-level mechanisms for building user interfaces: Dashboards and Wizards.

Dashboards.

Dashboards provide a way to create applications with multiple pages that display collections of components. Dashboards provide an interface similar to Tensorboard [START_REF] Wongsuphasawat | Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow[END_REF]. Dashboards are instantiated using the dashboard factory function. They are composed of pages, instantiated with the .page() method, that can display any component instance passed to their .use() method. Dashboards are hidden by default and can be displayed on demand as a full-screen overlay over the current web page. In order to avoid disturbing the host application when displayed on demand, we use hash-based routing for navigating between the various pages. Navigating to a page will mount its associated components' views. Dashboards also expose streams containing events describing their state: $visible specifies whether the dashboard is currently visible, and $page is mapped to the current page.

Wizards.

Wizards are dedicated to the creation of walk-through guides for beginners or end-users. Wizards are inspired by Teachable machine's training wizard3 that walks users through training their machine learning model.

Marcelle wizards are flexible and allow developers to specify what components should be displayed at every step. Our implementation of wizards is similar to that of dashboards: they are instantiated with a factory function and contain a number of panels. Panels are created using the .page() method, and contain a title, a textual description and a set of components to display. Wizards are also displayed on demand in a modal window. They expose a stream called $current synchronized with the index of the current panel.

Data Stores

While reactive programming facilitates real-time data communication, most scenarios also require data persistence so that parts of the application's state are stored. For instance, the training data provided by the user should persist, even when changes to the pipeline are made. Marcelle provides flexible data stores that can be instantiated with various backends: data can be stored in the browser's local storage or on a remote server. Choosing the backend location only requires passing a URL to the data store. Developers can create different backends to customize where different objects are stored. Some components rely on a data store -for instance, the dataset component that needs to store instances, -however data collections can be created on the fly to store custom information when relevant. This is particularly useful to store some of the state of the application (for instance the model's parameters) or session logs of the user's interactions.

Marcelle's backend package (marcellejs/backend) provides a simple Node.js server that can be easily configured to use either NeDb or MongoDb databases for storage, with optional authentication. We use the Feathers Françoise, Caramiaux, Sanchez flexibly handle custom data structures. Feathers provides real-time updates, enabling components to react to changes in data stores, even if they come from other clients.

Python Integration

Training and running inference on ML models in web browsers is possible with dedicated JavaScript libraries. Using Tensorflow.js [START_REF] Smilkov | TensorFlow.js: Machine Learning for the Web and Beyond[END_REF], it is possible to make real-time predictions with potentially large models with several million parameters. Marcelle's dataset architecture is optimized for training, using asynchronous iterators that can stream and process data lazily. Yet, the limited computing power of current web browsers harms scaling to larger datasets and models.

Marcelle partially supports interoperability with standard machine learning frameworks in Python. This gives Marcelle the capacity to scale easily according to the developer's computing resources. The marcelle Python package can be used to interact with a backend server, with read and write access to data stores. Our implementation currently supports Tensorflow, a common Python library for Deep Learning development. Complex neural network models can therefore be trained with large datasets from Python. Data can be logged to the backend during training, along with model checkpoints stored in Tensorflow.js format, and additional generic assets such as images or audio files (according to the type of data involved in modeling). For Keras, we provide a callback object that can simply be passed to a model's

.fit() method. For custom training loops in Tensorflow, we provide a Writer where logging can be done imperatively.

Marcelle components can then easily access the logged data in the data store, for instance to plot loss and accuracy curves, or to load models at various checkpoints to perform inference in an interactive application.

Command-Line Interface

Marcelle comes with a Command-Line Interface (CLI) to generate new projects, custom components and backends.

We use Vite [START_REF]Vite: Next Generation Frontend Tooling[END_REF] as a default build tool for projects. Vite is easily configurable, supports multi-page applications and provides a very fast development server with hot module reloading, which is convenient for prototyping. Additionally, the CLI can generate new components and backend services for existing projects.

CASE STUDY 1

This section describes how Marcelle can be used to implement Scenario 1, introduced in Section 2, and discusses the benefits of the architecture to support this task. A running demo of the case study is available online4 .

6.1 Implementing the Scenario feature extractor. The resulting $features stream is then fed to the classifier to create a stream of predictions that can be visualized. The initial prototype for the sketch application is shown in Figure 5.

6.1.3 Improved Workflow. After sharing her application as a web page and gathering preliminary feedback about the system, Suzanne chooses to improve the workflow by providing instantaneous feedback after each stroke drawn by end-users and by supporting incremental learning. In order to do this, Suzanne only has to edit the script by adapting how the prediction stream is built. Concretely, predictions should be updated at each stroke on the sketchpad, but also when the model is updated after training. The predicted label should also pre-fill the label input to facilitate data annotation. As an example, the updated prediction pipeline is reported in Listing 2b. Finally, she rearranges the layout of the components in the dashboard.

Finally, Suzanne can deploy her app online and log data on a dedicated server by changing the data store's location from localStorage to the URL of a remote server. She uses the CLI to generate this server. .map((imgData) => featureExtractor.process(imgData))

.awaitPromises(); const $predictions = $features .map((features) => classifier.predict(features))

.awaitPromises(); const predictionViz = confidencePlot($predictions);

(a) Initial workflow: predictions are only computed when the user requires them.

const $features = input.$images .map((imgData) => featureExtractor.process(imgData)) .awaitPromises(); const $trainingSuccess = classifier.$training .filter((x) => x.status === 'success'); const $predictions = $features .merge($trainingSuccess.sample($features))

.map((features) => classifier.predict(features)) .awaitPromises(); const predictionViz = confidencePlot($predictions); $predictions.subscribe(({ label }) => { classLabel.$text.set(label); });

(b) Improved workflow: predictions are computed at every stroke and when the model is updated.

Listing 2. Programming pipelines with reactive streams. In this example, a stream of predictions is built by passing input images to a feature extractor and a classifier.

Discussion

This scenario is largely inspired from the authors' own research experience [START_REF] Sanchez | How Do People Train a Machine? Strategies and (Mis)Understandings[END_REF]. We developed a similar application for a recent research study and initiated a collaboration for an exhibition dedicated to the general public. Here, we discuss Françoise, Caramiaux, Sanchez 6.2.1 Rapid iterative prototyping for IML experimental studies. Marcelle is particularly well suited to rapidly iterate on a design. While conducting this research project, we designed two versions of the app in less than a month of interval.

The first version was dedicated to a workshop held on the live-stream platform Twitch, and the second version to individual think-aloud sessions with participants. The first prototype was made to be used by many in a non-controlled experiment. The second prototype, however, was designed for a more controlled study. We included the usual elements of experimental design: instructions, a consent form or questionnaires. We also included a task where participants had to sort between images that have been well recognised or confused by the classifier. Dashboard pages were used to structure the different steps of a study. Instructions, forms and questionnaires were integrated using simple text components linking to external online forms. However, it would have been possible to create custom components for questionnaires with answers recorded in a data store.

In Marcelle, the combination of streams and data stores enables flexible logging mechanisms of participants' actions and application states. Any stream can be observed and recorded to a data store. For example, in our study we created additional datasets for recording every sketch drawn by participants, after every stroke drawn. We also logged, for each recorded drawing, the model prediction, and each action (for instance, adding a drawing to the dataset and training the model) with a time-stamp. 6.2.2 Share transparent and reproducible research. Marcelle applications run on the web and are easily shareable. Although online research studies are common, sharing ML models that can be trained by end-users in the browser remains generally challenging. This is particularly important at the time of writing, since the possibilities of conducting in-person studies are constrained by sanitary restrictions due to the pandemic crisis. With a help of a video-mediated communication tool, we conducted both a workshop and individual think-aloud study remotely during the first months of the pandemic [START_REF] Sanchez | How Do People Train a Machine? Strategies and (Mis)Understandings[END_REF]. Secondly, shareability makes the research process more transparent and reproducible: one can interact with the system used in an article, and another researcher could reuse and build upon the IML pipeline to conduct new research studies.

CASE STUDY 2

This section describes how Marcelle can be used to implement Scenario 2, introduced in Section 2, and discusses the benefits of the architecture for supporting collaboration between ML practitioners and domain experts. A running demo of the case study is available online6 .

Implementating the Scenario

Discussion

This scenario is inspired by recent publications emphasizing the potential of human-computer collaboration for medical diagnosis, for instance for skin cancer recognition [START_REF] Tschandl | Human-Computer Collaboration for Skin Cancer Recognition[END_REF]. While our implementation is a proof of concept, we discuss how Marcelle can support the work of ML experts and facilitate their collaboration with end users, in particular domain experts.

7.2.1 Interacting with Large Models. IML has historically been proposed as a way to involve the user in the model's training, and has therefore considered rather small datasets and shallow models. Interactively training deep models, on the contrary, is not realistic in many cases, and boils down to fine-tuning as in transfer learning. However, it is possible to directly interact with their predictions, which can help developers assess the behavior of models.

Marcelle is well integrated with Tensorflow.js, making it possible to load models trained with Tensorflow, a widely used ML framework, in order to perform inference in the browser. This enables both model developers and domain experts to interactively test how the model behaves, in particular with instances representing edge cases. Marcelle can easily incorporate new visualizations, and could support end-user interpretation of ML models using methods from the explainable ML literature [START_REF] Samek | Explainable AI: Interpreting, Explaining and Visualizing Deep Learning[END_REF][START_REF] Wang | Designing Theory-Driven User-Centric Explainable AI[END_REF].

Strengthening

Collaboration to Build More Reliable Models. In this medical scenario, Marcelle presents advantages for facilitating the communication between a model developer and domain experts. First, Marcelle supports an incremental approach where more elaborate visualization modules can be progressively integrated to match the user's needs. Second, data stores provide mechanisms to communicate data in both directions. For instance, the ML engineer can seamlessly update the model and synchronize it with the clinician's interface. In response, the clinician's interaction can also record data and annotations to serve as feedback to the model developer, effectively strengthening their collaboration. This collaborative authoring of ML models can have an important impact on ML transparency. There are many documented instances where intrinsically biased ML models have been harmful to individual and communities [START_REF] Cathy | Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy[END_REF].

Because these critical failures are not systematically handled by design [START_REF] Mitchell | Model Cards for Model Reporting[END_REF], it is essential to empower people with diverse areas of expertise to effectively understand, assess and act upon machine learning systems in various ways in order to foster accountability and fairness.

GENERAL DISCUSSION

We described Marcelle at both a conceptual and an implementation levels, and we showed how it can be used in two realistic use cases. In this section we discuss the practical use of Marcelle from personal experience, its limitations and the planned development roadmap.

Working with Marcelle

Working with Marcelle is straightforward for developers familiar with modern JavaScript and web development. From our experience using Marcelle, this was particularly handy to develop diverse activities using the toolkit and to easily Marcelle: Composing Interactive Machine Learning Workflows and Interfaces compatibility with OS). Installing the toolkit boils down to the installation of Node.js, available across OS. However, for accessibility purposes, we also deployed every example on the Glitch platform so that every student could access the example, remix the code and edit online7 . We found this process particularly efficient from a pedagogical perspective, for us and for the students. We believe that Marcelle can contribute to the recent pedagogical endeavor on ML education in providing educational resources to teach the general public about machine learning [START_REF] Kahng | GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation[END_REF][START_REF] Smilkov | Direct-Manipulation Visualization of Deep Networks[END_REF][START_REF] Wang | CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization[END_REF].

In parallel to the use of Marcelle for pedagogy, we also ran informal workshops with HCI researchers interested in using ML, and with ML researchers. During these workshops, we designed more advanced activities such as building custom components to illustrate the customization potential of the toolkit. Marcelle's architecture was designed with loosely coupled components to facilitate extension. Using the CLI, it is possible to generate component templates inside existing projects. Once mature, components or component collections can be published using the NPM registry, enabling other developers to integrate them in their applications. As an example, we implemented a custom component for point-based visualization using U-MAP [START_REF] Mcinnes | UMAP: Uniform Manifold Approximation and Projection[END_REF]. The code for the component is only 100 lines long and it is quick to implement thanks to seamless integration of third-party libraries 8 . Screenshots of applications created with Marcelle are available in Figure 7.

Workshop attendees were generally highly positive. For instance, a ML researcher working on image denoising expressed his need for interactive systems allowing him to explore how various models react to input noise and found in Marcelle an attractive solution. Another researcher related a current bottleneck in her collaboration with a radiologist that can be addressed with Marcelle: "the generation of image segmentation maps has to be done via a python script, which is not "user friendly" at all for radiologists, and even for me as an AI researcher, this is a bit tedious as I have to specify the input/output path, launch the script, then open the outputs in a software or in a Jupyter notebook. It is a real bottleneck for results analysis as this implies I have to generate the predictions on my own, then send to the radiologist the predictions, and this is a very inefficient workflow. "

During these workshops, we also asked what would be the aspects that would limit the use of Marcelle in their work.

To this question, some mentioned that Marcelle may not be suited to develop advanced tools dedicated to a specific task. One attendee took the example of image segmentation in 3D. She mentioned that having the possibility of rotating the volume in 3D, and having the individual input slices with the overlaid volume at the side could be hard to get, or a lot of implementation work. Another potential breakdown reported by attendees is the learning curve in handling JavaScript, which could spill over into their main activity as ML researchers. audio, text, time series and structured data, -developing components covering such a large spectrum is time-consuming for a small team of researchers. We plan to incrementally integrate support for other data types and tasks. Our choice of a minimalist interface for components is advantageous for integrating new data formats, input components, models and visualizations, as it does not require changes at the architecture level. However, this might lead to a large number of specialized components rather than a more limited set of components supporting polymorphism. Focusing on polymorphic components -for instance a generic prediction visualization that would provide appropriate feedback for classification, segmentation, or object detection results, -would facilitate the use of the toolkit by novices, but would hinder customization and appropriation.

CONCLUSION

We have presented Marcelle, a programming toolkit implementing our vision for composable interactions with machine learning. Through two case studies, we illustrated how Marcelle's component-based architecture and high-level API enable the efficient development of interfaces dedicated to various users: machine learning experts, designers, domain experts or the general public. Because components embed both processing and interaction, composing workflows boils down to (1) creating reactive pipelines linking user actions to machine learning tasks and (2) assembling components into user interfaces using flexible layouts. From our own experience, this approach makes it possible to quickly iterate over designs, and to reuse and share particular interaction techniques.

Finally, from a theoretical point of view, our conception of interactive machine learning components share similarities with the notion of information substrates [START_REF] Beaudouin-Lafon | Towards Unified Principles of Interaction[END_REF]. Substrates are computational medium that hold information, computation and interaction. They can be composed, extended or manipulated, and they transcend application boundaries. Our architecture provides a starting point for formalizing interactive machine learning substrates that would be shared and repurposed across users and applications. We see in this perspective a promising way to conceptualise our interactions with machine learning as design material, and to induce fundamental questions for the field of Human-Computer

1 U

 1 n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n .

  f o r d i s t r i b u t i o n .

Françoise

Fig. 2 .

 2 Fig. 2. Summary of the first scenario. The interfaces made in this scenario are dedicated to both remote study participants and exhibition visitors. The HCI researcher Suzanne developed the interface for remote study participants and the ML pipeline (dashed purple arrow). The designer developed the interface for exhibition visitors and reuses the ML pipeline made by Suzanne. Each application has their own data and models (green arrows).

U n p u b l i s h e d w o r kFig. 3 .

 3 Fig. 3. Summary of the second scenario. The two interfaces are dedicated to ML engineer Louise and the clinician Michel. Louise developed the ML pipeline and both interfaces (dashed purple arrows). Both dashboards share the same model and data (green arrows) to enable collaboration between Louise and Michel.

  could potentially be used to interact with the same component. Components should easily be displayed on their own in web applications, and higher-level composition mechanisms for generating layouts should be provided to further facilitate development. U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n . Marcelle: Composing Interactive Machine Learning Workflows and Interfaces

U n p u b l i s h e d w o r k

  i n g d r a f t . N o t f o r d i s t r i b u t i o n .

  [START_REF]Feathers | A framework for real-time applications and REST APIs[END_REF] framework for managing data stores, both on the server and client-side. In development mode, Feathers services are created dynamically when required by the client. They do not require the specification of a database schema, and can therefore U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n .

6. 1 . 1 Listing 1 .

 111 Setting up the Application Structure and Interface. Suzanne creates her application using Marcelle's CLI. Then she edits the src/index.js file and instantiates the necessary components from the marcellejs/core library: a sketchPad input, a mobileNet component to process input sketch images5 , a dataStore specifying where to keep the data, a dataset, a mlpClassifier (multi-layer perceptron classifier). These components are not interconnected yet and widgets are missing to enable the necessary user interactions. Suzanne needs a text input widget to enter the class labels and a button to add a sketch to the training set. She instantiates two new components: a textField and a U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n . Marcelle: Composing Interactive Machine Learning Workflows and Interfaces button. Then, she adds a second button to trigger model training. Finally, she adds a third button to trigger current instance prediction.To build her app's interface, Suzanne instantiates a dashboard. Using the .use() method, she adds the desired elements on the interface: the sketchpad, the text field, the two buttons. She also adds a datasetBrowser component to monitor the examples added to the training set. The resulting application is generated by a script containing about 20 lines of code, as reported in Listing 1. // Main components const input = sketchPad(); const featureExtractor = mobileNet(); const store = dataStore('localStorage'); const trainingSet = dataset('TrainingSet', store); const classifier = mlpClassifier({ layers: [64, 32], epochs: 20 }); // Additional widgets and visualizations const classLabel = textField(); const captureButton = button({ text: 'Capture this drawing' }); const trainButton = button({ text: 'Train the classifier' }); const predictButton = button({ text: 'Predict label' }); const trainingSetBrowser = datasetBrowser(trainingSet); // Dashboard definition const myDashboard = dashboard({ title: 'Sketch App', author: 'Suzanne' }); myDashboard .page('Main') .sidebar(input) .use([classLabel, captureButton], trainingSetBrowser) .use(trainButton, predictButton); myDashboard.start(); Script containing component definitions and interface composition for the interactive sketch recognition application. Components are instantiated using factory functions, and can be composed into a dashboard. 6.1.2 Connecting the Pipeline. Having the structure of her application, Suzanne creates the pipeline using the components' reactive streams. As an example, we report a simple prediction pipeline in Listing 2a. Here, Suzanne wants to plot the classifier predictions as a bar chart. She uses a classificationPlot component that takes a prediction stream as input. Using reactive programming operators, Suzanne creates a stream of predictions from the input sketches. She starts by sampling the $images stream of the sketchpad with the button $click stream, and passes the images to the U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n .

Françoise, Caramiaux, Sanchez Fig. 5 .

 Sanchez5 Fig. 5. Visual interface of the initial prototype for Case Study 1. This dashboard is generated by Listing 1.

6. 1 . 4

 14 Collaboration with a Designer. After successfully conducting the study, the application is reused as a installation in a museum exhibition for scientific popularization. Shan is included in the project and works on a new visual interface using the web framework he is familiar with. The separation between the ML pipeline and the visual interface in Marcelle facilitates the integration of the Shan's designed interface on top of the existing ML pipeline. U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n . Marcelle: Composing Interactive Machine Learning Workflows and Interfaces const $features = predictButton.$click .sample(input.$images)

  Marcelle's potential to support HCI research. U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n .

7. 1 . 1

 11 Integrating Marcelle in a Python Workflow. Louise is a ML engineer working with Keras, a deep learning library in Python. From the marcelle python library, she imports the KerasCallback class. She creates the callback and passes it to the fit method of her custom Keras model. Listing 3 depicts an example of use of the callback in a Python script.Louise specifies where to log the data by indicating the URL of a Marcelle backend, she also specifies the model formats and some training parameters. This data is progressively logged to the Marcelle remote data store during training.

7. 1 . 2 Listing 3 .

 123 Creating the ML Expert's Dashboard. After bootstrapping an application using Marcelle's CLI tool, Louise instantiates a few components in the main script. She uses a component called trainingHistory to access the log data of her training experiments in Python. She adds a select widget for select specific runs and a trainingPlot to U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n . Marcelle: Composing Interactive Machine Learning Workflows and Interfaces model.fit( Communicating with Marcelle data stores from Python is possible using Tensorflow or Keras. For Keras, Marcelle provides a simple callback to log values and record model checkpoints along the training. display loss and accuracy curves. To further inspect how her model behaves with specific instances, she creates a simple prediction pipeline combining a tfjsModel model to load a trained Keras model into the app, and a imageUpload component to upload images. Model checkpoints are loaded from the remote data store in the Tensorflow.js format, and inference runs in her web browser. Finally, by abstracting her prediction pipeline into a function, she sets up the side-by-side comparison of two model checkpoints. The resulting interface is shown in Figure 6 (left and middle panels).

Fig. 6 .

 6 Fig. 6. Dashboards for case study 2. The left and middle dashboards are designed for the ML expert, enabling her to interactively compare various training experiments. The dashboard on the right is designed for the clinician to run predictions and correct misclassifications on his own images.

7. 1 . 3 6 (

 136 Creating the Clinician's Dashboard. As Louise's recent results are promising, she reaches out to Michel to gather his feedback on the diagnosis performance. She copies her script to generate a simpler dashboard: she removes components related to monitoring the training and complements the prediction pipeline to gather feedback from Michel. She adds a new select menu and a button allowing the clinician to correct incorrect predictions, as shown in Figure right panel). She creates two datasets to separately record instances that are correctly or incorrectly classified. That way, Michel can analyze what images are misclassified. Finally, she uses a data store to synchronize the model used in both dashboards: from her developer dashboard, she can select a particular model version, that is then used by default in Michel's interface. U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n .

Françoise

  deploy the outcomes online. Here we provide examples stemming from the use of Marcelle in a pedagogical context and workshop activities. As part of Master's degree program, we developed a series of examples to introduce students to machine learning concepts through learning-by-doing. Using a standalone software could have brought accessibility issues (licence, U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n .

  f o r d i s t r i b u t i o n . U n p u b l i s h e d w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n .

Françoise

  f o r d i s t r i b u t i o n . w o r k i n g d r a f t . N o t f o r d i s t r i b u t i o n .

Françoise, Caramiaux, Sanchez [Fig. 7 .

 Sanchez7 Fig. 7. Screenshots of existing Marcelle applications and examples. (1-2) Wizard pages dedicated to capturing images from a webcam and training an image classifier. (3) Dashboard view of the image classifier training, including a visualization of the dataset and of training logs. (4) Picture of a user interacting with the final application, a web page integrating the classifier to run real-time predictions and play music loops accordingly. (5-6) Dashboard dedicated to training and interactively testing a flower classifier from the IRIS dataset. (7) Application running object detection from uploaded images using a pre-trained model. (8) Custom component implementing U-MAP visualization of an image dataset.

  Interoperability is key to foster Marcelle's usability. Current interoperability with common ML frameworks suffers several limitations. First, while Tensorflow.js provides good compatibility with models trained using Tensorflow's Python implementation, inference over models created with other frameworks such as Scikit-learn or Pytorch cannot be executed in web browsers. Marcelle currently integrates ONNX.js[START_REF]ONNX.js: run ONNX models using JavaScript[END_REF], a JavaScript library for running Open Neural Network Exchange (ONNX) models in web browsers, which can only be used for inference. However, at the time of this writing, ONNX.js runtime only supports a limited set of operators used in deep learning models. Based on preliminary prototypes, we plan to develop Python libraries enabling real-time communication over WebSocket with inference pipelines running server-side. , Caramiaux, SanchezSecond, Marcelle provides functions to log data from a Python script during training, however we do not yet support consistent synchronization of data and models between Marcelle and Python. In particular, we plan to further integrate data stores within Python frameworks, by providing appropriate Python interface to use data store objects within Python scripts. As Marcelle is currently compatible with Tensorflow.js, an option would be to provide custom Tensorflow datasets.Finally, we plan to provide better integration for Marcelle in notebook environments such as Jupyter by embedding Marcelle's components into notebook cells. Embedding Marcelle into such widely used programming tool could potentially increase adoption, further facilitate collaboration and diversify workflows with ML pipelines. 8.2.2 Data Types and Tasks. Marcelle currently provides good support for image processing tasks, in particular classification and segmentation. While we designed its architecture to handle arbitrary data types -including images,

	d r a f t .
	w o r k i n g
	8.2 Limitations and Future Work 8.2.1 Interoperability. U n p u b l i s h e d

N o t f o r d i s t r i b u t i o n .

Françoise

In our context, granularity means the different level at which the user can interact with the system: from high level class names in a classification task, to the low-level number of neurons in a specific layer of the network for example.

https://marcelle.dev/

Online demo: https://glitch.com/~tm-wizard

https://uist2021demos.marcelle.dev

This follows a common transfer learning strategy (for more details on this approach, see for instance[START_REF] Yosinski | How Transferable Are Features in Deep Neural Networks?[END_REF]). Mobilenet is commonly used model for pre-training[START_REF] Howard | MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications[END_REF].

https://uist2021demos.marcelle.dev

Some examples are available at at: https://glitch.com/@marcelle.crew/marcelle-examples

A demo of the UMAP component is available online: https://demos.marcelle.dev/umap/

ACKNOWLEDGEMENTS

This research was supported by the ELEMENT project (ANR-18-CE33-0002) from the French National Research Agency.

We want to acknowledge and thank everyone who was involved in each stage of the research, in particular the anonymous reviewers, students of the Interactive Machine Learning course in Université Paris-Saclay and workshop participants. We especially want to express our sincere gratitude to Frédéric Bevilacqua, Alessandro Delmonte, Clara Rigaud, Yvonne Jansen, Antoine Loriette, and Benjamin Matuszewski.