Pierre Dragicevic
email: pierre.dragicevic@inria.fr

Yvonne Jansen
email: yvonne.jansen@cnrs.fr

Andrew Vande Moere
email: andrew.vandemoere@kuleuven.be

Data Physicalization

Keywords: Data physicalization, physical visualizations, information visualization, tangible user interfaces, data sculptures, tangible thinking, making processes

Data physicalization is a rich and vast research area that studies the use of physical artifacts to convey data. It overlaps with a number of research areas including information visualization, scientific visualization, visual analytics, tangible user interfaces, shape-changing interfaces, personal fabrication interfaces, as well as graphic design, architecture, and art. This chapter surveys academic work on data physicalization up to and also provides a broad overview of nonacademic work. It discusses how data physicalization has been used for analytical purposes, communication and education, accessibility, self-reflection and self-expression, and finally for enjoyment and meaning. It also discusses enabling technologies, reviews empirical studies, and surveys models and theories of data physicalization.

Table of Contents Introduction

The discipline and practice of visualization aims to augment the human understanding of data. The primary challenge of visualization design consists of developing techniques that turn abstract data into easily perceivable and interpretable representations. Computer displays have demonstrated several clear strengths for this purpose, including their ability to convey visual imagery that is extremely rich in details, and their ability to let users explore data by dynamically altering its visual representation [ , , ]. Meanwhile, advances in tangible computing have illustrated how humans can interact with digital information by using their natural ability to perceive and manipulate physical objects and materials [ , ]. As such, a research area has emerged that questions why the display of, and interaction with, data should remain limited by the constraints of pixel matrices, two-dimensional gestures and keyboards buttons, and whether moving data representations from flat displays into the physical world could create novel and useful ways of exploring, experiencing and communicating data [ , ]. In this chapter we refer to a data physicalization (or simply a physicalization) as a "physical artifact whose geometry or material properties encode data" [ ]. Thus the artifact must be physical, and must encode data. In particular, there is a conceptual distinction between a physical model and a data physicalization: a scale model such as a solid terrain model, an architecture model, or a figurative sculpture are not data physicalizations, unless part of their shape or appearance conveys abstract data as it is sometimes the case for scientific visualization [ ]. Such pure physical models will not be covered in this chapter other than occasionally, in order to inform discussions on data physicalization.

Several terms exist that are closely related to data physicalization. For instance, physical visualization is a synonym for data physicalization [ ]. Data sculpture was one of the first terms that have characterized this emerging domain, yet the term generally denotes data physicalizations that aim for more artistic expressions of data in physical form [ , ]. In this chapter, we discuss data physicalization irrespective of its function and its creator's intent.

Data physicalization has been further defined as a research area which "examines how computer-supported, physical representations of data (i.e., physicalizations), can support cognition, communication, learning, problem solving, and decision making" [ ]. This chapter provides an overview of this research area. Although the research area as we define it refers to the support of computers, this chapter also discusses manually-crafted physicalizations, as they are often relevant for informing computer-supported physicalization. Furthermore, computer support can take various forms, such as the modelling and the digital fabrication of physicalizations, or their dynamic actuation during use.

Data physicalization is closely related to several research areas within and around HCI [ ]. The most closely connected area is visualization, which we use here as an umbrella term to capture data visualization, information visualization, scientific visualization and visual analytics. The general goal of visualization is to study "the use of computer supported, interactive, visual representations of abstract data to amplify cognition" [ ]. Data physicalization shares many research questions with visualization, but with an explicit focus on the physical, tangible and material character of physical data representations. Data physicalization is also closely related to the area of tangible interaction, whose goal is to study interactive systems that "give physical form to digital information, employing physical artifacts both as representations and controls for computational media" [ ]. Other areas within HCI that are connected to data physicalization include ambient displays [ ], shape-changing interfaces [ , ] and personal fabrication [ ]. Despite overlaps in terms of goals and methods, data physicalization differs from these aforementioned areas in that it exclusively focuses on data-driven tasks, such as data exploration and data communication. Finally, data physicalization and the production of data sculptures in particular have tight connections with more artistically-driven disciplines, such as graphic design, architecture, and installation art, among many others [ , ].

The structure of this chapter is loosely inspired by Munzner's multi-level typology of abstract visualization tasks [ ], which distinguishes three main motivations for creating and using visualizations: to discover, to present, and to enjoy. Similarly, we start by discussing the use of physicalization for analytical purposes, particularly in science and engineering (section ). We then discuss how data physicalizations have been used for communication and education (section ), for supporting accessibility to blind and visually-impaired users (section ), for self-reflection and self-expression (section ), and finally for enjoyment and meaning-making (section ). We complete this review by discussing recent and emerging technologies enabling different forms of physicalizations (section ), the different empirical studies that have been conducted on data physicalization (section ), and finally existing models and theories of data physicalization (section ). In several places, we will mention historical data physicalizations that have been in use before graphical computers existed, because they are useful to provide context and can serve as a source of inspiration for novel designs.

Physicalization for Analytics

One of the main purposes of visualization is to help people solve problems by making it easier to reason with or about data: much of visualization research focuses on the pragmatic aspects of data visualization [ ], and on helping people carry out useful tasks: "computer-based visualization systems provide visual representations of datasets designed to help people carry out tasks more effectively" [ ]. This has also been a major goal of early data physicalizations. Here we discuss examples of uses of data physicalization for analytical purposes, before and after computers were available. A number of scientists from the -th centuries crafted physical, threedimensional representations of data as part of their research. For example, in the s, the physicist James Maxwell created plaster models of three-dimensional thermodynamic functions that helped him and Josiah Willard Gibbs understand and develop the thermodynamic theory of state [ ] (see Figure -a). The construction of solid D molecular models also played an important role in many major discoveries in chemistry, including in Watson and Crick's discovery of the helix structure of DNA in [ ] and John Kendrew's discovery of the structure of myoglobin in [ ] (see Figure -b). All three chemists received a Nobel prize for their discoveries.

During this time period, physical D data representations were also in use in industry. In the s, human factors pioneer Frank Gilbreth created a number of wire-based representations of factory worker hand movements in D space, in order to better understand how to optimize these movements [ ] (see Figure a). In the s, several American electricity companies were building and maintaining solid D visualizations of electricity consumption, presumably to better anticipate power demands [ ] (see Figure -B).

All these D data representations were very time-consuming to build and with the advent of graphical computers, many of them were eventually superseded by virtual D models and visualizations [ , ]. However, physical data representations never stopped being used, and they are becoming more and more prevalent as they are easier and easier to build. These research labs saw benefits in complementing screen-based D renderings with solid models. For example, biochemists working with the SDSC TeleManufacturing Facility mentioned they got insights that they could not get from the screen-based D models alone, and concluded that "modern physical models are .

Dynamic Physicalization for Analytics

A clear limitation of most D-printed data physicalizations is that they are static: they cannot be updated and cannot update themselves. There are however several ways this limitation can be overcome. One simple (albeit cumbersome) way is to use construction blocks that allow physicalizations to be updated at will. For example, business executives from General Motors and WellStar Health Systems use LEGOs to help them log, update, and explore data [ ].

Meanwhile, there is ongoing research in trying to make data physicalizations that can update themselves. One simple approach to add dynamicity to physical artefacts, which has been commonly used in research on tangible user interfaces (TUI), is to combine physical objects with virtual displays [ ]. For example, the Scripps Research Institute has extensively experimented with adding augmented reality overlays to solid molecule models for the purposes of chemistry research and teaching [ , ]. With such systems, users can explore information about a molecule by holding a physical model and switching between different overlay representations (Figure -a). Such "hybrid" physical/virtual data representations have also been explored for industrial applications. For example, Planwell is an experimental system for petroleum well planning and drilling that uses a solid terrain model with overlaid visualizations in order to facilitate data exchange and collaboration between distributed teams [ ] (Figure -b).

One common limitation of hybrid representations, however, is that most of the data is typically conveyed by virtual means (e.g., augmented-reality overlays), while the physical part (e.g., a molecule model or a terrain model) does not convey much data in the information visualization sense. Thus, these examples of data representations are more virtual than physical. In order to address these limitations, researchers have been recently investigating "pure" dynamic physicalizations, i.e., data physicalizations that can physically reconfigure themselves. But since dynamic data physicalization is a nascent stream of research with currently few real-world applications, we will cover this topic in section on enabling technologies and in section on user studies.

Physicalization for Communication and Education

Data physicalizations are often used in the context of data communication and education. While analytical use suggests that the represented data is as of yet unknown and meant to be analyzed for insights, use for communication implies that the represented data has already been analyzed and that the purpose is to communicate specific data insights. Similar to analytical applications, we are interested in clear encodings but here combined with storytelling elements guiding an audience towards the discovery of the insights that the designer wants to communicate [ , ]. Possible communication scenarios are diverse and include pedagogical contexts as well as collaborative decision making contexts, for example when an analyst guides an audience of stakeholders through the findings of a data analysis.

. Early Uses

It can be difficult to determine in hindsight whether a specific data physicalization was created for the purpose of analyzing data or whether it was created for the sole purpose of communicating or teaching data insights, independent of how that finding came to be. We therefore focus here on examples where the context strongly suggests that the main focus was on some form of communication. In particular, in the th and th century, the use of physical data representations and models was popular for science education purposes, to explain concepts to students or lay audiences. For example, the instrument maker and science writer George Adams created and distributed solid models (Figure -a) to help explain solid geometry concepts. There exist some other examples of physical representations used to explain mathematical concepts [ ], but they are much less numerous than in fields such as medicine or chemistry, where models are still in wide use today, both for analytical purposes (as we saw in section ) and for communication and teaching purposes. Some of these models could be physically rearranged or taken apart [ ] to explain, for example, spatial relationships between organs or reactions between molecules and atoms [ ].

Besides such uses of physical representations for teaching purposes, some scientists also created physical representations of their findings for communication purposes. For example, Dorothy Crowfoot Hodgkin, who had worked out the three-dimensional structure of the penicillin molecule, created physical molecular models to demonstrate her findings to her colleagues (Figure -b). Later examples, such as the Moniac, a hydromechanical analog computer ( ), were used to teach dynamic processes, here basic economic principles [ ] (Figure -a). Its creator William Philips realized that flow metaphors were commonly used to teach economics, but had never been made physical [ ]. The hydraulic computer embodies a set of equations to model an economic system. The variables of the model can be set by adjusting flow rates and the effect of such adjustments can be immediately observed in the waterflow. While Philips initially constructed the machine to better understand Keynesian economics himself, it turned out to be of great educational value and about of these machines were constructed and used worldwide for teaching in the s and s [ ]. Other data communication efforts were more directed at the general public. For example, the biologist Charles Davenport aimed to explain the concept of variation in evolution and basic statistical principles by sorting people and sea shells according to some property such as their size [ ] (Figure -b).

.

Data-Driven Presentations

In more recent years, physical data representations have been a medium of choice for data-driven presentations. A commonality in these examples is that the data representations are composed of unit elements, similar to Isotype visualizations [ ] and to constructive visualization [ ] (which will be discussed in subsection . ). A presenter manipulates these elements while explaining the data to underline the message they want to convey. The audience can observe in real time how the data changes following the actions of the presenter. While the audiences seem to appreciate these data-driven presentations, their effectiveness compared to the use of screen-based data representations in such a context has so far not been studied and presents a rich open field of research.

.

Self-Directed Exploration and Public Displays

Another form of data communication to large audiences takes place in science museums, galleries, and other public places showing data-driven installations.

A difference to the aforementioned data physicalizations used in data-driven presentations is that the story to be told about the data needs to be integrated in the design of the data physicalization -generally no presenter is required to guide the audience through the data insights. Many data-driven installations have artistic purposes and will be discussed in section . Among examples of data physicalizations mainly used for communicative purposes, a research project undertaken by Microsoft Research studied how the use of physical charts influenced citizen engagement in local politics [ , ]. They provided small electronic voting units to the inhabitants of a local road, and deployed physical bar and pie charts located in a public place to draw people in to observe and discuss the voting results (see Figure -a). No specific storytelling elements were used in this example, as the charts benefited from being situated in the area where the data was collected, and from the audience being already familiar with the type of data being shown, since they had voted on these questions beforehand. Another study investigated the differences of showing data via physicalization versus a common, touch-enabled display in a public setting [ ]. Passers-by of a busy street were able to select particular line charts depicting local statistics either via a touch screen or by manipulating Plexiglas plates that showed the same graphs. Such studies will be discussed in more detail in section , which covers user studies specifically.

.

Participatory Data Collection and Physicalization

Certain data physicalizations installed in public spaces further promote public engagement by going beyond mere consumption, and by allowing people to engage in the data collection and in the construction of the data physicalization itself. For example, artist Hans Haacke created in an exhibition at the Museum of Modern Art (MoMA) in New York where he asked a current socio-political question and provided two ballot boxes for visitors to vote on the issue [ ]. The transparent boxes can be considered a physicalization: they functioned as physical bar charts giving an indication of the answer distribution.

Such physical surveys can take on a more complex form as illustrated by the project Cairn [ ] (see Figure ). This system was installed in a digital fabrication laboratory (fab lab) in a museum in Paris. The aim there was to replace questionnaires handed out to visitors to collect data about the activities in the fab lab. Each visitor was invited to construct a little stack of wooden tiles representing the different activities they engaged in, such as learning or teaching a skill. The placement on the table surface encoded how frequently they visited the space and how much time they spent there. A number of other examples of participatory data physicalizations exist, several of which have been created by the consultancy firm Domestic Data Streamers [ ].

All such data collection methods result immediately in a data physicalization, which has advantages, for example, as they may improve the experience of answering the underlying survey [ ] and may lead to higher participation rates. At the same time they may also result in limitations concerning the quality of the collected data: since all previous answers are visible to new participants, they may influence answer behaviors.

Physicalization for Accessibility

Accessibility of data to the vision-impaired is a particularly promising application area for data physicalization. As before, we briefly overview early systems and then discuss recent developments in HCI and related areas. to design tactile graphics, with extensive advice on how to reproduce classical statistical charts and thematic maps. Tactile graphics stand in a gray area between traditional data visualizations and physicalizations because they are constrained to a static D medium (paper), but they also extend slightly to the third dimension, allowing them to be sensed haptically. Tactile graphics is not limited to pure information consumption, as "drawing kits" can be used by blind students to create their own data representations [ , ]. Another method traditionally used in schools to teach mathematics and graphs to blind students is the graph board, which consists of a cork board with raised grid lines, on which students can place push pins and link them with rubber bands [ , ] (Figure -b). Like the tactile graphics drawing kits we mentioned before, graph boards allow students to construct tactile D graphs by themselves, with the difference that they are created by direct manipulation and can be freely rearranged once built. While computer-supported tactile graphics require embosser machines that are not easily accessible to the general public, D printers now make it possible to rapidly reproduce existing tactile infographics at home and create original ones [ , ] (see Figure -a). D printers are also able to generate more pronounced reliefs, which may facilitate graph reading [ ]. More importantly, D printers make it possible to experiment with a range of form factors that go beyond tactile graphics. In particular, some research is underway suggesting that extruded D charts or D chart cutouts may convey data more effectively to blind users than tactile graphics because they tap into their ability to follow object contours [ , ] (see Figure -b).

Although many data physicalizations for the blind are passive objects, there has been recent developments in making systems that are more interactive. For example, the traditional graph board has inspired the design of an experimental system called the Tangible Graph Builder, which uses a combination of tangible object tracking and sonification to let blind students easily build D graphs such as line graphs and bar charts [ ] (see Figure -a). Similarly, Tangible Reels is a system that helps blind users explore and construct networks of physical lines on a planar surface, and can be used for example to physicalize hierarchies or bar charts [ ]. Finally, an experimental system called Linespace has been developed that can support advanced data exploration tasks through interactive tactile graphics: a custom D printing system equipped with a mechanical scraper allows blind users to produce or erase tactile graphics on demand using a combination of gesture and speech input [ ] (Figure -b).

Despite some exciting developments in using data physicalization for the vision impaired, many opportunities for further research remain. For example, all the aforementioned approaches are based on D data representations that have been initially created and optimized for visual displays. Although giving access to standard charts is important, it is unlikely that D representations are fully tapping into human haptic perception abilities. Data physicalization makes it possible to convey data using richer physical representations that are fully three-dimensional and may have varying physical properties such as texture, softness, or temperature. However, these capabilities have not yet been used to empower vision-impaired users in their ability to explore data.

Physicalization for Self-Reflection and Self-Expression

Having covered the uses of data physicalization for the purposes of analysis, communication, education, and accessibility, we now discuss less pragmatic and more "casual" applications. In this section, we focus on the use of data physicalization for self-reflection and self-expression.

Our daily lives are becoming increasingly entangled in data, ranging from physical activity logs, over archives of social media posts, to any kind of digital resource pertaining to our hobbies. These data form a new potential to use in order to understand ourselves better and make positive changes in our lives [ ]. Perhaps enabled by an increasing accessibility to crafting and fabrication tools, specific forms of physicalization have emerged that specifically focus on the expression of one's self through data, and related to that, to facilitating different types of personal reflection on one's behaviors, attitudes or opinions. While such forms of "personal" visualization are not that uncommon in traditional visualization [ ], physicalization seems to encourage new kinds of data practices in this realm, such as in terms of how people can be enabled to self-create data representations for themselves, and in how the resulting physicalizations are then appropriated to be carried or worn for one's own benefit or for others to observe. While most research on personal visualization has focused on deploying traditional data representations that are primarily designed to be looked at, physicalization holds the promise to be more readily perceptible and might therefore be more easily interpretable to their owners or the people in their vicinity [ ]. For instance, the Activity Sculptures project [ ] mapped personal physical activity data generated by running trackers unto the dimensions of D-printed symbolic artifacts like a human figure, a necklace, a lamp or a jar (see Figure -b). In an attempt to control these resulting shapes, participants deliberately considered aspects of playfulness and aesthetics to plan and then perform their runs that ultimately generated those shapes. The SweatAtoms project [ ], shown in Figure -a, involved a more abstract translation of personal physical activity data, heart-rate data and steps taken, which became transformed into unique and sculpture-like objects generated by off-the-shelf D printers. Personal physical activity data has also been visualized in terms of the living conditions of a plant [ ]. This project brought forward the sensitive relationship between a physicalization and its data, as the owner is in the end emotionally and morally responsible for the life of a plant through and by her own activities.

.

Self-Expression

Data physicalization has also been used as a medium for self-expression. As any kind of visualization of personal data only forms a limited representation of one's body and self, the act of expressing one self to others via data reveals a particular tension in the subjective perception of information privacy [ ]. Receiving a physicalization of one's self as a gift from others can thus reveal the potential conflict between what people think of their self and what others are able to find [ ]. Proudly worn as a necklace, or put as a decorative piece of art on the shelf at home, such kinds of physicalization can act as expressive or provocative starting points of discussion with others. One of the earliest digital experiments in data-driven self-expression is a wearable visualization, a device that slowly generated dynamic folds in consecutive layers of fabric according to daily activity sensor data [ ]. Here, the main concern was to negotiate the level of privacy between the wearer and onlookers: the display was meant not to be readily understood by strangers, whereas friends and family were believed to learn how the folds related to the data, and thus the wearer's state of mind. More recent examples have explored the design space of data jewelry further, as Necklan [ ] conveyed particular facets of one's language knowledge and proficiency, with the goal to encourage onlookers to get involved in conversations with the wearer.

Due to its relative abstractness yet physical and accessible nature, a data physicalization can be appropriated in a variety of ways. Accordingly, some physicalizations have been explicitly designed so to encourage varying ways of use and ownership within people's life. For instance, facilitated by the possibilities of modern digital fabrication technologies, the same set of physical activity data has been represented by a diverse set of functional objects, such as jars, lamps, necklaces or D figures [ ] (see Figure -a). More abstract D-printed objects, such as the SweatAtoms [ ] that were shown in Figure -b, or the Fantibles [ ] that represent one's commentary about sports on Twitter along with the uniqueness of each sports match, are more meant to be used as improvised decorations within the home. Such practices however come at a cost, as for example they create a lot of plastic objects which will likely be discarded sooner or later. The users are unlikely to keep such activity representations for extended amounts of time, as their home would become very cluttered with objects which become meaningless as more and more of them appear.

Personal data physicalization clutter is less likely to occur if personal data physicalizations are considered as unique and rare objects. Social Fabrication [ ], the sharing of digital fabrication activities with others, has been suggested to facilitate conversations and reflections about the created outcomes and their creation process. Moreover, people that partake in such activities tend to invest a certain meaning into the artifacts that they made themselves [ ], which they might not do when confronted with something that was mass-produced. In the context of data physicalization, the process of making could equally provoke a certain level of curiosity, such as to find out what the fabricated shape will look like, and how it relates to the data [ ]. Moreover, by lowering the accessibility for lay people to create visualizations, various collaborative behaviors could be supported, and as such lead to a shared, improved understanding of particular datasets or the abstract concepts that Data physicalization can also be considered as a useful fabrication activity in and by itself, as its transformative qualities can become relatively easily integrated in existing fabrication and craft activities, such as crochet and its allied practices, among many others [ ]. Several projects have demonstrated how physicalization can democratize the practice of information visualization by requiring less skills or expertise than its digital counterparts. For instance, the ideas behind constructive visualization focus on offering an easily and intuitively adaptable collage of small blocks akin to kindergarten play to allow non-experts to visualize their data using easily manipulable tools [ , ] (see Figure ). Alternatively, physicalization authoring can be facilitated on a more technological level, as the Physikit [ ], shown in Figure -a, combines various custom hardware and data configuration tools to allow users to capture and visualize environmental data through physical cubes that self-illuminate, vibrate, move or cause air flows.

The recent focus on making reveals how data physicalization can be understood as a manifestation of data within a "trajectory of use", which starts from before the first acquaintance with the data, over its actual design and physical fabrication, and ending in the hands of others, in the future [ ]. By considering data as a trajectory, one can recognize how people also make sense of data during the fabrication of visualization itself [ ], as physicalization makers make conscious decisions about how the data should be represented in order to evoke associations with experiences or people, to support activities that allow for episodes of reminiscing, or to encourage potential social interactions. Moreover, during the physicalization construction itself, aspects of data collection, construction and self-reflection can be deeply intertwined [ ] (see Figure -b for some concrete results). These qualities have been exploited during the process of hands-on physicalization making workshops in order to make principles of data representation more graspable [ ], or to push the creativity of students in visualization [

]. Overall, research in personal physicalization fabrication has revealed a yet largely unexplored potential to represent more qualitative and subjective aspects that cannot yet be readily captured in data [ ].

Physicalization for Enjoyment and Meaning-Making

When data physicalization is understood within the context of the information visualization discipline, the representation of data occurs mostly for pragmatic purposes.

Yet broader conceptualizations like aesthetic [ ], artistic [ ], or casual visualization [

] have recognized how the use of visualization can also be exploited to express rather than purely communicate particular viewpoints that are related to data. As such forms of visualization tend to reach a much broader and larger, yet often less visually literate, audience, some researchers have focused on whether and how hedonic traits such as "fun" or "pretty" might actually become valuable and potentially complementary to the already existing pragmatic measures that benchmark visualization performance [ , ]. For instance, the Promoter-Inhibitor Motivation Model (PIMM) revealed how a variety of promoting and inhibiting motivational factors tend to drive the "consumption" of visualization, how they change over time and hereby influence both the frequency and duration of visualization use [ ]. Other research has explored the impact of visualization on recognition and recall [ ] and persuasiveness [ ], suggesting that visualizations that are memorable "at-a-glance" are also capable of effectively conveying the message of the visualization. Some consensus seems to be emerging that more abstract representations benefit task-performance and reflection, whereas more aesthetic, playful and pleasurable data representations are more able to "engage" people, an effect that can be further augmented by the The research area of ambient displays has created perhaps some of the earliest examples of computer-enabled physicalizations. In a desire to move the access to information away from the computer display and into the everyday built environment, the most early prototypes of ambient displays experimented with translating data values into environmental aspects that were unobtrusive but still could be perceived via multiple human senses, including air displacement, ambient temperature, lighting effects; or more abstract animated shapes and patterns, air bubbles or actuated physical objects [ , ] (see Figure -a .

Artistic Expression

Visualization is also used well outside the scientific community, such as by individuals and groups with a background in, graphic design, typography, architecture, interaction design, media art, among many others [ ]. Moreover, there exists a wide designerly spectrum of visualization, ranging from pragmatic renditions of data that can be recognized as being "visualizations" in that they are accessible and readable, to more artistic renditions that are not meant for the user to be readily understood, but rather aim to raise the awareness of some concerns that underlie the data [ , , ]. Naturally, similar practices can be recognized within physicalization, among which those that revolve around more liberal and abstract mappings that promote the -often artistically motivated -expression of viewpoints related to data.

For instance, well before the term "data physicalization" was in use, the term "data sculpture" was coined and defined as "a data-based physical artefact, possessing both artistic and functional qualities, that aims to augment a nearby's audience's understanding of data insight or socially relevant issues that underlie it" [ ]. Similarly, Loren Madsen, an artist and pioneer in the domain of data sculpture, defines data sculpture as a three-dimensional variety of "art whose form in large part is determined by data or information" [ ]. Accordingly, many data physicalizations have been exhibited in musea as art installations, with the goal to expand the human experience of particular data-evidenced phenomena [ ]. Meanwhile, the concept of "data object" frames the use of data as a "design material" that ultimately allows the development and fabrication of everyday objects that embody data [ , ]. For example, the physical dimensions and proportionality of a table (shown in Figure -a) can reflect certain statistical facts mirroring the inequalities in global health, while its important weight can represent the "difficulty to flip the economic system". The concept of "data materialization" [ ] proposes a more liberal and computational interpretation of using data as a design material, as it explores how physicalization can be merged with generative art and physical craftsmanship to create functional objects that inherently represent concept-relevant data. In Figure -b, the shape of a silver-plated teapot was determined by autobiographical financial data to demonstrate the viability of a data-informed yet artistically-inspired workflow. Objektiviserung [ ] framed a similar concept within the aspect of "neomaterility", i.e., the newly found ability of materials and their fabrications to incorporate networked digital technologies. Here, the resulting physicalizations are deliberately rough, abstract and "glitchy" in order to encourage the audience to reflect upon the crudeness of how data, algorithms and humans interact with each other. In contrast, some data physicalizations have deliberately used patina as a resource for aesthetic properties, in order to lend credibility and bestow feelings of "warmth" [ ].

Many other examples exist that exemplify the creative power of physicalization for artistic expression, yet the vast majority did not appear in peer-reviewed articles within the HCI and visualization communities. For those, we recommend the list of physical visualizations and related artefacts [ ].

.

Multisensory Expression

Through physicalization, data can be represented beyond the purely visual and tangible universe, to include our other human senses. While the actual feasibility of "touching, swiping, grasping, feeling, hearing, smelling, and even tasting data" [ ] can only be imagined in the future, these visions are often based on the assumption that representing identical datasets through different modalities may support a different user experience and affective response [ ].

Although most data physicalizations are meant to be explored primarily through sight, they can be designed to convey additional information or emotions through touch, and even audition [ ] and taste [ ]. In terms of mapping data unto taste, TastyBeats [ ] creates a personalized drink both in color and taste, offering alternative ways of engagement with the data, while EdiPulse [ ] creates D printed chocolates displaying cheerful messages. Both edible physicalizations make some kind of direct conceptual link with the data they attempt to convey, here to provide some kind of physiological reward after a physical activity. Yet, as the many examples from the Data Cuisine workshops compellingly demonstrate [ ], the potential space for abstraction is vast, and potentially should consider both taste as its visual and olfactory presentation.

Enabling Technologies

Historically, data physicalizations were crafted manually (see examples in subsections . and . ), often requiring considerable time and effort to come into existence. Technology is now making them easier and easier to build, and makes it even possible to embed dynamicity and computation in the data physicalizations themselves. These trends parallel the history of classical (flat) visualizations, which were also crafted manually and used to take considerable expertise and time to produce. Computer software and printers facilitated the design and production of static visualizations and statistical charts, while graphical computers made it possible to create rich and powerful tools for interactive data exploration [ , ].

Here we discuss similar enabling technologies for data physicalization, which are currently an active subject of research.

. Digital Fabrication

The process of creating static data physicalizations has been dramatically sped up since the availability of digital fabrication technologies such as D printing and laser cutting. We already mentioned how these technologies have been quickly adopted by scientists for visualizing three-dimensional scientific data (see section ) and have been more recently used for a range of purposes, such as producing accessible graphs (see section ) or giving physical form to personal data (see section ). Presumably, the wide availability of digital fabrication technologies has also led to an explosion of data sculptures in the past ten years [ ].

Digital fabrication machines facilitate the physical work and increase the accuracy of the resulting artifacts, but they require design files specifying what work a machine should be doing. These design files can be cumbersome to create manually and some work in HCI has focused on facilitating their creation by reading in data and proposing more or less parameterizable data physicalizations. Some systems are rather specialized and constrained in the types of physicalizations they allow to create, such as the physical activity physicalization tools from Khot et al. ] is a bit more generalist in that it supports a small set of D charts inspired from classical statistical charts such as bar charts, line charts, and prism maps (maps where height of map areas encodes a data variable). MakerVis also supports different fabrication technologies, namely D printing, laser cutting, and machining. Meanwhile, Bader et al. [ ] presented a fairly general method for physicalizing D scientific datasets using multimaterial D printers. Finally, as we already mentioned in section , Brown and Hurst [ ] proposed tools to facilitate the D-printing of classical statistical charts for the visually impaired (see Figure -a). Despite all these tools, there is currently no general-purpose tool that can allow any user to easily physicalize any dataset, and thus, there is a vast open area for future research.

. Actuation Technologies

While many of the examples mentioned in this chapter are not dynamic, that is, they are either immutable or rely on human manipulation to support interaction, the ultimate goal is for data physicalizations to be as interactive and responsive as screen-based visualizations. This vision was already presented by Ivan Sutherland in his essay "The ultimate display" [ ]. At this time we are still far from achieving such an ultimate display but we are definitely getting closer. Within HCI, the sub-community working on shape-changing interfaces is mostly concerned with proposing technologies that enable the required computer-driven control of physical geometries or material properties [ , ]. We discuss here a range of examples which each enable some form of actuation relevant for data physicalization, that is, which enables either positioning in space, control over an object's geometry or their material properties. ] improves considerably on these shortcomings but at the cost of speed, that is, objects move much slower and can only move in sequence, one after the other, which introduces a different scaling issue. For example, if the technology was used to display data as a D scatterplot, then a simple scaling action, which would require to update all data points, would take a long time to complete. "made of multiple elements whose topology can be reconfigured, or can reconfigure itself". Le Goc et al. [ ] define three dimensions for characterizing composite data physicalizations: i) their level of granularity refers to the number of elements they are composed of; ii) their degree of manipulability refers to the extent to which their elements can be manually rearranged; and iii) their level of actuation refers to the extent to which their elements can rearrange themselves without human intervention. They note that an ideal composite physicalization should be able to support a high level of granularity, a high manipulability, and a high degree of actuation, but that no such system exists yet. ]. These displays are able to react quite quickly to user or system commands but are fixed in their basic geometry, that is, the pin layout is fixed and cannot easily be extended or rearranged. Furthermore, pins cannot be manipulated sideways. Thus their degree of manipulability is While shape displays based on linear pin modules are best suited to realize data physicalizations resembling bar charts, many shape-changing systems have been proposed which transform their shape in a variety of ways. For example, Morphees [ ] discuss features of shape resolution such as curvature, amplitude, or zero-crossing which can be exploited to represent data in new ways. Most shape-changing interfaces focus on one specific shape manipulation, for example, LineForm controls the shape of a line [ ] while TiltDisplays [ ] use actuation to reorient display surfaces arranged in arrays. Tiltstacks [ ] combine linear actuation and tilting to enable actuation with more degrees of freedom. The combination of degrees of freedom is especially promising for data physicalization, but has so far not been explored. Shape-changing interfaces are still facing great challenges [ ] and as research progresses, it will greatly enrich the diversity of possible actuated data physicalizations.

Positioning in

Control over

Control over Material Properties. Besides position and geometry, data can also be encoded in material properties such as roughness, weight, or reflectivity. Controlling material properties is in most cases non-trivial, especially if one wants to control multiple properties at the same time. Nonetheless, solutions have been proposed to computationally control certain material properties such as stiffness. For example, Jamming User Interfaces [ ] relies on a hydraulic control to jam particles into different shapes and control their stiffness, and Materiable [ ] simulates different levels of stiffness using a pin array. Niiyama and colleagues explored how to control weight and volume changes by pumping liquid metal into interior channels of an object [ ]. Meanwhile, PneUI [ ] uses pneumatic means to enable texture changes through actuated air bubbles of different sizes. Metamaterial textures [ ] also enable the control of textures but through purely mechanical means, that is, a special design of D printed interior cells allow people to modify the texture of a D printed object.

. Specialized Applications for Creating Data Physicalizations

Most of the systems discussed above enable some form of actuation potentially useful for the purposes of data physicalization, but do not explicit target this application area. Here we discuss a few systems which principally target a specific type of data physicalization and are meant to facilitate their realization. As we already discussed in subsection . , several software tools have been proposed to facilitate the use of digital fabrication machines such as D printers and laser cutters to create static data physicalizations. Other tools exist that help users create data physicalizations with other technologies. For example, Vol velle [ ] is a software tool intended for data communication purposes, and which transforms screenshots from an interactive volume visualization system into a a printable document with cutting and assembly annotations for making a volvelle (a paper construction with rotating parts) (see Figure ). Physikit, which we already mentioned in subsection . on Physicalization Making, lets users create ambient dynamic physicalizations. It consists of a set of blocks which each support one type of actuation such as producing air flow, light, movement, and vibrations (see Figure ). The elements were used in combination with a citizen science project where data types such as temperature or air quality from an environmental sensor in people's home could be linked to each of these blocks [ ].

Studies of Data Physicalization

A vast body of research in educational sciences, developmental psychology and tangible user interfaces have suggested that the examination and manipulation of physical objects can promote understanding and learning [ , ]. Here, we focus exclusively on studies involving physical representations of data as opposed to, e.g., physical models or tangible controls. Although some of the systems discussed here have been already mentioned before in this chapter, we focus now on research questions and findings. Studies can be roughly classified into comparative, where physicalizations are compared with non-physicalizations, and non-comparative, where physicalizations are studied in isolation.

. Comparative Studies

In his unpublished doctoral dissertation, Dwyer [ ] presented participants with time-evolving country indicator data conveyed either as a visualization printed on regular paper or as a space-time cube physicalization [ ]. He asked participants questions about the data and found that the physicalization facilitated some tasks but not all. However, the purpose of the study was not to compare physicalizations with non-physicalizations but to compare D with D data representations. Since the two conditions used a different encoding, the study does not allow to conclude about the effect of physicalizing data.

The first study to compare data physicalizations with equivalent visualizations was probably by Jansen et al. [ ] (see Figure -a). They had participants perform elementary information retrieval tasks on interactive D matrix visualizations, screen-based D bar charts (both monoscopic and stereoscopic) and equivalent physical D bar charts. They found that participants performed best with the D visualizations, but when a D representation was used, the physical condition outperformed even the stereoscopic display. A second experiment suggested that the likely reason was that physicalizations allowed participants to quickly and efficiently "bookmark" data points (bars) of interest with their fingers. Bookmarking data points reduced the cognitive effort required to accomplish some of the tasks, and although the screen-based visualizations let users highlight data points by clicking on them, it was presumably not as efficient as using multiple fingers on physical objects. The authors also speculated that the higher visual realism of the physical objects (e.g., their perfect rendering of depth cues) might have made them easier to read. However, a study by Bérard and Louis [ ], although not specifically on data physicalization, challenged the notion that visual realism is always desirable. The study asked participants to solve D puzzles on a D screen, on a high-performance handheld perspective corrected display (HPCD), and on a physical model. The physical model was used as a "gold standard" control condition but it happened to be inferior to all digital conditions, likely because of visual legibility issues: inhomogeneous shading of the object and, most importantly, reflections resulting from its acrylic casing. Such issues were easy to overcome or simply non-existent on the computer-rendered D model. Meanwhile, Stusak and Aslan [ ] noted that data physicalization permits more complex designs than the basic D bar charts used in Jansen et al., and compared two physical star plot designs with two D interactive visualization designs in their ability to support information retrieval tasks. However, no advantage in performance was detected and participants reported finding the physicalizations harder to use, partly due to their lack of affordances and their fragility. The authors concluded that for physicalizations to be usable, it is important to get their physical design right. Both experiments tested implicit memory, i.e., participants were not told in advance that they would be tested for recall. Although these findings need replications to be definitely confirmed, they provide suggestive evidence that data physicalizations can help people memorize some facts better than flat visualizations shown on paper or on computer screens.

Moving beyond pure performance metrics, Hogan and Hornecker [ ] compared people's affective responses when shown data with a regular bar chart vs. tangible data-driven artefacts. They used the Repertory Grid Technique (RGT) to elicit thirteen personal constructs (e.g., instinctual-cerebral, artificial-organic, fundull) that capture differences in how people experience the data on an affective or hedonic level. However, the two tangible data-driven artefacts were not strictly physicalizations in that they did not convey data using physical properties or geometry, but using computer-driven audio and haptics. Later, Hogan and Hornecker [ ] conducted a similar study with three simplified designs and derived constructs to characterize the subjective impression elicited by data values conveyed through visual, audio and haptic modalities. Claes and Vande Moere [ ] examined how physicalization affects engagement using a "controlled in-the-wild" method. They deployed three versions of a data exploration device in public streets: one virtual (digital display + touch interaction), one physical (physicalization + tangible interaction, see Figure -b) and one mixed (digital display + tangible interaction). The device allowed passersby to explore the evolution of different local dimensions such as birth rate, unemployment, CO emissions or traffic accidents. Observations and interviews suggested that the ability to grab, share and investigate the physical line graphs lead to more playful, collaborative and active forms of information discovery.

Finally, one comparative study examined the construction, rather than the consumption of data representations. Wun et al [ ] ran a qualitative study comparing the use of spreadsheet software to the use of tangible tiles for authoring bar chart visualizations (see Figure ). They found that while the spreadsheet software imposed a certain workflow in the authoring process, the wooden tiles allowed more freedom in exploring different encodings and layouts. People also spent time exploring the spreadsheet tool itself, while this tool familiarization process was not necessary with the wooden blocks. While these observations emphasize qualities unique to physical objects, the authors also discuss how they can be used to inform the design of better software tools.

.

Non-Comparative Studies

Among the studies that investigated physicalizations alone, some involved showing physicalizations to people and recording their impression or their opinion. For example, Gwilt et al. [ ] showed examples of data sculptures to people from different communities (engineers/scientists, designers, and the general public), and interviewed them about the potential of this medium to communicate data. Despite some initial skepticism particularly among engineers and scientists, participants commented on the usefulness of having a direct tactile experience with the objects, and the fact that they were more likely to stimulate discussions than regular visualizations. Engineers and scientists expressed a preference for more analytical data representations, while designers preferred more open-ended, experiential representations. More recently, Boem and Iwata [ ] designed a shapechanging display for conveying vital signs of a hospitalized person to remote relatives, and exhibited it during two major Media Art festivals. They observed peoples' spontaneous interactions with the object and conducted interviews suggesting that the object is reliably experienced as a living organism, elicits both positive and negative affects, but its movements can be hard to interpret. Other studies, generally conducted in the lab, involved more extensive use of the physicalizations. ] conducted an observational study to gain insights into which types of interaction techniques participants prefer when interacting with a tabletop actuated bar chart. They found that people generally preferred to directly interact with the bars (pushing and pulling them) although these gestures do not naturally map to all functions (for example scrolling within a larger data set). In a follow-up study, Taher et al. [ ] asked participants to carry out data exploration and data presentation tasks on the same actuated bar chart. They analyzed hand actions and body movements, and found much variation between participants in their data exploration behavior, with some exploring very little of the available data. They concluded that overview techniques are necessary when displaying large data sets on a much smaller shape display. Meanwhile, Daniel et al [ ] evaluated the actuated ambient data physicalization CairnFORM (see Figure -a) by asking exhibition visitors to carry out elementary tasks similar to Jansen et al. [ ], and found people to be accurate except with the sorting task. They then conducted a lab study to assess people's ability to perceive and memorize motion sequences shown in their peripheral vision. Of the three motion types tested, a slow then accelerating motion was found to be the best. While all of these studies investigate specific physicalization designs, a study from Jansen and Hornbaek [ ] investigated in the tradition of graphical perception studies (cf. [ ]) how accurately people can judge the size of physical marks, that is, building blocks of data physicalizations. They showed participants two differently sized bars or spheres and asked them to indicate the size of the smaller one relative to the larger one (see Figure -a). They compared their findings to those of previous studies on graphical perception and report that physical bars are judged with a just slightly higher error than D bars. For spheres, they found that the systematic error reported by previous studies can be considerably reduced by encoding data relative to the surface of spheres instead of their volume. [ ] report a year-long study where they had residents from a street engage with community data using various technologies, including a dynamic physical chart acting as a public display (see Figure -a). The study however does not discuss the specificities of the physicalization medium, but instead examines the relationships between people, places, and data on a general level.

As with comparative studies, some studies focused on the construction of physicalizations. In an early study by Vande Moere and Patel [ ], design students were asked to convey a data-driven insight through a data sculpture. The authors analyzed the + data sculptures produced, allowing them to articulate qualities unique to physical representations as well as the different design strategies available (see section for a more extensive discussion). Huron et al. [ ] conducted an observational study to investigate how participants spontaneously use colored wooden tiles to communicate data to peers (see Figure ). They found that assembling physicalizations using physical tokens encourages experimentation and makes it easy to try out different ways of encoding data to optimize communication of insights, but that tokens are also cumbersome to update. In a field study, Thudt et al. [ ] taught participants how to manually craft data physicalizations with materials such as beads or plasticine, and then asked them to record personal data of their choice on a daily basis (see Figure -b). Through a diary analysis and interviews, they observed that the exercise promoted self-awareness and sometimes behavior change, and found that the iterative construction of personal data physicalizations allowed the processes of data collection, visualization and self-reflection to be deeply intertwined.

Models and Theories

Although physicalization practice has existed over many decades, academic research has only recently started synthesizing knowledge into models and theories. We briefly review this stream of work in this section.

Table of Contents

. Pipeline Models

A commonly used model of information visualization is the "information visualization reference model" or "infovis pipeline", which sees information visualization as a process of turning raw data into a picture through a series of successive transformations [ , ]. Although useful as a descriptive model of computer visualizations, this model is purely logical and does not account for the physical world, thus it does not capture data physicalization. been added (b). The physical referent is "the physical object or physical space to which the data refers" [ ]. For example, if the data is the power consumption of a consumer device over time, the physical referent is the device itself. When the distance between the physical referent (b) and the physical presentation (a) is small (c) -for example, device consumption is visualized on the device itself, the visualization can be considered as situated. Situated visualizations allow for a form of interaction that classical visualizations do not support, and that consists of For instance, it has been repeatably argued that physicalization potentially offers more freedom for encoding information in perceivable form, as it can exploit a wide variety of physical, material and tangible characteristics, and can allow humans to explore it using all their sensory and motor skills [ , , , , ]. As most common information visualization techniques are purely visual and two-dimensional in nature, and are rendered on paper or digital display media, one can easily imagine that no real compelling reason exists as to why the physical mapping of data should necessarily mimic these techniques. Our everyday experience of physical reality seems to offer much promise to map data in ways that are more accessible and intuitive to our human mental image, preconceptions or expectations. In other words, there exists a design space that relates to the presence, materiality and situatedness of a physicalization in that it "embodies" data [ ] rather than that it just communicates data by its physical dimensions. This design space potentially allows for the more hedonic senses and interaction modalities to become stimulated in purposeful ways [ ].

Early theoretical models of physicalization, then tentatively called "data sculpture", recognized the potential of combining functional with artistic qualities. For instance, the concept of "embodiment" was proposed to capture how a physicalization typically conveys data to its audience via the use of a metaphor that is linked to the meaning of the data it depicts, as well as the perceived reality surrounding the physicalization itself [ ]. For instance, while a particular monetary value can be easily represented "abstractly", such by the height of a water fountain [ ] which may not necessarily be easily understood by onlookers, it can equally be represented in a more "literal" way, such by a heap of currency coins that together exactly correspond to that monetary value. Yet to emphasize a particular meaning, the same monetary value could even manifest itself by a While a traditional data visualization infers some form of data abstraction and dimension reduction, this operation is not necessarily easily replicable in the physical world. Consequently, a more accessible metaphor might negatively impact the resolution of the data shown, such as the number or dimensions of data values it is able to convey. For instance, whereas an actuated physical ring chart [ ] allows for detailed insight creation regarding complex energy-related usage patterns (Figure -a), its symbolic abstraction requires some guidance to be interpreted. Similarly to ambient displays, such a symbolic visualization first needs to understood as being a visualization, and then as representing content that is related to energy [ ]. In contrast, the small metaphorical distance [ ] of a power cable that self-illuminates depending on its actual energy use [ ] is probably more accessible and intuitive to understand, yet only carries a single quantitative value (Figure -b).

Based on a study of representative exemplars, the parameters that physicalizations tend to deploy to convey information has been identified and captured in a comprehensive framework, shown in Figure [ ]. Notably, the resulting complementary design space of physicalization includes the purposeful combination and crossing of human modalities that stimulate our human senses, the materials from which the physicalization is produced, as well as the tacit, human experience they are meant to generate [ ]. In addition, the framework calls for articulating and benchmarking the interaction between people and physicalization, in terms of provoking passive or active forms of user engagement, as well as whether and how the data itself is static or dynamically changing.

The area of data physicalization can also learn from emerging discussions in computational design and fabrication [ ]. For instance, in architectural design, it has been argued that the designerly negotiation between structure and material production, between diagram and model, and between how something works and what it looks like, often determines the actual outcome. Likewise, the design rationale of a physicalization could be determined by reasoning more holistically about how the material choices, fabrication methods and structural aspects link its physical components to the intrinsic structures present within the data.

Lastly, the phenomenon of how some physicalizations are able to convey their meaning through their environmental context has been analyzed from various perspectives. As we saw in subsection . , embedded data representations combine the visual and physical representation of data in ways that are deeply integrated with the physical spaces, objects, and entities to which the data refers [ ]. When the corresponding physical data referent, i.e., the real-world entity or space to which data corresponds, is in close proximity, the visualizations are considered to be situated, whereas spatially coinciding referents determine embedded representations. A so-called 'indexical' visualization [ ] makes use of a referent that can be causally linked to its fabrication, as it achieves meaning by emphasizing a particular causal link between its own embodiment in relation to its immediate surrounding environment, and as such can become practically 'invisible'. Similarly, various 'invisible' data mapping metaphors and operations [ ] in physical space have been identified, such as those that exploit particular qualitative characteristics of physical phenomena, and the relationships between them. The embedded physicalization of Figure -b is one such example. More recently, so-called qualitative displays [ ] equally aim to exploit our natural ability to interpret real-world contexts, in order to subtly yet naturally augment digital interfaces with meaning [ ]. Plant-based physicalizations, mentioned in subsection . , provide good examples. For instance, the natural size and relative direction of a plant's growth can be interpreted as an alive physicalization of its own environmental conditions [ ]. This natural understanding has already been exploited by the plant-like robotic contraption Infotropism [ ] (Figure ). By deconstructing the processes and the level of involvement of a natural or deliberate phenomenon in fabricating such indexical physicalizations, these kinds of research highlight the yet unexplored potential width and breadth of data abstraction in physicalization practice and research.

Conclusion

This chapter provided an overview of data physicalization, a vast research area that studies the use of "physical artifacts whose geometry or material properties encode data" [ ]. This area overlaps with a number of disciplines and research areas including information visualization, scientific visualization, visual analytics, tangible user interfaces, shape-changing interfaces, personal fabrication interfaces, as well as graphic design, architecture, and art.

We went through different application areas of data physicalization by discussing how it has been used for analytical purposes, for communication and education, for accessibility, for self-reflection and self-expression, and finally for enjoyment and meaning. We also discussed enabling technologies, reviewed empirical studies, and summarized models and theories of data physicalization.

Data representations that are optimized for flat media such as paper and computer screens are extremely useful and effective in many situations, and will undoubtedly remain in use. However, for too long, visualization research has exclusively focused on such representations. Even historical accounts trace back the first visualizations to hand-drawn visualizations on paper, ignoring the physical data representations that have been in use for many centuries, and for some of them, were in use well before paper was even invented [ ]. The aim of this chapter was to restore the balance by offering an overview of the rich possibilities offered by physical representations of data, as well as the many pending questions and challenges.

Since the advent of graphical computers, interactive computer displays came to dominate static media such as paper and physical models for representing data. But now, with the advent of digital fabrication and physical actuation technologies, we are entering an era where the production of data physicalizations is becoming cheap and easy, and where computation and automation can be embedded in data physicalizations in a way that will eventually make them as dynamic, interactive and powerful as today's interactive visualization software. However, much research is still necessary to better understand the design space of data physicalizations, the processes involved in consuming and producing them, and their benefits compared to flat representations or virtual renditions.

  Fig. : (a) Maxwell's "thermodynamic surface" showing the relationship between energy, entropy and volume ( ), and (b) John Kendrew constructing a physical representation of the molecule myoglobin called the "forest of rods" ( ). Credits: (a) Photo S. Trewin, . YPM HST. . Courtesy of the Peabody Museum of Natural History, Yale University, New Haven, Connecticut, USA. (b) Copyright MRC Laboratory of Molecular Biology. Used with permission.

  Fig. : (a) Frank Gilbreth's D wire model of a factory worker's hand movement (c. ), and (b) a data physicalization by the Detroit Edison Company showing electricity consumption for the year , with a slice per day [ ]. Credits: (a) Collection: Frank B. Gilbreth Motion Study Photographs ( -). Repository: The Kheel Center for Labor-Management Documentation and Archives.

  currently involve the visualization of D data [ ], but examining D representations on computer displays is subject to a number of issues such as perceptual distortion and occlusion, and the difficulty of navigating in D [ ]. Meanwhile, with digital fabrication it became easy to create accurate and visually rich D data physicalizations. Mike Bailey from the San Diego Supercomputer Center was a pioneer in the use of D printing for scientific visualization. In , he created the SDSC TeleManufacturing Facility to help scientists visualize their data in physical form (Figure -a). The facility produced a number of physical scientific visualizations of molecular, physical, geological, anatomical, and mathematical data using laminated manufacturing [ ]. Two decades later, other research labs started to investigate the use of modern fabrication technology such as full-color D printing to physicalize scientific visualizations [ ] (Figure -bc).

  Fig. : (a) Model of the black beetle virus from the SDSC TeleManufacturing Facility [ ]; (b) A model of the HIV protease, a target for AIDS therapy from the Molecular Graphics Laboratory at the Scripps Research Institute [ ]; (c) model of diffusion-tensor MRI brain data from the Visualization Research Lab at Brown University [ ]. Credits: (b) Courtesy Prof. Arthur J. Olson, copyrightThe Scripps Research Institute.

  Fig. : (a) A proteine model (left) and the same model (right) with an augmentedreality overlay showing electrostatic field (small arrows) and the potential (red and blue clouds) [ ] (b) The Planwell system for helping distributed teams collaborate during petroleum well planning and drilling: the left interface is the overseer's interface and consists of a solid terrain model with an augmentedreality overlay; The right interface is the explorer's interface, who sees the same information in-place [ ].Credits: (a) Courtesy Prof. Arthur J. Olson, copyrightThe Scripps Research Institute.

  Fig. : Examples for physical data representations and models used for communication in Mathematics and Chemistry. (a) George Adams' Solid Geometry Models ( ) [ ], (b) Dorothy Hodgkin's electron density map and molecular model of penicillin ( ). Credits: (a) Boxed set of geometrical solids (models) by George Adams, c. . Wh. . Whipple Museum of the History of Science, University of Cambridge. (b) Photo courtesy of Jon Agar.

  Fig. : (a) A Moniac analogue computer with its interior visible. (b) A photograph by Davenport showing a histogram of men arranged by their height [ ].Credits: (a) LSE Library, public domain.

Fig. :

 : Fig. : Hans Rosling using physical boxes to explain changes in world population. Credits: (a) Still image from the video "Will saving poor children lead to overpopulation?" by Gapminder, CC BY . . (b) Still image from the movie "Don't Panic: The Facts About Population" © Wingspan Productions Ltd. Used with permission.

  Fig. : (a) The storefront where the physical charts of the Tenison Road Project where shown [ ]. (b) The physical visualizations placed in public places for the study by Claes and Vande Moere [ ]. Credits: (a) Image of physical charts displayed as part of the Tenison Road project. Image courtesy of Microsoft Research and Alex Taylor.

Fig

  Fig. : Close-up of the Cairn table [ ] showing individual data points (left), and overview of the Cairn table (right).

  Fig. : (a) An example of tactile graphics [ ] and (b) a graph board [ ].Credits: (a) Image from the Tactile Graphics Starter Kit, courtesy American Printing House for the Blind, Louisville, Kentucky.

  Fig. : (a) D-printed tactile graphics [ ]; (b) Two extruded charts [ ].

Fig. :

 : Fig. : Two tangible charting systems for blind users: (a) Tangible Graph Builder [ ]; and (b) Linespace [ ].

  Fig. : Two physicalization projects that were meant for self-reflection and selfexpression, both depicting physical activity data: (a) the Activity Sculptures explore multiple and different object typologies, among which a piece of jewelry [ ]; (b) SweatAtoms are simple objects that can be used as decorative objects for the home [ ].

  Fig. : In the Data Things project, people were directly involved in the fabrication of physicalizations, hereby fostering different forms of social interaction and reflection [ ]. In (a) participants visualized the sensor-logged movements of their crochet hooks as path-like patterns. In (b) participants were 'trying on' the resulting physicalizations.

  underlie them [ ]. The Data-Things project, shown in Figure , gives an example of how the translation of data into a physical form can by itself be perceived as an opportunity for meaning-making and reflection, hereby suggesting a rich and largely unexplored potential for a much more active role of the user in how physicalizations are created, and in how their meanings emerge alongside [ ]. By incorporating personal visualizations into everyday objects, mementos, keepsakes and gifts have been created of which the owners feel they reflect upon their social relationships with others [ ]. Moreover, the manual creation and subsequent habitual use of self-made physicalizations of personal data can augment their perceived value and authenticity, and can thus spark various forms of individual or shared recollection [ ].

  Fig. : Examples of constructive visualization, which allows lay people to construct graspable visualizations by arranging simple tokens [ ].

  Fig. : Examples of early ambient displays. (a) The Pinwheels installation presents information within a space through subtle changes in light, sound, and movement, which can be processed in the background of awareness [ , ]. (b) The BusMobile installation indicated how close each used bus was to the nearest bus stop by changing the height of the corresponding bus number [ ].

  Fig. : Examples of plant-based physicalization. (a) Infotropism is a robotic plant that grows into the direction of one of two lamps, which turn on depending on the use of a recycling bin [ ]. (b) PlantDisplay treats growth of plants as a way of information display, by connecting the watering and the lighting of the plant to data sources [ ].

  Fig. : Physicalization as an artistic form of expression. (a) The different surface areas of the Data Tablecorrespondto global wealth statistics. If the table would be flipped over, which is challenging due to its weight, the distribution of table surface is the same for each person [ ]. (b) As an example of "data materialization", a silver-plated D-print teapot that expresses the monthly finances of the artist was generated by an artistic workflow that focused on using an everyday object and a traditional material [ ].

  [ ] and Stusak et al. [ ] discussed in section (see Figure ). Lee et al. [ ] modified existing artifacts, activity trackers themselves, to represent the data they had collected. Another rather specialized system is the one by Zhang et al. [ ] for automating the creation of D-printed motion sculptures. MakerVis [

  Space. Physical positioning of multiple elements within D space has been a topic of interest for a while within the tangible interaction community. The actuated workbench by Pangaro et al. [ ] used magnetic forces to move objects on a tabletop surface such that their position corresponded to user actions. It required an array of electromagnets to generate the forces and could only move objects receptive to magnetic forces. Madgets by Weiss et al. [ ] used a similar setup but enriched the controlled objects such that they themselves became interactive user interface widgets giving haptic feedback or indicating state changes. Major drawbacks of approaches relying on arrays of electromagnets is that they do not scale well to larger interaction spaces, and they require a considerable amount of energy released mainly as heat to operate the electromagnets. The Reactile system built by Suzuki et al. [

  Fig. : (a) Constraint-based interaction with Zooids robots [ ]; (b) Different data points floating freely in D through acoustic levitation [ ].

  Shape and Geometry. Position control of independent objects is not the only way to represent data physically. For example, with the use of . -D pin array displays (as shown in Figure ), interactive D bar charts and surfaces can be realized [ , ,

  Fig. : (a) The EMERGE shape display featuring by bars [ ]; (b) The inFORM shape display featuring by bars [ ].

Fig. : A

 : Fig. : A Vol velle using a transparent medium to show different layers of a scan [ ].

  Fig. : (a) Comparison of a screen-based D bar chart with the equivalent physicalization [ ]; (b) Comparison of a D paper-based visualization with a D physicalization made of standing wooden blocks [ ].

Fig

  Fig. : A bar chart made with a spreadsheet software and a bar chart made with tangible tiles [ ].

  For example, McGookin et al. [ ] evaluated the usability of their Tangible Graph Builder (see Figure -a) by giving chart construction and chart reading tasks to both sight-deprived and blind users. They observed several advantages provided by the use of tangible objects such as "two-handed interaction, quick overviews, spatial frame of reference and flexibility to employ fingers for marking" [ ]. More recently, Taher et al. [

  Fig. : (a) setup of a psychophysical experiment on the judgment of physical size [ ]; (b) five data sculpture designs used in a study on personal activity data physicalization [ ].

Fig

  Fig. : Extended infovis pipeline that captures data physicalizations [ ].

  manipulating the physical referent to change the data (d) -for example, tuning a consumer device to observe effects on consumption. When a visualization consists of a combination of many instances of the pipeline of Figure --for example when a small visualization is placed on every device in the house, the entire visualization can be considered as embedded [ ]. Figure -shows an example of a highly-embedded physicalization of temperature data, where each point on the wall can be conceptually seen as implementing the situated visualization pipeline of Figure -. . Data Embodiment and Contextual Sensemaking Data physicalization promises to facilitate more elaborate forms of sensemaking that go beyond those used in traditional forms of information visualization [ ].

Fig

  Fig. : A model of embodiment in data sculpture, mapped along two characteristics: the metaphorical distance from data and the distance from reality [ ].

Fig. :

 : Fig. : Different forms of abstraction through physicalization [ ]: (a) symbolic physicalization of the unemployment rates between males and females in Australia as an abstract helix; (b) iconic physicalization of the relative growth of the Australian economy, resulting in an implicit imbalance of a flower shape; (c) indexical physicalization showing the dangers associated with wearing thongs by a series of physical thongs that were pierced with a nail for every thousand thong-related injuries that required hospitalization.

  Fig. : A parallel coordinate plot of different physicalizations with a focus on their multisensory capacity, organized across three design space dimensions [ ].

  Table correspond to global wealth statistics. If the table would be flipped over, which is challenging due to its weight, the distribution of table surface is the same for each person [ ].

(b)

As an example of "data materialization", a silver-plated D-print teapot that expresses the monthly finances of the artist was generated by an artistic workflow that focused on using an everyday object and a traditional material [ ]. Credits: (b) Photo © Courtney Starrett, used with permission. Work by Courtney Starrett, code by Susan Reiser.

.

Pennies: Obama budget cuts visualization (video). https://www.youtube. com/watch?v=cWt hTayupE ( ), accessed: --. Acevedo, D., Zhang, S., Laidlaw, D.H., Bull, C.W.: Color rapid prototyping for diffusion-tensor mri visualization. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp.

-. Springer ( ) . Alexander, J., Lucero, A., Subramanian, S.: Tilt displays: designing display surfaces with multi-axis tilting and actuation. In: Proceedings of the th international conference on Human-computer interaction with mobile devices and services. pp.

-. ACM ( ) . Alexander, J., Roudaut, A., Steimle, J., Hornbaek, K., Bruns Alonso, M., Follmer, S., Merritt, T.: Grand challenges in shape-changing interface research. In: Proceedings of the CHI Conference on Human Factors in Computing Systems. p.

. ACM ( ) . Bach, B., Dragicevic, P., Archambault, D., Hurter, C., Carpendale, S.: A review of temporal data visualizations based on space-time cube operations. In: Eurographics conference on visualization ( ) . Bader, C., Kolb, D., Weaver, J.C., Sharma, S., Hosny, A., Costa, J., Oxman, N.:

Making data matter: Voxel printing for the digital fabrication of data across scales and domains. Science advances ( ), eaas ( ) . Bailey, M.J., Schulten, K., Johnson, J.E.: The use of solid physical models for the study of macromolecular assembly. Current opinion in structural biology ( ), -(Apr ), https://www.ncbi.nlm.nih.gov/pubmed/ . Barrass, S.: Digital fabrication of acoustic sonifications. Journal of the Audio Engineering Society. Audio Engineering Society ( ), -( ), http: //www.aes.org/e-lib/browse.cfm?elib= . Baudisch, P., Mueller, S.: Personal fabrication: State of the art and future research.

In: Proceedings of the CHI Conference Extended Abstracts on Human Factors in Computing Systems. pp.

-. ACM ( ) . Beddoes, T.: Observations on the nature of demonstrative evidence: with an explanation of certain difficulties occurring in the elements of geometry, and reflections on language. J. Johnson ( -.

ACM (

) . Redström, J., Skog, T., Hallnäs, L.: Informative art: using amplified artworks as information displays. In: Proceedings of DARE on Designing augmented reality environments. pp.

-. ACM ( ) . Rich, M.: Wooden geometric models made by George Adams. http://www.hps. cam.ac.uk/whipple/explore/models/geometricmodels/ ( ), accessed: --. Roberts, J.C., Ritsos, P.D., Badam, S.K., Brodbeck, D., Kennedy, J., Elmqvist, N.:

Visualization beyond the desktop-the next big thing. IEEE Computer Graphics and Applications ( ), -( ) . Robertson, G., Czerwinski, M., Larson, K., Robbins, D.C., Thiel, D., Van Dantzich, M.: Data mountain: using spatial memory for document management. In: Proceedings of the th annual ACM symposium on User interface software and technology. pp.

-. ACM ( ) /TVCG. . . Wilson, M.: How gm is saving cash using legos as a data viz tool. https://www.fastcompany.com/ /how-gm-is-saving-cash-using-legosas-a-data-viz-tool ( ), accessed: --. Wisneski, C., Ishii, H., Dahley, A., Gorbet, M., Brave, S., Ullmer, B., Yarin, P.: Ambient displays: Turning architectural space into an interface between people and digital information. In: International Workshop on Cooperative Buildings. pp.

-. Springer ( ) . Wun, T., Payne, J., Huron, S., Carpendale, S.: Comparing bar chart authoring with microsoft excel and tangible tiles. In: Computer Graphics Forum. vol. , pp.

-. Wiley Online Library ( ) . Yao, L., Niiyama, R., Ou, J., Follmer, S., Della Silva, C., Ishii, H.: Pneui: pneumatically actuated soft composite materials for shape changing interfaces. In: Proceedings of the th annual ACM symposium on User interface software and technology. pp. -. ACM ( ) . Zhang, X., Dekel, T., Xue, T., Owens, A., He, Q., Wu, J., Mueller, S., Freeman, W.T.: Mosculp: Interactive visualization of shape and time. In: