Lucas M Bietti

Michael J Baker
email: michael.baker@telecom-paristech.fr

Françoise Détienne
email: francoise.detienne@telecom-paristech.fr

Joint Remembering in Co-Design: An Ethnographic Study of Functions and Multimodal Processes

Keywords: Joint Remembering, Co-Design, Ethnography, Multimodal Remembering Sequence

The aim of this paper is to provide empirical evidence that illustrates how the interweaving of verbal, bodily, social and material resources supports joint remembering of relevant aspects of co-design projects during group interactions. Our data comes from an ethnographic study we conducted in a video design studio in Barcelona. The analysis focuses on the role of questions triggering the formation of multimodal remembering sequences (MRSs). This study suggests that questions acting as reminders foster the formation of MRSs. MRSs are supported by an onthe-fly integration and coordination of multiple contextually relevant resources. Our preliminary findings are relevant for the development of new design-rationale systems in HCI that consider such complex dynamics.

INTRODUCTION

Designers collaborate with a view to reaching consensus on successive design project phases. Such consensus is crucial when elaborating solutions, constructing common ground and planning future tasks in design teams [START_REF] Détienne | Quality of collaboration in design meetings: Methodological reflections[END_REF], and often relies on joint remembering of previous co-design work and project milestones. Several methods have been developed for storing design knowledge and decisions throughout design projects. One such method is exemplified by "design rationale systems", which provide documentation of the evolution of the design project, attempting to capture the reasons why that particular design is the way forward [START_REF] Lee | What's in design rationale?[END_REF]. Although design rationales embody shared design project memory, they can not completely anticipate and incorporate all aspects of the co-design processes that may be viewed as relevant at future stages of the project development. We therefore argue that their existence does not obviate the need for interactive contextualisation and negotiation of meaning of design elements, as represented in the rationale. In that case, it is important to understand the contexts and the processes by which past design decisions are interactively recreated, or "jointly remembered". In other  ACM copyright: ACM holds the copyright on the work. terms, joint remembering, as an interactive phenomenon, both goes beyond design rationales and will always be a potential necessity for design teams, given evolving contexts that require new meanings to be co-created. In team of experts such as designers, conversations are one way in which people develop shared memories of the past. Designers engaged in face-to-face conversations in the design studio often coordinate linguistic (words and syntax), bodily (e.g. gestures, gaze, posture, facial expressions) and material resources (e.g. sketches) in order to achieve shared goals (e.g. modelling and prototyping). A recent study [START_REF] Bietti | Embodied reminders in family interactions: Multimodal collaboration in remembering activities[END_REF] examined the multimodal aspects of joint remembering of a previous interactive encounter in which a group had to co-design its dream house under certain constraints relating to number of occupants, relationships, and funds. The results suggested that participants collaboratively remembered better those creativity moments when they were more jointly involved in elaborating the features of their design. That is, they remember better what initially generated most joint activity during the previous co-design phase. Based on these results, the authors concluded that participants did not necessarily collaboratively remember what was more important, but what initially generated most joint activity during codesign. In this exploratory study we pay particular attention to the central role that questions have acting as reminders play in the formation of multimodal remembering sequences [START_REF] Bietti | Embodied reminders in family interactions: Multimodal collaboration in remembering activities[END_REF]. Building on the notion of adjacency pairs in Conversation Analysis [START_REF] Sacks | Opening and closing[END_REF], that is a unit of conversation composed of two turns, each coming from a different speaker and being functionally complementary types of utterance, here we define a multimodal remembering sequence ("MRS") as: a multimodal unit of joint remembering triggered by the compliance with cooperative rules to respond with information relating to the past, in relation to the semantic content of questions. The end of MRS was marked by either an acknowledgement or a change in topic. Of course, joint remembering can occur without being triggered by explicit questioning; our methodological choice here is nevertheless to identify for analysis clear examples on the basis of positioning with respect to questions. At the co-linguistic and multimodal level, MRSs reorient group behaviour and establish shared focus of visual attention [START_REF] Bietti | Embodied reminders in family interactions: Multimodal collaboration in remembering activities[END_REF]. Such reorientations of behaviours and new focuses of visual attention indicate the joint and dynamic configurations of shared goals [START_REF] Tomasello | Understanding and sharing Intentions: The origins of cultural cognition[END_REF].

In this paper we argue that joint remembering is necessary for and therefore occurs in co-design. We provide empirical evidence that illustrates how the interweaving of verbal, bodily, social and material resources support joint remembering relevant aspects of design projects during group interactions. Our data comes from an ethnographic study we conducted in a video design studio in Barcelona. The analysis focuses on the role of questions triggering the formation of MRSs. This study suggests that questions acting as reminders foster the formation of MRSs. MRSs are supported by an on-the-fly integration and coordination of multiple contextually relevant resources. We argue that the complexity involved during joint remembering of codesign mirrors the complexity of memorable past co-design work. Our preliminary findings are relevant for the development of new design-rationale systems in HCI that consider such complex dynamics.

DOING ETHNOGRAPHY AT A VIDEO DESIGN STUDIO

For a period of five working days we recorded the activities of a group of graphic and animation designers while they developed a commercial video for Russian television. The setting for this real-world study was an animation and video production studio located in Barcelona, Spain. The stakeholders involved in the making of the commercial were the Russian subsidiary of an American multinational food manufacturing company (client), the Russian branch of a major international advertisement company, a Moscow-based film production studio, and a Barcelonabased animation and production studio, which was where we conducted our fieldwork in February 2014. The overall production of the commercial lasted from late December 2013 to mid March 2014 (March 12th), when it was delivered to the Russian channels. The production of the commercial included shootings with real actors in Moscow and the design of animated characters in 3D.

Participants

The team of designers in Barcelona where we did our fieldwork includes: i) a project leader, who was directly in contact with the client in Moscow, and as expert designer, supervised the overall design process and progress to address the client's requests (PL); ii) a project manager, who led the design process and worked on the actual design (PM); and iii) two designers who actually worked on the 3D animation and had to respond of the project leader project manager's requests (D1 and D2). PL was then an Argentine animation and live-action director based in Barcelona Spain. At the time of the filming he had more than 15 years of working experience. He has directed commercials of major interactional companies in the automobile, telecommunications and foods sectors, for major European airlines, and Spanish banks. PM was a Spanish animation designers and project manager from Catalonia. She had had nine years of work experience as professional designer and more than five years as a project manager. D2 and D1 were animation designers from Madrid, and Stockholm respectively. D2 had seven years of professional experience in the field whereas as D1 had four. While PL, D1 and D2 worked as freelancers for the video design studio for the co-design project, PM had a permanent work contract with the firm as project manager of several projects. As D1 did not speak Spanish, when he was present English was the language of interaction. PL, PM, D1 and D2 were involved in production of the commercial from the beginning of the project in late December 2014. In addition, PL was also responsible for the directing the shooting with real actors in Moscow in late January 2014.

Materials

Our recordings at the design studio in Barcelona were made with six static (fixed) cameras (4 GoPro HERO 3+ Black, 1 Canon VIXIA HF S21, and 1 Drift HD Ghost), as well as with one head-mounted wearable video camera (Looxie LX2). The audio and video recordings were transcribed in detail in ChronoviZ, [START_REF] Fouse | A system for supporting navigation of time-coded data[END_REF].

Corpus

During the working week we spent at the video design studio we collected 45+ hours of video and audio recordings. In addition to this dataset, we were given copies of the documents (e.g. production timing and storyboard) that were used to coordinate efforts among the different stakeholders involved in the development of the commercial. Although the design studio was an open working space in which more than 20 designers were working on several projects in parallel, for this exploratory study we only coded those interactional sequences in which at least two of our four participants were interacting (n=232). For pragmatic reasons, we defined interactional sequences as instances in which at least two designers out of the four members of the team were interacting. Next, we wanted to know whether these interactional sequences in which at least two of participants were part of, included two, three or the total four designers working on the commercial for Russian television. Here we found that the vast majority of interactional sequences were between two designers (84%), followed by sequences in which three designers participated (13%) and to a considerably less degree, where the four of them were involved (3%).

JOINT REMEMBERING CO-DESIGN: AN EXAMPLE

The MRS that we selected as illustrative example of joint remembering for this exploratory study lasted 00:11.88 seconds. This MRS was taken from longest interactional sequences in the four designers interacted with each other. In the first turn of MRS (L.1, fig. 2), D1 asked questions to get information about the actions occurring in the third shot of the TV commercial. D1's beginnings of questions are marked by pitch variations. Higher pitch is a prosodic resource often used by speakers in cases in which they do not align with the actions proposed by the previous speaker (e.g. change in the topic of conversation) [START_REF] Gorish | Pitch contour matching and interactional alignment across turns: An acoustic investigation[END_REF]. In this specific example, the question marked the beginning of the MRS. Both questions acted as reminders in the interaction as led D2 and PM to collaborate in the reconstruction of the actions occurring in shot three and triggered the formation of a MRS. While formulating the questions, D1 used a manual gesture to simulate the falling of the animated character in shot three (fig. 2 a andb). In the meantime, PM held up the storyboard with a written description of shot three, and showed it to D1. In doing so, she placed the written description of shot three within D1's existing focus of attention. PM's action was accompanied by a change in eye-gaze direction towards D1. PM's eye-gaze reinforced the accountability of D1 to have a closer look at the written description of the actions occurring in shot three in order for him to remember what the shot was about. In the next turn, D2 took the floor to answer to D1's request for information. In doing so, he performed a manual gesture to simulate the falling of the animated characters (fig. 2 d ande) while gazing at D1 first (d) and changing eye-gaze direction towards the written description of the shot afterwards (fig. 2 e). PM corrections led D1 to self-repair in the next turn (L.4). According to PM it seemed that D1 should have referred to shot three instead of shot two. In line 4 D2 changed eye-gaze direction towards the storyboard that PM was holding. D2's pointing gesture touching the storyboard tried to move the D1s' attention to the storyboard in order to create a shared focus of visual attention upon which the grounding of collaborative recall could occur. Immediately after, D2 performed another manual gesture to simulate the animated character being in the air while falling (fig. 2 g). In the next turn, D1's signal of agreement seemed to indicate that he remembered what the shot was about.

CONCLUSION

In this paper, we have argued that joint remembering is necessary for and therefore occurs in co-design. As codesign is a collaborative activity which relies on multiple cognitive processes at individual and shared levels (i.e. creativity; reasoning; problem-solving; remembering; and planning), we decided to focus our analysis on specific types of interactional sequences. These sequences dealt with how joint remembering relevant in a co-design project were supported by the inter-animation of verbal, bodily, social and material resources. As our illustrative example has shown, joint remembering of co-design is a complex multimodal process, and, that complexity mirrors the complexity of memorable past codesign work. Although the MRS we selected as illustrative example was not intended to be representative of how designers remember relevant aspects of co-design in general, we used it as illustration of the phenomenon in question. Our micro-qualitative analysis has demonstrated that MRS plays an important in role co-design in the realworld. These collaborative activities occurred by means of interactions between mutually dependent elements of the specific cultural eco-system [START_REF] Hutchins | The cultural ecosystem of human cognition[END_REF] under investigation. In the MRS, environmentally-coupled gestures [START_REF] Goodwin | Environmentally coupled gestures[END_REF] with the storyboard operated a common point of reference for collaboratively remembering the actions of the animated characters in shot three and four. Changes in gaze direction and pointing gestures towards the written descriptions enabled designers to create a shared focus of attention. This allowed PM and D2 to help D1 in his search for relevant information about the order of the shots in the commercial. Our micro-qualitative analysis has reliably shown that external resources did operate as artifacts externally grounding the entire interactional dynamics during joint remembering. Future studies on joint remembering in teams of designers along the evolution of design projects should take a closer and systematic view at such complex multimodal dynamics. That is, joint remembering in co-design should not be considered as merely a joint action the aim of which is to retrieve information about previous phases of the project. Joint remembering in co-design is future-oriented and necessary to accomplish common ground, make decisions and plan future actions at the design studio.Finally, if, as we have argued, multimodal, embodied joint remembering must play an important role in co-design, whether design rationales are available in the situation or not, an intriguing question for future research in HCI would concern how more timely and effective joint remembering could be favoured, in a way that improves overall coherence of the co-design process and product. A new generation of effective design rationale systems in HCI should take into consideration how designers actually remember in order to be fully integrated into co-design practices and do not represent design records detached from the actual co-design activity.

Figure 1 .

 1 Figure 1. Plan of the design studio in Barcelona.

Figure 2 .

 2 Figure 2. Example of MRS: Actions in shot two and shot three of the commercial

ACKNOWLEDGMETS

We gratefully acknowledge the support of the European Commission (P7-PEOPLE-2012-IEF-MC-IEF 326885). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.