Suguru Goto
email: suguru.goto@ircam.fr

The Case Study of An Application of The System, "BodySuit" and "RoboticMusic" -Its Introduction and Aesthetics

Keywords: Robot, Gesture Controller, Humanoid Robot, Artificial Intelligence, Interaction

This paper is intended to introduce the system, which combines "BodySuit" and "RoboticMusic," as well as its possibilities and its uses in an artistic application. "BodySuit" refers to a gesture controller in a Data Suit type. "RoboticMusic" refers to percussion robots, which are applied to a humanoid robot type. In this paper, I will discuss their aesthetics and the concept, as well as the idea of the "Extended Body".

INTRODUCTION

The system, which I introduce in this paper contains both a gesture controller and automated mechanical instruments at the same time. In this system, the Data Suit, "BodySuit" controls the Percussion Robots, "RoboticMusic" in real time. "BodySuit" doesn't contain a hand-held controller. A performer, for example a dancer wears a suit. Gestures are transformed into electronic signals by sensors. "RoboticMusic" contains 5 robots that play different sorts of percussion instruments. The movement of the robots is based upon the gestures of the percussionist.

Working together with "BodySuit" and "RoboticMusic," the idea behind the system is that a human body is augmented by electronic signals in order to be able to perform musical instruments interactively. This system was originally conceived in an art project to realize a performance/musical theater composition.

This paper is intended to introduce this system as well as the possibilities from my experiences in an artistic application.

General Description

This system is intended to be utilized in a project, which is entitled, "Artificial Body and Real Body." The theme is to explore this dualism and the relationship between artificiality and reality of human body in a context of musical theater. Artificiality and reality sometimes seem to be conflicted with each other, but they can work together, or their meaning can be transformed for an audience depending on the context. The context provokes the audience to play with the ideas of reality and artificiality. A performance involving "RoboticMusic" and "BodySuit" challenges the audience by confusing the line between virtual and reality. I am a composer and intend to create this composition, which emphasizes the importance of performance aspects with this system. This project originally started in 2002. This system is intensively experimented with and was shown on several occasions during 2005. The last performance was realized in, "Utopiales," a festival in Nantes, France in November, 2005 (Fig; 1). Fig. 1: A performance with "BodySuit" and "RoboticMusic". A photo from its rehearsal. © Utopiales "BodySuit" was first created by an electronic engineer Patrice Pierrot, in 1997. Although it was originally conceived to work with "RoboticMusic," it had to wait many years until "RoboticMusic" was ready. Meanwhile, many possibilities of "BodySuit" were explored, for instance, it was experimented with to control computer generated sounds and video images (Fig. 2).

Fig. 2: BodySuit can also control sounds and video images in real time.

"RoboticMusic" was created in 2003. The original concept and the design were done by me, and the robots were realized by a humanoid robot specialist, Fuminori Yamazaki, of the iXs Research Corporation in Japan. The project is still a work in progress, the goal is to eventually form a robot orchestra.

A gesture of performer with "BodySuit" is translated to gestures of "RoboticMusic." Instead of having a computer-generated sound, one can interactively have an acoustic percussion sound.

One of the important elements is the relationship and the communication method explored within this system. One may consider "BodySuit" and "RoboticMusic" as a relationship between a conductor and an orchestra, where dancelike gestures merely trigger instruments. In other words, this is an instrument that relies on physical gestures.

Another point is the method of translation used by the computer. For example, signals from "BodySuit" are transformed by Mapping Interface and Algorithm in a computer, and then are sent to "RoboticMusic." One gesture may trigger one attack on one instrument. However, it is also possible to trigger 5 instruments at the same time. Otherwise complex musical data, which is automatically generated by a computer and then reproduced by "RoboticMusic," is altered by gestures with "BodySuit" to modify the parameters of algorithm in real time.

Detailed Description of Percussion Robots -"RoboticMusic"

RoboticMusic contains 5 robots (Fig. 3), which play percussion instruments, such as a Gong, Bass Drum, Snare Drum, Tom-Tom, or Cymbal. These instruments can be replaced as long as the replacement instruments can be played with Mallets (Fig. 3).

© R a p h a ël C h i p a u l t

One of robots plays numerous pipes, and rapidly spins to create Flute-like sounds, which are generated while the air goes through them. These pipes are different lengths according to the pitches one desires. As it spins faster, the pitches become higher as following an overtone series (Fig. 4). The latest technology of humanoid robots is applied to this, but "RoboticMusic" doesn't walk on two feet, nor does it contain eyes, a mouth, etc. "RoboticMusic" does contain robot's arms. The gesture of a human percussionist is modeled in order to have musical sound and expression. Yet a robot can perform without any rest, more precisely and faster than a human being.

Max, Cycling'74 is utilized as an interface and to generate musical data. With this, one can also send basic parameters to the robots, such as a position of the robot's arm, an offset position, intensity (how hard it hits) and so on. This sends the signals to another computer with Linux via UDP. A software in Linux is developed by iXs Research Corporation. This has an important roll, since it controls the movement of robot. From the computer with Linux to the robots, these are connected via USB. Each robot has its own interface, which is connected with an actuator and a sensor.

The robot has a special sort of springs to imitate a human muscle. Each holds a mallet at the end of his arm (Fig. 5). Fig. 5: There is a special sort of springs in the arm of the robot. At the end of this, it holds a mallet.

The major advantage to "RoboticMusic" is that it interactively plays an acoustic instrument with the aide of a computer. There is no problem to play complex rhythm and it easily goes beyond the limit of human performance capabilities. Therefore, it gives new potentialities in a composition for acoustic instruments.

Another point is an acoustic sound. While a computer generated sound has many capabilities, an acoustic instrument has rich sonority and enormous possibilities of expression, especially from the point of view of a composer. When it is played on a stage, the vast possibilities of the acoustic aspect are obvious when compared to sound coming from speakers. Another benefit is that the audience may observe both sound and its gesture of performance.

To master one instrument is huge task for a musician, but to play together with others in an ensemble is another difficulty. Having 5 robots, one may extend the new possibilities of ensemble. For example, "RoboticMusic" allows 5 different tempos at the same time, or intricate accelerando and ralentando, but theses are exactly synchronized in music.

There is not only an artistic advantage with "RoboticMusic", but also a research aspect. As one works more with a robot, which works with the gestures of a musician, one can discover how a human gesture contains complex movement, although it sometimes looks fairly simple, for instance, the gesture to hit a percussion instrument. A musician knows how to play an instrument, but he may find it difficult to explain exactly how he controls each part of his muscles and bones, and how he increases and reduces speed and intensity instinctively within a very short instant.

When one hears the word, "robot", one is perhaps reminded of an industrial robot, or maybe sometimes a robot in a science fiction movie. However, it is not the case here. This is due to the latest development of artificial intelligence and is the case of application to hardware. This has a lot to do with the robot, which performs instruments with a human-like gesture. In particular it refers to the humanoid type of robot that contains sensors and advanced programming, which allows the robot to control itself automatically. It differs from the slave type robot in a factory, and at last we can profit from this in the field music. One may consider these robots as collaborators with humans.

Detailed Description of Gesture Controller -"BodySuit"

"BodySuit" has 12 sensors, which are placed on each joint of the body, such as a wrist, an elbow, a shoulder on the left and right arm an ankle, a knee, and the beginning of the left leg and right leg. The bending sensors are placed on the outer sides of the arms and on the front sides of the legs and fixed on a suit. Each sensor is connected with a cable to a box, and then it is connected with A/D interface.

A performer wears this suit, but doesn't hold a controller or any instruments in his hands (Fig. 6). Therefore, his gesture doesn't have to be based upon playing an instrument, but could be liberated to become a larger gesture, like a mime. This allows for collaboration with a person in a different field, for instance a dancer or an actor.

The audience easily observes this larger movement. That is to say it can be well adapted to a performance and musical theater situation.

One may consider this as a body instrument. This efficiently works in a percussionist-like gesture. This is one of the best controller conjunctions with RoboticMusic.

Since this is not like a physical controller or instrument, which is held by hands, it allows to be collaborated with the idea, " Augmented Body" or "Extended Body" in the work. His body is amplified by electric signals to control something remotely or to be extended from his abstract gesture to a meaningful gesture.

The system in "Augmented Body and

Virtual Body"

The system of BodySuit and RoboticMusic is much explored within the last work, "Augmented Body and Virtual Body"(Fig. 7). In this work, the gestures with "BodySuit" are translated or altered by the algorithm in a computer, and then are sent to "RoboticMusic." The interesting point is an idea of programming in order to alter one gesture to another state. For example, one single movement with a left elbow appears to be hitting a percussion instrument in the air. This triggers "RoboticMusic," which plays 5 percussion instruments from left to right in a space with a gradual slight delay on each. However, with the rising of a right shoulder, it changes the amount of delay to play in the order of left to right alternatively. The other case is one gesture translated into one gesture. A gesture with an arm in "BodySuit" is completely imitated by an arm of a robot, like someone copying another person. The method of communication has a great deal of importance here. In this sense, "BodySuit" and "RoboticMusic," two of them, but with a computer, three of them should be perhaps regarded as one system.

Since this is done in stage work, the fact, "gesture vs. gesture" should be much considered. The gesture with "BodySuit," which doesn't emit any sound by itself, is related with the gesture with "RoboticMusic," which is intended to create sound. In terms of interaction, the visual aspect between one gesture and another gesture provides clearer feedback and brings a different and interesting dimension. Incidentally, this communication does not refer to the one between the first person to another. In this case, the communication means the point of view from a third person to observe this relationship objectively.

Application of work

This system was applied to a performance/musical theater work in a composition (Fig. 8).

Fig. 8: This system is applied to a music theater work. There is a 3D image (by Yann Bertrand) behind RoboticMusic. © Utopiales At the end, this provides a lot of further and newer possibilities than I originally expected. While the communications of gesture with "BodySuit" and the gesture with "RoboticMusic" are observed, one notices different phases, which are the interaction and its perception, and the interaction with its consciousness. These two poles are the important keys in this field. With this articulated visual and oral experience in this work, one may cognate different experiences that constantly deal with something to expect, to understand, to notice, and to perceive.

Furthermore, the relationship between gesture and sound are also regarded by a very different view with this system. In other words, the idea, "music to see, visibility to hear" brings a different context in a theatrical performance.

While the concept, "Extended Body" was conceived to be realized with this system, the theme "Augmented Body and Virtual Body" is meant to question what a human body is and what his own identity is with this. Man and Machine seems to be dualistic, in which one may think that these are conflicted each other. Here, they coexist within this system, in fact, it is more correct to say that these are regarded as being one, "Extended Body". Therefore, our identity is not merely within our own body, but may communicate with outside and may be extended.

Conclusion

Historically, art has been always relating with a society where it exists, and has been always profiting from contemporary culture. As mentioned earlier, the robot here means the application of the latest development of artificial intelligence. Robotized instruments may have a lot of possibilities with this. At this point, this "Extended Body" is a reflection of society, especially where I grew up, in Japan. This is not an abstract image, but is just practically realized technically and aesthetically.

These technical possibilities and the aesthetic points have created further new potentialities by this system, however, this is not my only goal. While interacting with this system and myself, creating a new language and its perception are the most important goal in terms of a composition.

Furthermore, one may consider again that his own identity is not merely limited within his possessed body. While doing this "Extended Body", one may perhaps ask, if such an identity really exists or not, since there is not much limit or borderline to extend. Music has to progress anyway, or it goes further by itself, whether we want it to or not. Likewise, a definition of music is limitless. Perhaps co-existence with this "Extended Body" may help to develop new possibilities in the composition of music.

Fig. 3 :

 3 Fig.3: RoboticMusic. From the left to right, Gong, Bass Drum, Tom-toms, Snare, Pipes. © R a p h a ël C h i p a u l t

Fig. 4 :

 4 Fig.4: Pipe Robot appears behind the Cymbal Robot on this photo. Pipe robot changes the pitches according to the speed of spins.

Fig. 6 :

 6 Fig.6: Upper half of body with BodySuit. The 12 bending sensors are placed on each joint of body.

Fig. 7 :

 7 Fig.7: The performance of "Augmented Body and Virtual Body" in Nantes, France, in November, 2005. © Utopiales

Acknowledgments

I would like to thank Fuminori Yamazaki (iXs Research Corporation), Patrice Pierrot, Alain Terrier (IRCAM), and Patrick Gyger (Utopiales) for their assistance in the realization of this project. I also thank Tom Johnson and Andrew Stewart for their useful comments on the manuscript.


Umlauf David

Michal Sobecki

Laure Crabbe

Methyl Adenine Identification (MadID): High-resolution detection of protein-DNA interactions

Keywords: MadID, DamID, m6A, proximity labelling, M.EcoGII, Telomeres, Centromeres, LADs, Dotblot, High-throughput sequencing

Mapping the binding sites of DNA-or chromatin-interacting proteins is essential to understand many essential biological processes. Methyl Adenine Identification (MadID) is a proximity methylation-based assay that allows the visualization, quantification, and identification of binding sites from DNAinteracting proteins in eukaryotic cells. Chromatin-binding proteins of interest are fused to the newly described bacterial methyltransferase M.EcoGII. This enzyme catalyses the methylation of adenine residues with no sequence specificity. Consequently, adenines within and in the vicinity of the protein binding sites will be decorated with a methyl group (m6A), modification that can be further detected using different methods. M.EcoGII-dependent DNA methylation can be monitored in situ using immunostaining, at the genome-wide level using a combination of m6A-specific immunoprecipitation and whole-genome sequencing or locally at DNA regions of interest purified by chromatin immunoprecipitation or probe-based capture techniques. MadID is conceptually similar to DNA adenine methyltransferase identification (DamID) that relies on the methylation of GATC motifs. However, MadID provides a higher resolution, deeper coverage, and open the ways to identification of binding sites in genomic regions that were largely inaccessible such as telomeres, centromeres and repeated elements.

Introduction

During interphase, chromatin folding in the three dimensional space of eukaryotic nuclei as well as the composition of the chromatin fiber itself are key to understand genome regulation. Chromatin immunoprecipitation (ChIP) has long been the method of choice to characterize interactions between proteins and DNA allowing access to chromatin composition [START_REF] Orlando | Analysis of chromatin structure by in vivo formaldehyde crosslinking[END_REF]. However, ChIP entirely relies on the availability of ChIP-grade antibodies and is incompatible with the detection of transient interactions.

Here, we present MadID, a proximity-labelling technique that circumvents these limitations and allows the mapping of protein-DNA interactions with various experimental setups [2].

MadID is an optimized method based on DNA adenine methyltransferase identification (DamID), which has emerged as a comprehensive technique to map genome-wide occupancy of proteins of interest [3]. DamID uses the ability of the bacterial Dam methyltransferase to methylate adenine residues within GATC motifs. When fused to a chromatin binding protein in an inducible system, this enzyme targets the methylation of GATC motifs residing within and in the vicinity of the protein binding sites. DamID has been instrumental in unveiling nuclear organization of specific parts of the genome such as Lamin Associated Domains (LADs) [4][5][6] and has also been successfully applied to study genome organization on a global scale (see for example [7]). Overall, DamID is a powerful approach that allows the study of chromatin composition and its dynamics. However, it is inherently limited by its dependence on the distribution of the GATC tetrameric recognition site, which represents only 0.9% of the human genome [2]. In addition, several genomic regions are deprived or poor in the GATC motifs required for Damdependent methylation, such as AT-rich regions, telomeres, or centromeres, which are therefore blind to DamID.

To overcome these limitations, we took advantage of the newly described M.EcoGII methyltransferase from E. coli to implement MadID. Unlike site-specific methyltransferases such as Dam, M.EcoGII methylates adenine residues in any DNA sequence context with a high efficiency [8].

Consequently, GATC-null or GATC-poor regions are now fully accessible to MadID, which constitutes an unbiased strategy to map protein-DNA interactions with a deeper coverage and higher resolution [2].

Fused to a protein of interest, M.EcoGII drives adenine methylation (m6A) at the protein binding site, methylation that can be further analysed by a variety of methods. M.EcoGII was successfully targeted in human cells to telomeres, centromeres, and the nuclear envelope using fusion proteins with TRF1, Centromeric Protein C, and LaminB1, respectively [2] (plasmids available at Addgene, see materials).

However, MadID can be implemented in any model organism in which transgenesis is feasible, and for any protein of interest provided the fusion with M.EcoGII remains functional.

In this method chapter we present a detailed protocol of MadID, from the generation of cells expressing the M.EcoGII-fusion proteins using retroviral transduction, to the different readouts available to assess, quantify or map DNA methylation: i) m6A detection in situ by indirect immunostaining using a m6A antibody, including a protocol to combine this detection with DNA-FISH, ii) genome-wide m6A detection after m6A-immunoprecipitation (m6A-IP) followed by high-throughput sequencing, and iii) detection of m6A on a dotblot (m6A-dotblot) with total genomic DNA or after purification of a particular DNA regions of interest. We provide the method for telomere purification using a probe-based capture technique as an example.

12. Wash the beads once with wash buffer II.

13. Resuspend bead in 50 µl of elution buffer and elute telomeric DNA by incubating at 50°C for 20 minutes gently rotating the tube. Remove the beads and save the eluate.

14.

Step 13 may be repeated once (optional).

15. Measure DNA concentration using Qubit. [START_REF]Reveal the membrane using the Western ECL substrate according to the manufacture's instruction. Chemiluminescence signal obtained is analysed using the ChemiDoc Imaging System[END_REF]. Perform the dotblot as indicated in steps 3.8 (see note 18).

4

Notes.

1. M.EcoGII constructs are stably expressed in cells using retroviral transduction. Prepare one 10cm dish of Phoenix for each transduction you wish to perform.

2. Prepare one 10cm dish for each transduction you wish to perform.

3. After retroviral cell transduction, the expression cassettes will randomly integrate into the host genome for stable expression. The M.EcoGII constructs that we generated (see material section) were cloned into the pRetroX-PTuner plasmid (Clontech) that allows the regulation of the amount of the expressed protein of interest. The vector encodes a 12 kDa destabilization domain (DD) that is expressed as a N-terminal tag on all M.EcoGII constructs and causes their rapid degradation via the proteasome. The DD-tagged protein can be rapidly stabilized by addition of Shield-1, a permeable molecule that will bind the DD tag and prevent the proteasomal degradation. Although we found that 6-7 hours of induction was sufficient to induce significant methylation of genomic DNA, experiments that do not require short induction time should be carried out after 24 hours of induction to increase the signal to noise ratio. It is important to note that constitutive overexpression should be avoided as it will result in permanent m6A methylation. We strongly advise to include a condition with no Shield-1 treatment to test for background methylation and efficiency of the induction. As Shield-1 is solubilized in Ethanol, add the corresponding amount of Ethanol in the non-induced samples.

4. Fixation is a crucial step. Extended fixation may result in antigen masking therefore decreasing the primary antibody efficiency. Also, for some specific proteins the percentage of paraformaldehyde used and/or fixation time may be adapted.

5. m6A immunofluorescence when used to detect methylated DNA requires an RNase treatment. Indeed, mRNA m6A methylation is a very abundant modification in eukaryotic cells. If not removed it will automatically result in a high fluorescence background masking the localization of methylated DNA.

6. This step will allow a better accessibility of m6A by the anti-m6A antibody.

7. This step is required to fix the antibodies for subsequent PNA hybridization steps.

8. At this stage, the coverslips can be stored for a couple of days in the dark in 1X PBS at 4°C. 9. Any other cell counting apparatus can be used. The key point is to have a clear idea of the amount of cells harvested for each condition. For example, an excess of cells may impede genomic DNA extraction by cloaking the columns used. Conversely, a low starting amount of cells may result in a sub-optimal DNA yield for high throughput sequencing or a suboptimal telomere yield at the end of the Telocapture hence resulting in a poor signal on the slot-blot membrane.

10. At this stage, the cell pellets can be stored at -20°C for up to 2 months.

11. Using commercially available kits for genomic DNA extraction is a time saver. However, standard protocol can also be used.

12. At this point, genomic DNA can be stored at -20°C for a least a year.

13. We perform this step in the lab using a Bioruptor Plus apparatus from Diagenode equipped with a water cooler. Sonication was performed using the following settings: 40 cycles 30s ON, 60s OFF, low power.

14. As mentioned in note 6, the m6A specific antibody has a better affinity for m6A on denatured DNA. As Illumina adaptors need to be added to double-stranded DNA, the steps of End-repair, A-tailing and ligation need to be performed before the immunoprecipitation step.

15. The unbound fraction can be collected as well for further analysis if needed.

16. i) To load DNA on the membrane using the dot-blot apparatus, the volume of the sample should ideally be between 100 and 200 µl. We don't recommend loading less than 50 µl, as air bubbles can form in the well that may result in unequal spreading of the DNA. ii) The amount of genomic DNA to load can vary between experiments. We recommend loading several amounts to test the linearity of the m6A signal later on. 17. It is good practice to check digestion efficiency by running an aliquot of digested DNA (usually 1-2 µg) on a 2% agarose gel.

18. It is possible to assess telomere capture efficiency using a Q-PCR approach (Cawthon et al).

Fig1:

  Fig1: Migration patterns of sonicated or digested human genomic DNA. (A) 1µg of human genomic

Acknowledgment

We would like to thank all the members (past and present) of the Crabbe's lab for helpful discussions especially Sonia Stinus Ruiz de Gauna for critical reading of the manuscript. This work was supported by an ATIP starting grant from CNRS in the framework of Plan Cancer 2014-2019 (to L.C.), the ANR Tremplin ERC teloHOOK (ANR-16-TERC-0028-01 to L.C.), and a European Research Council (ERC) grant (TeloHOOK, 714653 to L.C.).