Letizia Jaccheri
email: letizia.jaccheri@ntnu.no

Deepti Mishra
email: deepti.mishra@ntnu.no

Siv Hilde Houmb
email: siv.houmb@ntnu.no

Aida Omerovic
email: aida.omerovic@sintef.no

Sofia Papavlasopoulou

SikkerhetsLøypa -Knowledge toward Sustainable and Secure Paths of Creative and Critical Digital Skills

Keywords: Digital Society, Online Behavior, Video Games, Security, Gender

Children spend numerous hours on the Internet daily. While online, they meet a great number of opportunities as well as risks. Of these risks, cyber bullying and privacy violations are of major concern, in addition to exploitation and child pornography. Our hypothesis is that the solution is not to keep children and teens away from the Internet, but to ensure that young citizens are empowered with the necessary knowledge and skill set to become critical consumers and creators of new secure and sustainable digital services and products. Our objective is to develop a knowledge and skill set base and offer learning through playful solutions for empowering children and young people with creative and critical digital skills, in an engaging and motivating way. The aim is to build on the method and lessons learns of Kodeløypa, one of the scientific offerings for children at the Norwegian University of Science and Technology (NTNU), and the related scientific efforts made to empower a new generation of online users to avoid risks in the modern digital society. The project, SikkerhetsLøypa, is highly inter-disciplinary and spans across the fields of information security, user experience, software engineering, and computer science education. The scientific results of this project aims at strengthen education methods and practices in secure and privacy-aware behavior in the digital world.

Introduction

The digital literacy level of children and young adults has increased significantly in recent years. Unfortunately, their digital safety and privacy skills have not kept up with the digital literacy level. The wider use of the Internet to seek information, share ideas, consume entertainment and network using social media brings great opportunities, as well as larger risks to young users. Despite a growing body of rules and regulations implemented to enhance safe online behaviour, more efforts in this direction are still needed. Children require support to be aware of the potential risks and threats of Internet use for their online security. Staksrud and Livingstone [START_REF] Staksrud | Children and online risk[END_REF] define online risks as a set of intended or unintended experiences that increase the likelihood of harm to the Internet user, and include encountering pornographic, racist or hateful content online, as well as inappropriate or potentially harmful contact via harassment and bullying. Online risks to children generally comprise a set of wanted or unwanted inappropriate activities by children (as actors, receivers, or participants) that are of concern [START_REF] Teimouri | Assessing the Validity of Western Measurement of Online Risks to Children in an Asian Context[END_REF]. Child pornography is addressed by the Cybersecurity SRA [START_REF]Cybersecurity Strategic Research Agenda -SRA[END_REF] as one of their list of serious cybercrimes and malware. Everyone, child or adult, has the right to the protection of their personal data, but many young people normalise the sharing of intimate details on social media without understanding the risks, and possibly with an overestimation of their governments ability to protect them from online risks Parents can play an important role in providing a safe Internet environment for their children. However, it is not easy for parents to engage effectively in an instructive mediation of their child's Internet use, and co-viewing is much less likely when it comes to Internet compared with other audio-visual mediums such as television. Therefore, proper training and teaching is crucial for imparting online security-related life skills. This is further supported by Reid and Van Niekerk [START_REF] Reid | Back to Basics: Information Security Education for the Youth via Gameplay[END_REF], who assert that cyber security education is becoming a necessary precaution for individuals to protect themselves against the dangers of online technologies and resources. It has also been observed that learning activities that employ gamification [START_REF] Stanescu | Using Gamification Mechanisms and Digital Games in Structured and Unstructured Learning Contexts[END_REF] generally lead to better learning outcomes. However, gaming brings the gender challenge to the table, which needs to be considered when using gaming as a learning platform for online security-related life skills. Although women remain under-represented in cultural representations of gamers, in the design considerations of game producers, and in the production companies themselves, the percentage of girls and women playing and creating video games is continuing to increase. Female gamers tend to play as much as male gamers or longer online, per session and per week than males [START_REF] Hussain | Online gaming addiction : Classification, prediction and associated risk factors Online gaming addiction : Classification , prediction and associated[END_REF]. As women have become more present online and in the Internet economy, this brings challenges such as becoming the prey of cybercrime and bullying, arguably at a higher level of risk or different manner of risks than for men.

The paper describes work in progress. Initially a series of workshops were conducted in the framework of Koderløypa (see section 3) with the aim to explore using gamification as a learning platform. With the above-mentioned motivations and the results of these workshops, the present study describes an ongoing project, SikkerhetsLøypa, with the aim to create new knowledge, methods, and tools that will lead to new services and products by gamifying education on online opportunities and risks with innovative gaming technologies for young online users.

The paper is organized as follows: Section 2 describes the challenges and concerns regarding digital safety and security of young users, along with gender issues in gaming. Section 3 provides a walk-through of the Kodeløpa workshop and method. The proposal to develop knowledge and offer playful solutions for empowering children and young people with creative and critical digital skills is discussed in Section 4. Section 5 provides discussion and conclusions.

Literature Review

A European study of 21 countries stated 75% of children are using the Internet [START_REF] Livingstone | Comparing children's online opportunities and risks across Europe[END_REF].

Another study finds that more than 65% of Facebook and Myspace users are children [START_REF] Lenhart | Teens and Sexting: How and why minor teens are sending sexually suggestive nude or nearly nude images via text messaging[END_REF]. This increasing Internet usage among children brings both opportunities and risks for those children [START_REF] Livingstone | Comparing children's online opportunities and risks across Europe[END_REF]. This means that the more opportunities children gain online, the more likely they are to encounter increased risk [START_REF] Livingstone | Children's online risks and opportunities: Comparative findings from EU Kids Online and Net Children Go Mobile Executive summary[END_REF].

Online Safety and Security Issues

A research network founded by the European Commission's Safer Internet Programme, known then as 'EU Kids Online' and now as 'Better Internet for Kids', developed a classification [START_REF] Hasebrink | Patterns of risk and safety online[END_REF] of online risks comprising: Content risks where the child is a recipient of unwelcome or inappropriate mass communication; Contact risks where the child participates in risky peer or personal communication; Conduct risks where the child themselves contributes to risky content or contact. Lorenz et al. [START_REF] Lorenz | A Model to Evaluate Digital Safety Concerns in School Environment[END_REF] identified that the challenges or concerns regarding digital safety and security are related to reputation (self-inflicted damage, outside damage), data (data loss, data exposure), fraud (dishonesty, money loss), health (physical and mental health factors), and freedom. Some major security risks in relation to online behaviour of children are as follows:

Children are trusting. The number of children believing everything they read on the Internet is significantly increased, according to an Ofcom study which has found that 'digital natives' are too trusting of what they find online [START_REF] Ofcom | Children Believe Everything they Read Online[END_REF]. Online grooming. Online grooming is another major problem in online communities where groomers often pretend to be children in order to become friends and establish a relationship with their young victims [START_REF] Ashcroft | A Step Towards Detecting Online Grooming --Identifying Adults Pretending to be Children[END_REF].

Unwanted exposure to sexual material. Children are more likely to experience unwanted exposure to pornography [START_REF] Teimouri | Assessing the Validity of Western Measurement of Online Risks to Children in an Asian Context[END_REF]. Furthermore, boys experienced more exposure to the risks compared to girls [START_REF] Teimouri | Assessing the Validity of Western Measurement of Online Risks to Children in an Asian Context[END_REF]. Unwanted exposure to violent content. Children are exposed to potential harmful risks such as seeing bloody movies or photos, seeing people being beaten up, and seeing hate messages [START_REF] Teimouri | Assessing the Validity of Western Measurement of Online Risks to Children in an Asian Context[END_REF]. Contact risks. This includes contacting someone online whom the child never met faceto-face, meeting someone face-to-face whom the child knows only online, meeting someone whom the child knows only online and being harassed [START_REF] Teimouri | Assessing the Validity of Western Measurement of Online Risks to Children in an Asian Context[END_REF]. Sharing personal details. Children may share and reveal personal data because they do not realize the possible consequences [START_REF] Tsirtsis | Cyber security risks for minors: A taxonomy and a software architecture[END_REF]. Minors are more likely than adults to give out personal information in order to receive an award [START_REF]Implementing the Childrens Online Privacy Protection Act: A Report to Congress[END_REF]. Eight out of 10 adolescents who use social networks share personal information about themselves to a much greater extent compared to previous years [START_REF] Tsirtsis | Cyber security risks for minors: A taxonomy and a software architecture[END_REF], revealing sensitive information about their family and friends as well as themselves [START_REF] Marwick | Youth, privacy and reputations[END_REF].

Online bullying/being bullied. According to Ktoridou et al. [START_REF] Ktoridou | Exploring parents' and children's awareness on internet threats in relation to internet safety[END_REF], one in three children have experienced Internet bullying. 38% of girls and 26% boys have faced online bullying [START_REF] Ktoridou | Exploring parents' and children's awareness on internet threats in relation to internet safety[END_REF].

Internet addiction. Overuse and addiction to the Internet among children has been examined by many researchers [START_REF] Gentile | Pathological video game use amongst youths: A two-year longitudinal study[END_REF][START_REF] Huang | Mental health, personality, and parental rearing styles of adolescents with internet addiction disorder[END_REF][START_REF] Kwisook | Internet overuse and excessive daytime sleepiness in adolescents[END_REF]. Several studies [START_REF] Gentile | Pathological video game use amongst youths: A two-year longitudinal study[END_REF][START_REF] Lemmens | Development and validation of a game addiction scale for adolescents[END_REF][START_REF] Van Rooij | Online video game addiction: Identification of addicted adolescent gamers[END_REF] have also investigated online gaming addiction among school-going adolescents. Digital safety areas are scattered across every element defined in the above list, for example: understanding internet safety trends; choosing secure devices to surf online; recognize potential insecure behaviours or threats; knowing how to act when something bad happens or how to seek help when needed; perceiving and judging the dangers of your own and others' online behaviour; knowledge about account privacy and maintenance; helping students to learn how to interact online positively and with consideration for others [START_REF] Lorenz | A Model to Evaluate Digital Safety Concerns in School Environment[END_REF].

There are also major differences in online behavior depending on the age of children and young people. Each of these demonstrable differences in experience of the Internet and online gaming, and it is critical to take this into consideration when developing tools and techniques to improve online security skills for children.

Gender Issues in Gaming

Gender issues in technology and specifically in computer games have been a research topic for many years. Twenty years have passed since Cassel and Jenkins wrote the groundbreaking "From Barbie to Mortal Kombat" that highlighted the ways gender stereotyping and related social and economic issues characterize digital game play. Kafai et al. [START_REF] Kafai | Editors: Diversifying Barbie and Mortal Kombat, Intersectional Perspectives and Inclusive Designs in Gaming[END_REF] have recently edited a collection of contributions around the still-relevant question about women and gaming. Video games are an important part of the life of adolescents. According to Gorriz and Medina [START_REF] Gorriz | Engaging Girls with Computers through Software Games, Commun[END_REF], boys reported playing an average of nearly 43 h per week, or over 6 h per day, and girls reported playing nearly 30 h per week, or over 4 h per day. According to Fron et al. [START_REF] Fron | The hegemony of play[END_REF], 88.5% of all game development workers are male; 83.3% are white; 92% are heterosexual. Bryce and Rutter [START_REF] Bryce | Gender Dynamics and the Social and Spatial Organization of Computer Gaming[END_REF] have argued that 'the popularity of domestic and online gaming among females, and the development of female gaming clans, highlights that leisure activities and spaces are becoming less gendered, and can provide sites for resistance to societal notions of the gender appropriateness of leisure activities. In the years since that study was published, female video game players have long since ceased to be a negligible minority. Women are also actively creating video games-not only as employees of computer game production companies but in a few notable cases as the owners of those companies as well. Women-owned computer game companies include HerInteractive, Girl Games, Girltech, Purple Moon (developed by Brenda Laurel, who had established Purple Moon games with the explicit goal of designing products which reflected sociological and ethnographic research into young girls' play patterns). Gorriz and Medina [START_REF] Gorriz | Engaging Girls with Computers through Software Games, Commun[END_REF] provides an excellent classification of games for girls from Barbie Fashion Designer to tools for boosting self-esteem such as Let's Talk About Me.

The gender aspect of Internet and online situational awareness is important, and is related to questions around users' age and maturity. It is important to understand these gender differences and to develop a training program that is both tailored to the gendered experience of learning (women learn differently than men) and the differences in the cyber situations that girls and boys (or women and men) could be exposed to.

Kodeløypa Workshop

Building upon several documented efforts [START_REF] Giannakos | Happy girls engaging with technology: assessing emotions and engagement related to programming activities[END_REF][START_REF] Giannakos | What motivates children to become creators of digital enriched artifacts?[END_REF], we designed a one-day workshop program for 15-years old students in secondary schools in Trondheim, Norway. Students attended this five-hour workshop as part of their school day [START_REF] Papavlasopoulou | Creative Programming Experiences for Teenagers: Attitudes, Performance and Gender Differences[END_REF]. We adopted the constructionist approach as one of its main principles is learning by making. This approach was chosen because of the observation that young people of that age tend to rebel against concrete advice, and learn better through engaging with educational content in a creative and self-directed way. We have chosen to use Scratch (a digital learning environment) due to its ease of use for young students and its connection to the principles of constructionism. For a hardware platform, we selected Arduino due to its well-established and smooth integration with Scratch. We also used Scratch for Arduino (S4A), an extension of Scratch that provides extra modules for robot control.

Fig. 1. Kodeløypa workshop

Five teaching assistants facilitated the workshops and designed the process together with a researcher and an artist. In general, children who attended the workshop worked collaboratively in triads. Digital artifacts (robots) were placed next to each of the computers, and the students had the option to select the robot they wanted (figure 1).

The workshop was organized in the following two main sections.

Playing (and Learning) with Robots

At the beginning of the workshop, one assistant welcomed the students and presented the scope of the workshop to the students. Then, the assistant demonstrated the robots and advised students to keep their attention on the tutorial placed next to each of the computers. The tutorial contained instructions with examples and pictures similar to the robots they were using. Students were asked to investigate the robots and find the exact place of their sensors and lights, write the location of the LED lights and sensors while answering some questions. Examples were deliberately sparse of text and imagerich, with images demonstrating exactly what the students were asked to do. As students had no experience with programming we wanted a smooth start that would increase their motivation to engage, so they were first asked to do a series of simple tasks making the robot react to the environment through visual effects (for example, cause a light to switch on when there was less light on the sensors). The robots were built from recycled computer parts/materials. Students could touch and play with the robots but not change any parts of them, since those robots have been constructed to support computational thinking (CT) concepts (i.e. sequences, loops, parallelism, events, conditionals, operators, variables, and lists). During this phase, students made simple loops that could control the robots. Through this activity, students understood the metaphors and interactions between the physical design elements and the Scratch environment. The first section lasted one and a half hours.

Create Games with Scratch

In the second part of the workshop, we wanted students to be creative and implement simple game development concepts using Scratch. The students were given another tutorial with the basic CT concepts, with images arranged in an order to help them with the development of their own game. This part lasted three hours and did not use the robots.

Students were able to create their own game structure and stories by designing and programming. It is very important that students had a plan for what kind of game they want to create. Therefore, they were asked to draft a storyboard first. In order to stimulate creativity, examples of different loops and existing game characters were given to students to be used as elements of the game. Examples and visualizations of the process helped students to ideate their own original project. Help was provided anytime students asked for it and the number of assistants was enough to provide sufficient help to all teams. More complex programming concepts were introduced on an individual level when the students asked for them and those concepts were relevant to their projects. Throughout the whole process, students were iteratively testing and trying to debug their games, and they worked collaboratively according to their own pace. At the end, each of the students' teams created a Scratch game based on their own preferences and ideas. Students' games were not presented formally, but all teams had the chance to play each other's games.

Participants

Seven KodeLøypa workshops took place during Autumn 2015, and students from four different schools participated. In total, 128 students attended, consisting of 60 male and 68 female students. All workshops followed the same structure as outlined above and each of the workshops had between 18 and 22 participants [START_REF] Papavlasopoulou | Creative Programming Experiences for Teenagers: Attitudes, Performance and Gender Differences[END_REF].

Data Collection and Measurements

A range of data was collected across the seven workshops, including surveys, photographs and observations. For the purpose of this study we focused on the quantitative data. To collect the quantitative data, we conducted a post-workshop survey based on questions adopted from similar studies in the literature. Students responded to the survey at the end of the workshop day. In summary, we collected 105 responses (53.3% males and 46.7% females). Each attitude (construct) consisted of three to four questions (items) and was measured using a seven-point Likert scale.

Lessons Learned

There were several lessons learned from the KodeLøypa workshop that form the basis for the planned SikkerhetsLøypa workshop series. In particular, the results showed that age and maturity heavily influence the learning capability. However, age and maturity level does not necessary align, which is an essential lessons learns when planning Sik-kerhetsLøypa workshops. For example, while 14-16 year old's have an equal need for the same level and types of online security awareness and skills, their readiness to receive and understand the workshop's training modules varied significantly based on their age and inconsistently age-linked maturity levels. The workshop could ideally be tailored to match the learning styles and maturity of each student, but the wide variability and rapid pace of change in teenagers' development makes it difficult to make useful assumptions about their learning preferences and abilities ahead of meeting the individual learners themselves.

Towards an Intersectional Understanding of Online Security, Gender, Creative Careers and Learning Styles

This project SikkerhetsLøypa lies at the intersection between a) the gamification of digital empowerment of children and young people and b) user-centered and adaptive training in cyber security risks and online opportunities.

Objective of the Project

The objective of the SikkerhetsLøypa project is to: create new knowledge, methods and techniques that will lead to products and services of gamified education about online opportunities and risks. This will enhance the level and competitiveness of the Norwegian education system and at the same time open up market opportunities for Norwegian publishing, gaming and e-learning industries, both nationally-and internationallyfacing. The targeted end users of these products and services are children at the level of the Norwegian middle school (Ungdomskole). The gamification of security training for youth is cross-disciplinary and the main research challenge is in the intersection between cyber security, user experience and learning science. The challenge is two-fold: (1) Identify and dynamically update the most recent knowledge on online opportunities and cyber security risks, including documented guidance on how to deal with them; and (2) Design and empirically validate the SikkerhetsLøypa products and services so that, as well as efficiently conveying the abovementioned up-to-date knowledge, they are attractive, highly motivating and entertaining to the users, so that kids actually want to use them and the teachers and policy makers want to adopt them.

Research Questions

We aim to investigate the following research questions with the aim to offer playful solutions for empowering children and young people with creative and critical digital skills, in an engaging and motivating way: How could we develop a security knowledge tale into a video game? How could we evaluate the video game?

The second question can be separated into two sub-questions: How do we evaluate the game after production? And how do we evaluate the game during the design and production phase to create a game that better achieves the creators' goals? The literature review will identify papers that can most usefully serve as guidance for choosing an evaluation model and methodology. Papers of relevance include examinations of evaluation methods previously applied to education-oriented video games, as well as suggestions for future evaluations of both the learning outcomes and the quality of engagement with video games among diverse audiences. It is very likely that findings related to how best to evaluate the video game will inform the methods for developing (designing and producing) the game.

The proposed work will contribute to the knowledge, methods and tools needed for: (1) acquisition of the cyber security knowledge that needs to be communicated to the user groups, (2) storytelling around the knowledge to be communicated, (3) dynamic content update of the SikkerhetsLøypa products and services with most recent knowledge and corresponding stories, and (4) adaptive learning which customizes the SikkerhetsLøypa products and services to the level and learning pace of the user.

Main Activities, Objectives and Deliverables

The main activities, objectives and deliverables of SikkerhetsLøypa are as follows: Knowledge management regarding online opportunities and risks: Develop an approach to the acquisition of knowledge of online opportunities and risks, as well as ways of dealing with online resources associated with both.

Serious game design and user experience: Develop an approach to interactive and motivating storytelling. Develop method and metrics for evaluation of usability. Develop an approach to updating the design of serious games with new content. Pedagogical solutions for game-based learning: Develop an approach to knowledge transfer through storytelling as well as guidance for when and how to leverage the different learning means and mechanisms. Develop method and metrics for evaluation of learning effects. Develop a method for adaptive learning. Empirical evaluation in pilot environments: Identify requirements for SikkerhetsLøypa products and services. Evaluate the feasibility of SikkerhetsLøypa early products and services for education about online opportunities and risks. Evaluate the effect of Sik-kerhetsLøypa products and services on user experience and learning. Conduct an overall evaluation of the final version of the SikkerhetsLøypa products and services. Dissemination and internationalization. Open seminars, participation in social media, scholarly articles. The pilot at NTNU and in schools will generate attention in the media. We mention that Kodeløypa has been broadcasted at NRK News and other National media channels. Kodeløypa is one of the main case studies in Horizon 2020 Umi-Sci-Ed [START_REF]Exploiting Ubiquitous Computing, Mobile Computing and the Internet of Things to promote Science Education[END_REF], and SikkerhetsLøypa will utilize existing relationships with the international partners and previous event/activity partners of Kodeløypa.

Conclusions

The rapid increase over recent years in the digital literacy level of children has not been equaled by a parallel increase in online situational awareness or digital security skills. This paper has provided a brief summary of statistics outlining increasing use of the Internet by young people, including girls. This paper has also explored in brief the awareness in the literature that the Internet ecosystem, and video games especially, are increasingly inhabited by girls and women, and that although women remain underrepresented in depictions of and opportunities in the gaming industry, the percentage of girls and women playing and creating video games is continuing to increase. In summary, exposure to the Internet provides a wealth of opportunities to young people, and therefore, rather than preventing exposure to the commensurate risks and dangers by favoring restriction of access, it would be more beneficial to encourage young people to become aware of those risks and dangers, and to equip them with tools to improve their own digital safety. A one-day workshop for school-age Norwegian children (age 15) was developed, taking a constructionist approach that used gamification to motivate students to learn about programming by playing with robots and designing their own simple video games. These "Kodeløypa" workshops were delivered to 128 students in total, with representative participation by different genders, and employed Scratch software as well as Arduino hardware. Quantitative feedback was effectively gathered from the majority of participants and can be used to improve workshop structure and learning outcomes of similar workshops in future. The Kodeløypa workshop model developed by our team is summarized here to provide evidence of the efficacy of this model of innovative learning, and to propose that a new workshop, called SikkerhetsLøypa, be developed based on the experiences and lessons learns from the Kodeløypa's workshops. The aim of the SikkerhetsLøypa workshop is to deal specifically with issues around online safety awareness and skills. The objective of the SikkerhetsLøypa project is to create new knowledge, methods and techniques that will lead to products and services of gamified education about online opportunities and risks.

To that end, we found that a number of building block studies will need to be developed in order to deliver a workshop that addresses the intersecting needs of school-age students of different genders, ages, maturities and learning styles. These studies, which will take the form of literature reviews, original research and iterative workshops, will deliver a new knowledge base that can be used to guide pedagogy, service creation and product development around teaching internet safety to young people.


Kazuma Nagata
email: nagata.kazuma@entcomp.esys.tsukuba.ac.jp

Denik Hatsushika
email: d_hatsushika@vrlab.esys.tsukuba.ac.jp

Yuki Hashimoto
email: hashimoto@iit.tsukuba.ac.jp

Virtual Scuba Diving System Utilizing the Sense of Weightlessness Underwater

Keywords: Scuba Diving, Head Mounted Display, Underwater, Virtual Reality

We propose a virtual scuba diving system using an underwater head mounted display (UHMD). This system has two advantages. First, a strong sense of reality is obtained by combining the weightlessness obtained by diving with a highly immersive image presented by a waterproof UHMD. Second, the system is very versatile, such that a user can experience the system in easily accessible waters such as swimming pools or shallow water. In this paper, we describe the design and implementation of the proposed system and the creation of underwater virtual reality (VR) content to provide a diving experience. The proposed system consists of the UHMD with a head tracking sensor, laptop PC, and a hose to protect the cables. The UHMD and laptop PC are connected by cables for power, communication, and video transmission. Each part is waterproof. The created VR content makes it possible to freely swim in the virtual undersea world by head tracking and operation of the controller. We conducted a pilot experiment to test the proposed system while users swam in a pool of about 4 m in depth. The UHMD was confirmed to have sufficient waterproofness at water depth of about 4 m. It is also suggested that it is possible to create the illusion that the water depth in the virtual underwater space is the user's own water depth.

Introduction

Scuba diving is an attractive sport that enables participants to enjoy a mysterious and beautiful underwater world. However, scuba diving cannot be enjoyed easily anytime. A license is required for free diving, and there is a limit to the depth of dives for safety.

In addition, it is necessary to consider the appropriate location, time, and weather to do scuba diving.

Our goal is to realize a virtual scuba diving system so that anyone can enjoy a diving experience with high reality at any time in familiar waters such as pools. Many systems to experience the underwater virtually have been proposed [1][2][3][START_REF] Dhruv | Amphibian , Immersive Terrestrial Scuba Diving Using Virtual Reality[END_REF]. However, many of them present underwater images while the user is not in the water. Therefore, it is difficult for these systems to present weightlessness and water resistance. In contrast, several virtual underwater experience systems in which the user is underwater have been proposed in recent years. DOLPHYN [START_REF] Abdelkader | Augmented reality for underwater activities with the use of the DOLPHYN[END_REF] is an underwater experience system equipped with sensors such as a webcam, GPS, thermometer, flowmeter, and tablet PC. Users can experience the virtual undersea world through the DOLPHYN display. However, this system has less immersion than a head mounted display (HMD). In addition, body movement is restricted because it is necessary to operate the equipment with both hands. AquaCAVE [START_REF] Shogo | AquaCAVE: An Underwater Immersive Projection System for Enhancing the Swimming Experience[END_REF]7] realizes the experience of swimming in the ocean by projecting images on the wall of the aquarium. However, the available places for experiencing this system are limited because of the necessity of preparing a dedicated aquarium. The movement of the user is also limited by the narrowness of the aquarium.

Hence, this research proposes a new virtual underwater experience system using an underwater HMD (UHMD). Several UHMDs have been proposed [START_REF]Nautilus VR Virtual Reality Underwater[END_REF][START_REF] Wenlong | A safe low-cost HMD for underwater VR experiences[END_REF][START_REF]Experience Weightless VR With This Waterproof Headset[END_REF]. These systems use a smartphone with a waterproof case, and users can enjoy a virtual sea world synchronized with the movement of their head underwater. However, because of the limitations of smartphone performance, the quality of the computer graphics (CG) that can be provided is not high. Moreover, there is a possibility of virtual reality (VR) sickness due to insufficient head tracking performance. Furthermore, the feeling of immersion is lower than with a dedicated HMD. In contrast, we aim to design and implement a system to solve this problem.

In this paper, we report on the design and implementation of our system, its content production, and the result of a verification of the system's operation in a pool.

System Configuration

Design Principle

Our system is designed to be used at a depth of about 4 m, in which the whole body of a person can move freely in the water. For this reason, we decided to make the system waterproof enough to withstand a water depth of 4 m. According to Oculus best practices [START_REF]Oculus Best Practices[END_REF], we tried to render CG at 75 fps or higher, and selected a sensor so that the tracking delay would be less than 20 ms. To address portability, we decided to design the system to a weight and size that can be carried by one person.

Implementation

Our system consists of a UHMD with head tracking sensor, laptop PC, and a hose that protects cables. The UHMD and laptop PC are connected by cables for power, communication, and video transmission. The use of a laptop increases the system portability.

The UHMD system is shown in Fig. 1. For the UHMD body, we disassembled an Oculus Rift DK 2 (Oculus VR, Inc.) and used the circuits and display. The cables were 10 m in length so that the system could move freely at a water depth of 4 m. For head tracking, the internal sensor of Oculus Rift DK 2 was used. The laptop GPU (NVIDIA, Geforce GTX 980M) was used for video output.

The UHMD was entirely covered with a circuit sealing agent. The connector was protected with a waterproof-type heat shrink tube. Hose was used to protect the wired cables. Further, to ensure close contact between the user and the UHMD, the frame of an underwater mask was joined with the HMD section (Fig. 2). We applied defogging liquid to the HMD's lenses.

Production of Content

Scene Design

We produced underwater content using Unity, which can create high-quality CG and is easy to link with external equipment. To achieve a good VR experience, we adjusted the number of 3D objects and effects so that the frame rate did not fall below 75 fps. We produced two diving environments. One is a shoal (Fig. 3, left). The depth of this scene is 5 m. The user can watch seaweed, coral, and a school of fish. The other virtual environment is large vertical hole (Fig. 3, right). The depth and diameter of this scene is 40 m. The surroundings become darker as the water depth increases. To make the surroundings visible, even in deep water, we installed a light that illuminates the field of vision on the head of the user. The user can see various types of seaweed, various corals, and two whales.

Interaction Design

The movement of the user's head is recognized by the sensor for tracking and fed back to the visual field of the CG in real time. A controller is used to move the user. The joystick is used to move in the planar direction, the trigger is used to change the depth, and buttons provide instantaneous movement to preset locations.

Pilot Study

Content of the Pilot Study

We conducted a pilot study using our system in an indoor pool (maximum water depth of 3.8 m) to confirm that the system operates normally in water and determine the immersion obtained by this system. In this study, participants dived completely into the water and looked around for as long as they could hold their breath (Fig. 4). Our system followed the rotation of the head, but could not acquire other movement. We instructed the participants not to move during diving and they did not use a controller. The vertical hole scene was used. The depth of the virtual underwater world was about 20 m.

Result

As a result of the study, a visual presentation with high immersive feeling was achieved in the water, and the sensor for head tracking kept the same accuracy and refresh rate as it did on dry land. Therefore, it has been demonstrated that this system operates normally at a depth of about 4 m.

In the questionnaire that we distributed after the experiment, all participants commented that they felt like they were "in deep water even though it should feel like shallow water." This indicates that our system has the potential to provide underwater experiences of various depths despite being used in a pool. 5 Future Work

Intuitive Operation

Virtual movement in the current system is not intuitive because it uses joysticks, the trigger, and buttons. To achieve a more intuitive movement, it is necessary to sense the movement of the body. There are two ways to achieve this. One is to use an external sensor system such as a camera (Fig. 5, left). The other one is to use an internal sensor such as a bend sensor (Fig. 5, right). Currently, we are working toward achieving motion capture underwater using a stereo camera. We also track the walls and bottom of the pool using edge detection and markers to avoid allowing the user to bump against them.

Fig. 5. Schematic of movement capture methods

Quality of the Content

For this study, we created content that would not fall below 75 fps. However, the reality was decreased somewhat. To achieve both high content quality and refresh rate, it is necessary to find ways to reduce the processing load such as adjusting the mesh of a 3D model, canceling undisplayed part rendering, or introducing single-pass stereo rendering.

Conclusion

In this paper, we proposed a virtual scuba diving system using a UHMD and attempted to realize an arbitrary underwater environment experience in familiar underwater areas such as a pool. Our aim is to achieve the overwhelmingly immersive feeling obtained with a dedicated HMD and enable the user to move freely. We developed a prototype of our system and created underwater content. The results of the pilot study confirmed that our system works well in water and has the potential to provide underwater experiences of various depths despite being in a pool. We will continue to improve image quality and pursue further immersive reality by improving the content so that it can be linked to user movement.

Fig. 1 .

 1 Fig. 1. Configuration of the UHMD system.

Fig. 2 .

 2 Fig. 2. Image of the UHMD.

Fig. 3 .

 3 Fig. 3. Left: shallow water. Right: vertical hole

Fig. 4 .

 4 Fig. 4. Diving while wearing the UHMD.


Taisei Kurihara

Luo Li

Masanori Ishida

Junichi Hoshino
email: jhoshino@esys.tsukuba.ac.jp

Japanese Anime Production Support System with Digital Storyboards

Keywords: Animation production, Storyboards, Schedule

Storyboards are used in the beginning stages of making animation to explain the story or screenplay. This paper proposes a system which assists with storyboard writing and the production process. This system is used with a tablet that allows for input with a pen. It automatically calculates the number of cuts and time required for each scene and can automatically produce a schedule from storyboards. This system makes management for overall animation easy.

Introduction

The number of animation titles on TV in Japan is increasing in recent years. In 2015, 233 new animation titles were produced throughout the year, while there were 108 continuing titles, making this the largest since 1963 [1].

Animation is produced by around 100 staff members working in directing, screenplay, character design, art direction, original drawings, video, art, mastering, shooting, and voice acting. A portion of this is also often outsourced [2]. It is therefore important to convey the director's designs to all staff through the production process in order to avoid delays and decrease the number of accidents during production. It is especially important in production management to ensure that the director's storyboards and the calculated length of each cut on the production process table match since the total length of an anime show is precisely decided. However, since animation production in Japan calculates time by combining the number of seconds and the number of frames, mistakes are prone to occur which causes a huge mental burden to the director and staff. Since calculating the wrong length causes mistakes in the total downstream process such as original art and video, this can result in the issue of materials being needlessly made and increased communication due to constant reorganization of the production process, putting pressure on the production industry.

This study proposes an animation production support system that makes production of the director's storyboards and management of the production process consistent.

System Design

A storyboard app is created that uses a tablet PC as a pen tablet allowing one to make drawings similar to that of storyboards using conventional paper, and also makes it easy to calculate lengths with a combined number of seconds and number of frames. We will learn about the storyboarding management and drawing features of a tablet necessary for creating storyboards, design a UI, and develop a prototype focusing on ease of use for directors.

Automatically producing a process management table from the layered structure of scenes and cuts created by the director and from the time lengths reduces workload and mistakes when starting work. It also makes staff assignments for each cut, assists in making verifying and updating work status simple, and achieves the ability to individually edit areas that need more detailed scheduling.

In addition, it assists in managing the viewing of storyboards for staff involved in animation production. Fig. 1 shows the system interface. Fig. 2 shows the system structure (system transition diagram). A homepage displays when a user logs in. From there, they can jump to the following three main features.

Fig. 2. Transition diagram

Storyboard drawing

The storyboard drawing feature allows for drawing on the spot with a pen input on a one page, five column drawing screen (the same as a storyboard paper used in Japan). Colored handwriting, text, and straight lines can be entered here. Text can be inserted from the keyboard using the text box. Photos and existing illustrations can be called from an image file and pasted in the drawing area.

Columns are automatically filled when a cut that is no longer needed is deleted when drawing. Conversely, columns are automatically carried over when adding a new cut in between columns.

Here, lengths can also be automatically calculated. The number of seconds for each column arbitrarily set by the user is automatically added up, allowing them to know the total length. It also can calculate the length in frames from the number of frames per second. Since a calculator is taking care of the parts which conventionally were calculated manually, this saves time and also avoids any incorrect calculations.

The user can also playback a storyboard by displaying the columns in order according to the number of seconds for each column that they have set.

Previously drawn storyboards can also be displayed in a list and be referenced. Storyboards are stored in this area when saving them.

Schedule management

A sample schedule can be automatically generated in the schedule management screen from completed storyboards. This is generated by counting back the amount of cuts from the deadline. Afterward a schedule can be easily put together as the user finds suitable. This also allows for modifications in accordance with the production process.

Besides an overall schedule, individual detailed schedules can also be assembled for sound, shooting, and drawing.

User management

Staff data can be browsed by registering the production and drawing staff involved with a work, and browsing restrictions for storyboards can also be managed.

Conclusion

A storyboard system was proposed which assists with reducing mistakes in the production of animation by controlling storyboards on a tablet PC and automatically generating schedules.

Verification tests will be conducted in the future while furthering implementation of the system in order to verify the usefulness of the support methods and system proposed by this study. Tests in the animation contents industry are planned. Actual animation directors will be asked to use the system and their physical and mental reduction for production management will be studied using a survey to measure how much production time was shortened and identify how much easier storyboard quality and creation became. Overall judgment will be done through a survey using a fivestep assessment and filling out comments.

Fig. 1 .

 1 Fig. 1. Up) Storyboards Drawing tool Left) Scheduling tool Right) User management tool


Yuki Kubo
email: kubo@iplab.cs.tsukuba.ac.jp

Hirobumi Tomita
email: tomita@iplab.cs.tsukuba.ac.jp

Shuta Nakamae
email: nakamae@iplab.cs.tsukuba.ac.jp

Takayuki Hoshi

Yoichi Ochiai

Bubble Clouds: 3D Display Composed of Soap Bubble Cluster

Keywords: Bubble display, Soap bubbles, Ephemeral user interfaces

We examine Bubble Clouds composed of a bubble cluster for a threedimensional display. A bubble cluster is flexible in that its shape can be modified and it can float in the air with helium gas confinement of the bubbles. By varying the density of the bubble clusters, we project images onto the bubbles without requiring fog confinement or use of special equipment. Moreover, we investigate whether a soap bubble cluster can become interactive by electrifying it.

Introduction

Previously, bubble displays have been achieved as singular [START_REF] Nakamura | Mounting and application of bubble display system: Bubble cosmos[END_REF], multi-layered [START_REF] Ochiai | The Colloidal Metamorphosis: Time division multiplexing of the reflectance state[END_REF], or multiple bubble surfaces [START_REF] Kwon | Fu-gaciousFilm: Exploring attentive interaction with ephemeral material[END_REF]. A single bubble surface cannot display three-dimensional (3D) structures. Although a multi-layered bubble surface can show a semblance of 3D assemblies, they cannot display a complete range of the same.

Bubble displays have several limitations in their size, structure, and projection methods. This motivated us to use a soap bubble cluster as a display system (Fig. 1). We can express 3D objects by reshaping the bubble clusters without fog confinement, and can project images onto the bubbles. In this study, we present Bubble Clouds [START_REF] Kubo | Bubble Cloud: Projection of an image onto a bubble cluster[END_REF], as displays that use a bubble cluster (Fig. 1). A bubble cluster is flexible in that its shape can be modified. In this regard, we examined five shapes of a bubble cluster. Furthermore, it can float in the air by helium gas confinement of the bubbles and adjustment of the bubble density. We can project images on the bubble clouds without requiring fog confinement or special equipment, and by confining helium in the bubbles. We can provide the utility of bubble clusters when producing a bubble display. For example, we can project images onto a bubble cluster without fog confinement, as shown in Fig. 2.

Related Work

Research on bubble cluster displays is related to studies that involve bubble displays and ephemeral user interfaces.

Bubble Display

Some research studies have proposed displaying images by using soap bubbles. For example, Bubble Cosmos [START_REF] Nakamura | Mounting and application of bubble display system: Bubble cosmos[END_REF] provides tangible interactions with bubbles, and a sound is played when a user bursts a bubble. Bubble Cosmos is a single-bubble display that projects an image onto a bubble with fog confinement. FRAGWRAP [START_REF] Kyono | FRAGWRAP: Fragranceencapsulated and projected soap bubble for scent mapping[END_REF] encapsulates fragrances in a bubble confined with fog, so that when the bubble bursts, the fragrance is released to the user. SensaBubble [START_REF] Seah | SensaBubble: A chrono-sensory mid-air display of sight and smell[END_REF] is a display system that uses the fog confinement of the bubbles to deliver information to users by using a projector and fragrances. Colloidal Displays [START_REF] Ochiai | The Colloidal Metamorphosis: Time division multiplexing of the reflectance state[END_REF][START_REF] Ochiai | Theory and Application of the Colloidal Display: Programmable bubble screen for computer entertainment[END_REF] project images onto a soap film by using the ultrasound waves emitted from an ultrasonic-phased array. The reflectance can be varied by vibrating the film using the phased array. Similar to this work, we also used an ultrasonic-phased array as one of the methods to project images on a bubble cluster. Liquids, Smoke, and Soap Bubbles [START_REF] Sylvester | Liquids, Smoke, and Soap Bubbles: Reflections on materials for ephemeral user interfaces[END_REF] form bubble display that consists of bubbles, a soap liquid, and fog, and can be made to interact by moving or blowing bubbles over a dark surface. Sahoo et al. [START_REF] Sahoo | Control of non-solid diffusers by electrostatic charging[END_REF] proposed a method that could alter the trajectory of a bubble by confining an electrostatically charged fog in the bubble and applying an electric field. In comparison with these displays that utilize single soap bubbles or a soap film, our proposed Bubble Clouds utilize a bubble cluster. An example of a display that uses multiple soap bubbles is Shaboned Display [START_REF] Hirayama | Shaboned Display: An interactive substantial display using soap bubbles[END_REF]. This display consists of an array of soap bubbles, and each soap bubble works as a pixel of an image. Flogos [START_REF]Global special effects, I[END_REF] is a device that can form characters and logos that can float by utilizing soap clusters. Bubble Clouds can combine multiple bubbles into a single bubble cluster, and can express 3D objects by reshaping the bubble cluster. Furthermore, Bubble Clouds can project images onto the bubble cluster.

Ephemeral User Interfaces

A soap bubble display is a type of an ephemeral user interface [START_REF] Döring | A design space for ephemeral user interfaces[END_REF]. Similarly, several displays using ephemeral materials such as water, smoke, and fog have been previously proposed. For example, Barnum et al. [START_REF] Barnum | A multi-layered display with water drops[END_REF] proposed a display that projected images on multi-layered water drops. Cloud Display [START_REF] Tokuda | Cloud Display[END_REF] is a space filling display composed of smoke rings. Tangible Sound [START_REF] Yonezawa | Tangible Sound: Musical instrument using fluid water[END_REF] uses fluid water as an input of a musical instrument. By adjusting the flowing water, a sound is produced. HydroMorph [START_REF] Nakagaki | Hydro-Morph: Shape changing water membrane for display and interaction[END_REF] is a water display that can vary the shape by sensing the users input by camera. Cool Interaction with Calm Technologies [START_REF] Virolainen | Cool Interaction with Calm Technologies: Experimenting with ice as a multitouch surface[END_REF] is a multi-touch screen built from an ice-wall, that can detect the palm of a user and achieve multi-touch by using an infrared camera. Cloud Interface [START_REF] Tobita | Cloud Interface: Designing aerial computer environment for novel user interface[END_REF] is a mid-air display that can move in the air. It consists of a blimp, a gondola, and a projector. Bubble Clouds use soap bubbles as the ephemeral material.

Implementation

We use a bubble generator to produce the bubbles that comprise the bubble cluster (Fig. 3). The bubble generator consists of a helium cylinder, a pressure gage, an air tube, a bucket containing soap solution, and an acrylic case with five holes. To generate bubbles that can float, helium gas is passed through the soap solution that passes through the holes in the acrylic case. A floatable bubble cluster is generated by separating the bubbles from the bucket by using an air tube having an inner diameter of 3 mm. The pressure gage regulates the pressure inside the air tube. We place an acrylic panel on the bucket with a 25 cm square hole at its center from which the bubbles escape from the bucket. To hold the soap bubble solution and store bubbles, we use a bucket with a diameter of 45 cm and height of 15 cm. The solution should generate bubbles that do not burst easily, and we use a solution mixture of water, a detergent that includes a surfactant, and laundry starch that includes polyvinyl alcohol in a ratio of 5:1:5.

An ultrasonic-phased array with 283 ultrasonic transducers is shown in Fig. 4. Ultrasound waves, generated from an ultrasonic-phased array and emitted to the bubbles, change the reflectance of the soap film, and thereby produce vibrations and project images onto the film. An ultrasonic-phased array generates an ultrasonic wave front to stimulate the entire surface of the bubble cluster.

Bubble Clouds 4.1 Dense Bubble Cloud

When the bubbles are small, they seem to appear as if they are pixels of a projected image, as shown in Fig. 2. Dense Bubble Cloud allows each bubble to function as an individual pixel, and hence does not require fog confinement or additional material to project images.

Sparse Bubble Cloud

Images cannot be projected onto bubbles when their diameter is extremely large because the soap film permeates light. However, we can project images onto such bubbles by using ultrasonic waves, and we call this Sparse Bubble Cloud.

In Colloidal Display, a method of projecting images by changing the reflection properties of the film by ultrasonic wave was introduced. We investigated whether this method can work effectively for bubble clusters. Fig. 5a shows a state when the film is not vibrating. The image is not projected by light; it is permeated. Fig. 5b depicts the bubble cluster when the reflectance is varied by applying an ultrasonic wave to the soap film via an ultrasonic-phased array. Compared with Fig. 5a, the reflectance changes in Fig. 5b. We attempted to project images onto a bubble cluster by using an ultrasonicphased array (Fig. 4). The image projected was slightly clear compared with a bubble cluster without ultrasonic waves (Fig. 6).

,ĞůŝƵŵ LJůŝŶĚĞƌ

Evaluation of Bubble Generator

We evaluated the effect of the pressure and diameter of the holes of the acrylic case.

We conducted this experiment to evaluate whether these conditions meet the smallest diameter requirement of the bubbles that enables the display to float. The pressure was measured by using a pressure gage. In our experiment, we applied the following values: 0.02, 0.04, and 0.06 MPa. We consider two diameter sizes for the acrylic cases: 0.02 and 0.04 mm, and there are five holes in each case, positioned at the same distance from each other (Fig. 7). The diameter of a bubble cluster is measured by a ruler. The bubbles cannot be uniform in size, and thus, we measured the diameters of numerous bubbles and averaged the result.

D E

Fig. 6. Sparse Bubble Cloud using an ultrasonic-phased array: (a) non-vibrating bubbles with projection, and (b) vibrating bubbles with projection (projection characters:"ACM SIGGRAPH"). The results are shown in Table 1. We observe that higher pressures or larger holes produce larger bubbles. These conditions satisfy the smallest diameter requirement enabling a bubble to float. We chose the following conditions to generate the smallest bubbles possible: pressure of 0.02 MPa and diameter of 2.0 mm.

Shape of Bubble Cluster

We investigated the shapes that could be generated with a bubble cluster. In this experiment, we formed five shapes with the bubble cluster, namely, cube, quadrangular pyramid, sphere, mountain, and two mountains. Fig. 8 shows the different cluster shapes. Bubble cluster reshaping was performed by using a 5 × 30 cm plate. We poured the soap bubble solution into a bucket and fixed an acrylic case to the bottom. A helium cylinder was connected to the acrylic case by an air tube. Helium gas was released at a constant rate from the acrylic case into the soap bubble solution. The reason helium gas was confined is because it is easy to process and maintain the shape of the bubble cluster without being influenced by gravity. Bubble clusters escaped from a 25 × 25 cm hole on a plate that was placed on the bucket. Thus, the user could reshape the bubble cluster to any shape. Fig. 8 shows the five shapes of the bubble cluster we produced. The cube shown in Fig. 8a was formed by placing a cubic mold on the top of the hole. The cubic mold was made from five plates, and we separated each plate individually to create a cube. The quadrangular pyramid shown in Fig. 8b was formed by sharpening the tip by scraping the bubble cluster using a plate. The sphere and mountain shown in Fig. 8c andd, respectively were generated by adjusting the hole with the plates. The two mountains shown in Fig. 8e were formed by scraping the middle of the bubble cluster.

h h h h h h h h h h h h h h h

Interaction with Electrified Bubble Cluster

We investigated whether an electrified soap bubble cluster could be interactive. This was verified in an indoor still air environment. To electrify the bubble clusters, we used a polyvinyl chloride pipe and tissue paper. Static electricity was generated by their friction that was charged by touching the bubble cluster with the polyvinyl chloride pipe in the air. After that, we examined how the bubble cluster behaved when a user placed a hand near the bubble cluster. Consequently, we could observe a bubble cluster being tracked by the hand of a user. As shown in Fig. 9, the bubble cluster moves in the same direction as the hand. We believe that this occurs owing to the Coulomb force between the hand of a user and the bubble cluster. Therefore, we can manipulate the soap bubble cluster in mid-air. We also noticed that the tracking was lost once the Coulomb force ceased to exist. We propose that characters could be generated by using a Bubble Cloud. A user could then interact with the characters using his/her hands and with objects. For example, we could formulate a game in which a user could use a toy sword to attack the generated characters (e.g., monsters). Each character could be projected differently, and thus, a user would be able to classify each character.

D E F G H

9 Discussion and Future Work

We attempted to project images onto a Sparse Bubble Cloud in mid-air by using an ultrasonic-phased array. However, this Sparse Bubble Cloud moved because of the ultrasonic wave emitted from the phased array. This suggests that if we project images onto Sparse Bubble Cloud in mid-air, we must track the movement of the bubble cluster and counter the effect of the ultrasonic wave to prevent the movement.

In our experiment, we observed the phenomenon of a bubble cluster tracking hand of a user. However, it was unable to track when its size was extremely large. Our results showed that the Coulomb force between the hand of the user and bubble cluster was not sufficiently strong. It is to be noted that we electrified the bubble cluster by using a simple method in this experiment. For this reason, we believe that the bubble cluster could be moved if it was strongly electrified by a specific equipment. Moreover, this interaction was easily affected by wind.

Dense Bubble Cloud is a type of Bubble Cloud that could project an image to the surface of a bubble cluster. In addition, we found that we can utilize an entire soap bubble cluster including the bubbles inside, as pixels and use it as a 3D display (Fig. 10). This is possible apparently because each of the bubbles in the bubble cluster reflects and permeates light. In the future, to achieve 3D projection on a bubble cluster, we will investigate conditions such as the diameter of the bubbles that compose the bubble cluster. We will also examine if we can realize a true 3D display by using multiple planes because presently our display only has a single plane.

In this work, we used panels to reshape the bubble cluster. However, this method has a poor reproducibility in terms of shape because the process of reshaping is different for different users. The reproducibility of shape can be ensured by a method that uses an ultrasonic-phased array. In the future, we will reshape the bubble cluster automatically to ensure shape reproducibility by using an ultrasonic-phased array.

Furthermore, in the future, we plan to conduct quantitative evaluations such as how long the bubble cluster lasts, and character recognition rate. Additionally, we aim to conduct quantitative evaluation such as the user perception of the applicability of the display for entertainment purposes.

Conclusion

In this study, we presented Bubble Clouds using bubble cluster as displays. A bubble cluster is flexible because its shape can be varied and it can float in the air by helium confinement. We introduced two types of Bubble Clouds, namely, Dense Bubble Cloud and Sparse Bubble Cloud. By changing the density of the bubble cluster, images could be projected onto it without fog confinement or use any special equipment. We formed five shapes with the bubble clusters: cube, quadrangular pyramid, sphere, mountain, and two mountains. Moreover, we also explored whether an electrified soap bubble cluster could be interactive.

In the future, to realize 3D projection on a bubble cluster, we will investigate conditions such as the diameter of the bubbles that compose the bubble cluster. Furthermore, we plan to explore reshaping the bubble clusters automatically to ensure shape reproducibility by using an ultrasonic-phased array.

Fig. 1 .

 1 Fig. 1. A bubble cluster. Fig. 2. Dense Bubble Cloud.

Fig. 2 .

 2 Fig. 1. A bubble cluster. Fig. 2. Dense Bubble Cloud.

Fig. 3 . 4 .Fig. 5 .

 345 Fig. 3. Bubble generator. Fig. 4. Ultrasonic-phased array.

Fig. 7 .

 7 Fig. 7. Acrylic cases with five holes (scale is 54 × 54 × 24 mm). Left: hole diameter of 2 mm. Right: hole diameter of 4 mm.

  25 mm 25 mm -30 mm 30 mm -35 mm

Fig. 8 .Fig. 9 .

 89 Fig. 8. Five shapes of bubble cluster: (a) cube, (b) quadrangular pyramid, (c) sphere, (d) mountain, and (e) two mountains.

Fig. 10 .

 10 Fig. 10. 3D form display of bubble cluster. Photo from the (a) left,(b) front,(c) right (projecting the inverted ace2016 logo).

Table 1 .

 1 Effect of hole diameter and pressure on the bubble diameter.


Yuto Sugita

Keiichi Zempo
email: zempo@iit.tsukuba.ac.jp

Koichi Mizutani

Naoto Wakatsuki

The Alteration of Gustatory Sense by Virtual Chromatic Transition of Food Items

Keywords:

When we visually recognize food items, their color influences the impression of their taste, because the gustatory sense is affected by other senses such as smell and eyesight. In this study, in order to control the gustatory sense of human beings, we examined the influence on gustatory sense by giving a virtual chromatic transition of food through Head-Mounted Display. The experiment was conducted under six conditions that changed three factors:"before or after eating", "viewing through HMD or naked eyes", "with virtual chromatic transition or without". As a result, we confirmed that the difference in the amount of visual information influenced the impression of food items before eating. It is expected to realize more applicative control of the gustatory sense by analyzing this difference.

Introduction

The taste information obtained from the taste cells present in the tongue accounts for the majority of the human gustatory sense [START_REF] Yutaka | Mechanisms for taste signaling in taste buds[END_REF]. However, it is known that the gustatory sense is affected not only by taste information but also by other sensory information such as visual information [START_REF] Tomomi | Nippon jin no mikaku[Japanese taste[END_REF][START_REF] Keiji | Mikaku,Kyukaku[Taste,Smell[END_REF]. Recently, studies have been increasingly being made to change dining experience by externally changing sensory information [START_REF] Marcelo | DinnerWare: Why Playing With Food Should Be Encouraged[END_REF]. In particular, the influence of olfactory and visual information on taste sensation is large, therefore a lot of studies have been done by using this [START_REF] Alley | The influence of physical state and color on perceived sweetness[END_REF][START_REF] Bayarri | Influence of color on perception of sweetness and fruit flavor of fruit drinks[END_REF][START_REF] Stillman | Color influences flavor identification in fruit-flavored beverages[END_REF][START_REF] Takuji | Meta Cookie:Pseudo -Gustatory Display Based on Cross -Modal Integration[END_REF]. These effects are recognized also in psychological experiments [START_REF] Johnson | The psychophysical relationship between color and flavor[END_REF]. For example, Meta Cookie gives a change to the feeling of human gustatory sense by changing both visual information and olfactory information [START_REF] Takuji | Meta Cookie:Pseudo -Gustatory Display Based on Cross -Modal Integration[END_REF]. It's also said that auditory information when chewing food, tactile information such as the texture of food and chewy feeling influences the feeling of taste [START_REF] Daichi | Change of Mouthfeel by Means of Cross-modal Effect Using Mastication Sound and Visual Information of Food[END_REF][START_REF] Tetsuro | Research on Sensory Integration in a Taste of Food[END_REF][START_REF] Koizumi | Chewing jockey: augmented food texture by using sound based on the cross-modal effect[END_REF]. Previous research suggests that changes in sensory information from the outside can also change taste information. However, quantitative evaluation of how taste information is affected by changes given to sensory information has not yet been made. If a quantitative evaluation is possible, more applicative control of the gustatory sense becomes possible. Among the sensory information affecting the taste like this, the visual information can change most easily the feeling of the taste significant. Unlike other sensory information, the visual information can be separated three primary colors of light, therefore it is easy to present any visual information. Therefore, by changing the color of the food through the visual sensor and visually recognizing the food through the display, it is possible to change the visual information of the food. In addition, as the penetration rate of smartphones is increasing in recent years, there are many scenes of possessing visual sensors such as cameras. There are also many Head-Mounted Display(HMD) that can provide VR environment easily by wearing smartphones, therefore it is possible to impart real world to a chromatic transition of food.

Also, in recent years, content production related to dining experience has been actively performed [START_REF]Verchan Reality[END_REF][START_REF]PROJECT NOURISHED[END_REF]. Dining experiences that are familiar to humans should be always new and enjoyable by adding such entertainment characteristics. It is expected that the control of the gustatory sense will greatly contribute to such content creation in the future.

In this research, we aim to verify the influence of a chromatic transition of food through a HMD on the impression of taste for food. For this purpose, we constructed a system that gives a chromatic transition to the color of food through HMD. In this research, we also examine the difference between visually recognizing food with the naked eye and the recognition it through HMD. Therefore, to control the impression of human taste by optimal color change, it is necessary to consider the difference between the case of visually recognizing with the naked eye and the case of seeing through the display.

VR System of a Chromatic Transition

In this research, smartphone and HMD were used as a color change presentation device. iPhone 7 Plus which iOS 10 introduced by Apple Co., Ltd. was introduced, and VOX + 3DVR goggle provided by VOX was used for HMD. We can really find the image which we acquired from the camera of the smartphone like the image actually in front of the eyes by attaching HMD which set the smartphone. By giving a color change to the image acquired from this camera and outputting it to the display, it is also possible to give a chromatic transition to the user. These devices are shown in Fig. 1 .

And, we installed the application created in Xcode on iPhone 7 Plus and used it to detect target food and change color to that food. The source code of the application is written in Objective-C and introduced the OpenCV framework for image processing such as a color change to target food. A flow chart showing the flow of the created application is shown in Fig. 1 , and detailed specifications are shown below. The food whose color has been changed by the above operation is outputted to the display and made to be visible to the human through the HMD.

Experiment of Verification

The Experimental Environment and Target Food Item

This experiment was conducted for 11 males and females in their twenties who are healthy with both visual and taste. The experimental environment is shown in Fig. 1. In this experiment, the application was carried out in a room that turned off the light, unifying the background to black so that the application can easily detect white food. A target food was placed on the inside of which processing was made by attaching a black felt fabric around an acrylic box which is a cube having one side of 300 mm and the front side does not exist, and this subject was visually recognized by the subject.

In this experiment, the self-made fizzing candy was used as the target food item. All of the ingredients were made equal in 0.1 g unit so that each of the fizzing candy had the same taste. Those colored by using coloring agents were prepared for each of them. The colors used this time are five types of red, blue, yellow, green and purple shown in Fig. 1. Experiments were carried out using a total of six kinds of color fizzing candies to which white of the reference color was added.

Method of Experimentation

In this experiment, we asked each subject to answer the questionnaire about the impression of the taste of the target food. In addition, the experiment was conducted under six situations in which three factors, "before or after eating,""viewing through HMD or naked eyes,""with Virtual chromatic transition or without" were changed. A table summarizing these is shown in Table 1, a diagram showing these conditions is shown in Fig. 2. To this "six colors of fizzing candy" will be added.

The questionnaire prepared two kinds of contents different "before or after eating". The contents of the questionnaire before eating are two items, "What kind of taste do you think it is?," and "What kind of food do you imagine?". Next, the contents of the questionnaire after eating are three items, "What kind of taste did you feel? ","What kind of food did you imagine?","Were there any differences of impressions before eating?".The first questionnaire item evaluated what kind of taste it felt before and after eating about five colors excluding white, in 5 grades each of the five flavors. At this time, white was defined as the evaluation value 3 in all the five flavors before and after eating, and it was used as a standard.

Result

From the experimental results, evaluate how far away the impression was from the white fizzing candy received as a reference against the colored. At this time, we define the taste space as 5-dimensional space and evaluate the evaluation value of each basic taste answered by the subject to Euclidean distance when white is taken as the origin and evaluate.

First, in order to compare "before or after eating" and "viewing through HMD or naked eyes", we perform Analysis of variance using the result of experiment 1, 2, 3 and 4, as factors. Next, in order to compare "before or after eating" and "giving or not virtual chromatic transition", we perform an analysis of variance using the result of experiment 3, 4, 5 and 6, as factors. Graph showing these results is shown in Fig. 3.

The abscissa of Fig. 3 shows the six situations of experiment shown in Table 1 , and the ordinate shows the average value of the Euclidean distance of the result calculated for each situation of the experiment.

Firstly, as a result of analyzing the experiment #1, #2, #3 and #4, the interaction of "before or after eating" and "viewing through the HMD or naked eyes" was superior at 5% level or less. It was also revealed that the impression before eating was different depending on the presence or absence of the HMD by the simple main effect test. It seems to be due to the difference in the amount of visual information obtained when comparing the state in which the food is viewed through the HMD and the state in which the food is visually recognized with the naked eye. The information acquired from the camera has a difference in information amount as compared with information obtained from the naked eyes due to various factors such as image quality and viewing angle. It seems that the difference in this amount of information affected the impression of the food before eating.

Next, as a result of analyzing the experiment #3, #4, #5 and #6, the interaction of "before or after eating" and "with virtual chromatic transition or without" was superior at 1% level or less. Moreover, it was revealed that the impression before eating is caused by the presence or absence of color change by examination of simple main effect. This seems to be related to the amount of visual information as well. The amount of visual information decreases if it passes through a display like a camera. The application used in this experiment was to add a chromatic transition from the top of the image acquired from the camera. In other words, when a chromatic transition is added by the application used in this experiment, it can be said that it is easier to obtain visual information because clear colors can be seen compared with the case where no chromatic transition. It is seemed that the difference in the amount of visual information has influenced the impression of the food before eating.

Conclusion

In this study, we examined the difference in the feeling of taste especially when visually recognizing the food with the naked eye and through the HMD. For this purpose, we installed an application on the HMD and made an application to change the color of the image acquired from the camera mounted on the smartphone. The experiment was conducted under six situations in which three situations, "before or after eating", "viewing through HMD or naked eyes", "with virtual chromatic transition or without" were changed. In each case, differences in the taste impression due to these factors were verified by subjective evaluation about the taste impression of the target food. The result of analysis revealed that "viewing through HMD or naked eyes," and "with virtual chromatic transition or without" respectively influence the impression before eating food. Therefore, the experiments suggested that the impression of food tastes were changed due to the difference in the amount of visual information between the naked eye and the screen. In other words, when giving a chromatic transition to a food through a screen such as HMD, it is possible that considering the difference in impression of the case of visual recognition with the naked eye leads to more advanced control of taste.

Moreover, verification experiments have revealed that there is almost no influence on the feeling of the taste of eating food merely by changing the visual information. Changes in olfaction, tactile sensation, and auditory information are thought to be useful for effecting this. From now on, we will investigate how they affect taste sensation by changing these, and also realize a taste control system by constructing a multimodal interface including these.

Fig. 1 . 3 . 4 .

 134 Fig. 1. Experiment environment and a virtual chromatic transition system

Fig. 2 .Fig. 3 .

 23 Fig. 2. Situation of experiment

Table 1 .

 1 Situation of experiment

	Experiment No. Before or After	Viewing Method	Chromatic Transition
	#1	Before	Naked eye	None
	#2	After	Naked eye	None
	#3	Before	HMD equipped	None
	#4	After	HMD equipped	None
	#5	Before	HMD equipped	Transitioned
	#6	After	HMD equipped	Transitioned


Yu Ishikawa
email: ishikawa.yu@entcomp.esys.tsukuba.ac.jp

Yuko Furudate
email: furudate.yuko@entcomp.esys.tsukuba.ac.jp

Junichi Hoshino
email: jhoshino@esys.tsukuba.ac.jp

TamaPeeler: An Interactive Cooking Tool for Children's Dietary Education

Keywords: Dietary education, Children, Cooking tool

With the change in our diet, importance was placed on dietary education, and various systems for dietary education have been proposed. However, there has been no system that deals cooking tools. This article, therefore, proposes a peeler-type device, 'TamaPeeler', in the hypothesis that touching the food directly and being involved in cooking would raise their interest in food. TamaPeeler detects motions of peeling vegetables, and simultaneously makes various peeling sounds from the wirelessly connected smart phone. As we observed children during its exhibit, we have observed them proactively peeling vegetables, or showing interests in peeling other vegetables, and it was suggested that using an interactive cooking tool can raise the interests of children in food and diet.

Introduction

As our diet keeps changing, dietary education was considered important in Japan, and in 2005, Basic Act on Food Education that stipulated the basic concepts and items of the measures for food education was established. Its foreword defined food education as an effort to "cultivate knowledge on food and abilities to select them through various experiences, and foster individuals who can practice healthy diet", and importance was placed on the dietary education of children particularly.

Although some systems have been proposed related to food and diet such as a forktype device [START_REF] Kadomura | EaTheremin[END_REF] that makes sounds while eating, and as a system for food education, a fork-type device [START_REF] Kadomura | Persuasive Technology to Improve Eating Behavior using a Sensor-Embedded Fork[END_REF] linked with an application, and a tray-type device [START_REF] Lo | Playful Tray: Adopting Ubicomp and Persuasive Techniques into Play-based Occupational Therapy for Reducing Poor Eating Behavior in Young Children[END_REF], as well as projection system [START_REF] Mori | Dining Presenter: Augmented Reality system for a dining tabletop[END_REF] have been proposed, but all of these are systems that can be used after food is laid out on the table. This article proposes "TamaPeeler"a cooking tool that would motivate children to help in cooking voluntarily (Fig. 1)in the hypothesis that touching the food directly and being involved in cooking would raise their interest in food. TamaPeeler is a peelertype device that detects motions of peeling vegetables, and coupled with the wirelessly linked smart phone, it makes various sounds. TamaPeeler is a peeler that has an eye-catching aesthetics for children, and system that makes various sounds in response to the motions of peeling as an entertainment factor so it raises children's interests in cooking itself. It detects: 1) when the peeler touches the vegetable, and 2) the motions of peeling, both of which are called 'soundcreating conditions', and when both conditions were fulfilled simultaneously; the sound will be played on the smart phone.

For judgement of the sound creating conditions, a touch sensor and an acceleration sensor are used in consideration of children's safety. An acceleration sensor alone can be adequate to detect the motions of peeling, but by coupling with a touch sensor, it can prevent from making sounds when children swing the device around with a playful manner.

The peeler also has a function to change some of the sound effects according to the number of times of peeling motions. It counts the number of times that fulfilled soundcreating conditions and after 3n-th time (n=1, 2, 3…), it will make another kind of sound. As the sound changes after certain times of peeling motions, children could be motivated to keep peeling to hear different sounds.

Fig. 2. System Configuration

Implementation

System configuration is shown in Fig. 2. To simplify the implementation, the prototype has a rotary DIP switch on the peeler to select kinds of sound, and the sound is played on the wirelessly connected smart phone.

Detection of Contact between the Blade and a Vegetable

Electrodes are attached to the both ends of the blade of the peeler, and they work as a touch sensor mounted. Then, it detects the blade contacts with a vegetable by the change of the electric voltage. As there normally is an allowance for the angle of the blade of a peeler, the terminal of the touch sensor was aligned so it connects at the proper angle when the blade touches the vegetable.

Detection of Peeling Motions

An acceleration sensor is used to detect the peeling motions. In the prototype, based on the most basic peeling motion, the shift of an axis by time are measured and processed with threshold value. The threshold value are tested and set at the value so that it did not make sound when adults swing the peeler around in the air.

Playing the Sound

The sound is played via the specific application on the smart phone's speakers. It has 14 different kinds of sound and part of them is shown in the Table 1. The electric circuit of the peeler and the smart phone are connected wirelessly by an energy-saving close range transmitter, 'Bluetooth Low Energy. The variations of the sounds according to the times of peeling motions are such as, when it is set to play the meows of a cat as in Table 1, No. 1, after 3n-th time, it switches to bow-wows of a dog.

Evaluation

To verify the applicability of the prototype, we exhibited it, and observed children peeling vegetables with it. The exhibit and the observation were conducted at the Hardware Contest, GUGEN, held in December 2015 in Akihabara.

5-year old girl was observed to be absorbed in peeling vegetables for long time. When the sound effect is played, she smiled, and looked amused, and she was willing to continue peeling vegetables until the sound changed to a new one. This girl finished her 'peeling' and went off to see other exhibits, but came back afterwards and wanted to do it again. She was once again absorbed in peeling until her father finally suggested that they should go home. Also, a boy was observed to keep peeling until he could hear all the different kinds of sound. Moreover, it was observed that multiple children wanted to peel other kinds of vegetables voluntarily for the sounds from the smart phone. The fact that children kept peeling vegetables, and wanted to hear other sounds suggests a possibility that, being able to make sounds by their own motions raised their interests in cooking and positively influenced their voluntary actions. And from their remarks that they wanted to peel other vegetables, it is thought that it raised their interests in other vegetables as well. This suggests that the peeling work they were motivated to try by TamaPeeler triggered their interests, not only in the vegetable they actually touched, but also other related food as well.

Conclusion

We experimentally created a peeler-type device, TamaPeeler, to promote voluntary participations of children in cooking, and to help food education at home. And at the exhibit of the device, it was observed that, from their motivation to play sounds, children repeatedly peeled the vegetables, and wished to peel other vegetables, wanting to touch many vegetables proactively. From this, it was suggested possible that TamaPeeler can raise children's interests in food through the cooking activities at a kitchen.

In the future, it is our hope that we can compare, by conducting a mid-to-long term experiment, the difference in their interests with or without this device, and study the effects of the sounds as well as the modification of the system and application to other cooking tools besides a peeler.

Fig. 1 .

 1 Fig. 1. Overview of TamaPeeler: Left) TamaPeeler, Right) Appearance in experience

Table 1 .

 1 Some sound examples

	No.	Name	Onomatopoeia
	1	Cat	meow
	2	Dog	bowwow
	3	Jack-in-the-Box	boing


Yu Ishikawa
email: ishikawa.yu@entcomp.esys.tsukuba.ac.jp

Junichi Hoshino
email: jhoshino@esys.tsukuba.ac.jp

Novest: Position Estimation of Finger on Back of Hand with a Small Ranging Sensor Array

Keywords: Smartwatch, position estimation, ranging sensor array

This article suggests a technique to make back of the hand (BOH) a track pad by attaching a distance ranging sensor on the side of a smart watch. It does not only estimate the position of a finger on BoH but also distinguishes the presence or absence of distance between the operating finger and BoH, which functions as an operating surface. Distance data and signal strength data are obtained by distance ranging sensor array, and the presence of distance is distinguished by machine learning. Our performance evaluation shows the accuracy of finger position estimation is 4.2mm(SD=1.2mm).

Introduction

The input method of smartwatches depends on a touch panel and buttons. The size of their interface gets smaller in proportion to the size of the devise; therefore, an operating finger becomes relatively large and hides information on screen, which leads to a problem of lowered operating efficiency.

For this problem, a technique to use the fringe area of the device as an input interface is being suggested, and there are types that use permanent magnet [START_REF] Harrison | Abracadabra: Wireless, High-Precision, and Unpowered Finger Input for Very Small Mobile Devices[END_REF], interchange signal [START_REF] Zhang | SkinTrack: Using the Body as an Electrical Waveguide for Continuous Finger Tracking on the Skin[END_REF], and supersonic wave [START_REF] Nandakumar | FingerIO: Using Active Sonar for Fine-Grained Finger Tracking[END_REF].

This article proposes a technique to use BoH like a trackpad with a small ranging sensor array. The two-dimensional position of a finger and distance between the finger and BoH is distinguished by obtaining distance data and signal strength data with a sensor attached to the side of a smart watch. Using BoH as a trackpad allows operation without hiding any information on display. Also, it solves the problem of having a difference between an operating object and an input area on screen, by estimating the two-dimensional position of the finger while distinguishing the presence or absence of distance between the finger and BoH.

Concept

This study estimates the position of a finger on BoH, and distinguishes the presence or absence of distance between the operating finger and BoH by attaching a small sensor on the side of a smart watch.

Fig1 (Left) shows the overview of Novest, and 3 states (out of range, hovering, touching) are differentiated while distinguishing the presence or absence of a distance. Out of range is a state where the sensor does not detect the finger, hovering is a state in which there is a certain level of distance between the finger and BoH, and touching is a state in which there is a small amount of distance between the finger and BoH or they are in contact. To distinguish the distance between a finger and BoH, ranging sensors are arranged vertically on BoH. The gap between the operating object on display and the input area on BoH is solved by treating them like the relation between a mouse and a mouse pointer, setting up the touching state. Touching is equivalent to clicking on a mouse. This article only refers to the case of operating with one finger (index finger on right hand) Fig. 1:Left).

Implementation

The implemented ranging sensor array is shown in Fig. 1 (Rgiht). The size of the sensor board is 15mm in length, 50mm in width and 8mm in thickness. The width is determined in reference to the side of SONY Smart Watch 3 [START_REF]Sony Smart Watch 3 SWR50[END_REF] (10mm height, 51mm width). As ranging sensor, 18 (2 x 9) of Time of Flight system STMicroelectronic VL6180Xs are arranged. This arrangement is made during the preliminary experiment to make sure that a finger 20mm away from the sensor is in the view area of at least one sensor. The minimal vertical intervals implementable are set, and the intervals in the luminescent part are approximately 8mm. This sensor board is connected to NXP Semiconductors prototyping board mbed LPC1768 with a flat cable, controlled by I2C communication.

Additionally, the noise of distance data and signal strength data obtained by sensor array is suppressed using a low-path filter.

Finger Position Estimation

Coordinate system is set as Fig. 1 (Left). Strength data of 9 signals of a top column on the ranging sensor is used for Y coordinate derivation. Since this time one finger is assumed to operate, strength data of 9 signals is processed with spline interpolation, determines the position with the strongest signal as Y coordinate.

After that, the sensor distance data closest to the derived Y coordinate is used as X coordinate.

Finger State Classification

Support Vector Machine is used to classify the distance between the operating finger and BoH. 44 features are used, and they consist of the signal strength ratio of lower column to upper column (9) and its average value, total value, standard deviation, maximum value, minimum value, range, signal strength difference with the adjacent sensor ( 16), signal strength difference between the lower column and upper column (9), its average value, standard deviation, total value, and the ratio of total signal strength in lower column to total signal strength in upper column. These are scaled from -1 to 1 for use.

Evaluation

To evaluate the performance of this technique, experiments under 4 situations were carried out, with a finger and a round stick grasped by a robot arm as a direction method, and a flat surface and BoH as a directed surface. Subjects were 9 university and graduate school students (1 female, 8 males, 23.8 years old (SD=3.7 years)). The purpose is to confirm the finger and BoH effect on this technique by comparing the position estimate accuracy and the distance distinguishing accuracy under 4 experimental situations.

In this evaluation experiment, pointing task which is to point a marker on BoH or a surface with one's index finger on right hand was implemented, and 20 markers were set up for a random indication.

The task was implemented under the environments of 1) finger and BoH, 2) finger and flat surface, 3) round stick and BoH. The above 3 environments were of evaluation in touching, and after they were completed, same tasks were implemented with hovering in the environment of finger and BoH. With hovering, it was instructed to keep 10 ~ 15mm distance between a finger and BoH.

As a result, the position estimation accuracy of touching was 4.2mm(SD=1.2mm), and the estimate accuracy of hovering was 5.5mm(SD=2.8mm). Additionally, the distance distinguishing accuracy was 91.8%(SD=8.3%) with per-user classifier.

Conclusion

We proposed Novest which is a technique to make BoH a trackpad by setting up a small ranging sensor array on the side of a smart watch. Novest did not only distinguish the two-dimensional position of a finger, but also the presence or absence of a distance from BoH. Then, accuracies of the position estimate and distinguishing distance between touching and hovering were confirmed by implementing pointing tasks under 4 kinds of environments for experiments. As for the position estimate accuracy, a highly accurate estimate of 4.2mm(SD=1.2mm) was possible with touching. Finally, this technique can be expected to be applied to small devices other than a smart watch such as Smartglasses, because it distinguishes the two-dimensional position of a finger and the presence or absence of distance without attaching a sensor directly on skin.

Fig. 1 .

 1 Fig. 1. Overview of Novest: Left) Usage of Novest, Right) Ranging sensor array of Novest