Cristian Bogdan
email: cristi@kth.se

Declarative Interaction Towards Evolutionary User Interface Prototyping

Keywords: prototyping, declarative, user interface, evolutionary, conflict

This paper examines the potential of describing interactive systems in a declarative manner with concepts familiar to both developers and designers. Such declarative interaction descriptions will enable evolutionary prototyping processes. This new type of design and development processes that can emerge with declarative interaction is described along with benefits for humancentred system design. A few challenges are raised for future research in this area.

Introduction

Attempts were made to describe interactive systems declaratively for several decades, for example Model-Based UI Development (MBUID, e.g. [START_REF] Patternó | Model-based design and evaluation of interactive applications[END_REF]). Once a declarative description of an interactive system is available, there are several advantages: the system can be analyzed in regard to its associated usability, safety, human error (e.g. [START_REF] Fahssi | Enhanced Task Modelling for Systematic Identification and Explicit Representation of Human Errors[END_REF]) or other aspects, and it can be transformed, which includes the generation of code towards the running interactive product. These advantages stem from declarative models being relatively easy to process by computing systems, unlike procedural code The downside of MBUID approaches is that they are often highly theoretical, aiming to drive the description of the interactive system from a very abstract model (e.g. the Tasks and Concepts level of the CAMELEON framework [START_REF] Calvary | A Unifying Reference Framework for multi-target user interfaces[END_REF]), which is hard to understand by designers, users or even by developers. Furthermore, MBUID has very little support for user interface prototyping [START_REF] Beaudouin-Lafon | Prototyping Tools and Techniques[END_REF][START_REF] Lim | The anatomy of prototypes[END_REF]: when a user interface is generated, it is hard for users and designers to adjust it and iterate with it for improvement according to their needs.

As declarative research approaches like MBUID do not constitute a solution for achieving a more declarative UI design and development practice, the approach we can take is to analyze the current practice and consider where procedural code dominates. For that analysis, one could employ e.g. the Model-View-Controller [START_REF] Krasner | A description of the model-view-controller user interface paradigm in the smalltalk-80 system[END_REF] approach to conceptualizing an interactive system. Important parts of the Model are already declarative in current practice, at least in regard to describing how the data is structured, and what methods are available to process it. While the Model methods are most frequently implemented as procedural code, the declarative part of the model is often sufficient for describing the user interface in its other conceptual modules (View and Controller). Similarly, View templates are often described in a declarative manner, in languages such as HTML or variants of XML. In the quest proposed here to find fully or predominantly declarative representations of interactive systems, it is therefore the Controller where the procedural code still dominates. Since Controllers describe the "feel" of the user interface, the interaction itself, I use to refer this quest as "Declarative Interaction". It is important to note that once a declarative specification of an interactive system exists, a fully testable implementation can be produced. That is, if an online (running) prototype has been made in a declarative manner, with an approach that allows a declarative description of interaction, that prototype can be iterated into the running product. Therefore, declarative interaction can lead to Evolutionary Prototyping [START_REF] Beaudouin-Lafon | Prototyping Tools and Techniques[END_REF] processes.

In this paper I aim to address the issue of finding such low-level declarative UI representations, so that they are understandable for developers and end-users. A subsequent objective is to support UI prototyping processes, whereby developers, users, or customers have the freedom to iterate with the interactive prototype. This work is inspired from a large, long-term case study [START_REF] Bogdan | IT Design for Amateur Communities[END_REF][START_REF] Bogdan | Makumba: the Role of Technology or the Sustainability of Amateur Programming Practice and Community[END_REF] that observed users, designers and voluntary developers, who were able to develop, maintain and extend predominantly declarative interactive systems for long periods of time.

Potential of Declarative Interaction

The traditional UI design and development process

In order to illustrate the potential of declarative interaction for achieving a more flexible and human centred User Interface design and development process, it is important to visit the traditional process and emphasize its aspects relevant to this paper: various stages of prototyping and the transition between design and development. The top of Figure 1 illustrates this current process of designing and developing interactive systems). A few (cheap) paper prototypes are produced, and based on designer judgment and user feedback, some ideas continue to the online prototyping phase. After a number of formative iterations, one of the online prototypes is delivered ("Online Delivered" in the figure) for the developers to implement. Variants of the Model-View-Controller [START_REF] Krasner | A description of the model-view-controller user interface paradigm in the smalltalk-80 system[END_REF] approach are often used, whereby the non-interactive Model (backend, business rules) is manipulated via a user interface programmed in the target technology. The interactive behavior, described by the Controller, is, for the largest part, implemented procedurally. The produced interactive system is iterated with, based on feedback from users and designers. In this traditional approach, designers do not produce artifacts that can be directly used by developers, since most elements of an online (running) interactive system prototype refer to the View component, but they need to be rebuilt at the developer side. There are also power issues in this traditional arrangement: developers are often more powerful than designers and even managers [START_REF] Cooper | The inmates are running the asylum[END_REF], since their work is the most expensive and their work object (software) is resistant to changes. There exist therefore technology 'viscosity' issues: one cannot iterate fast because iterations with nonprototype software are expensive. Usability problems found at later stages are difficult to address.

Example of a declarative User Interface approach

One approach to defining a declarative interaction (Controller) takes advantage of declarative constraints specified for the Model-View data correspondence (binding) and declares on which UI events these constraints should be updated. Figure 2 depicts a fully declarative user interface, where two Views of a factory production Lines containing production Tasks (diagram view at the top and table view at the bottom) show data from the same Model. The Views are connected to the Model by way of declarative SQL-like queries (studied with designers in [START_REF] Bogdan | Makumba: the Role of Technology or the Sustainability of Amateur Programming Practice and Community[END_REF]) shown in blue.

A simple declarative interaction can be illustrated in regard to the user editing in textboxes. For example the designer can choose to update e.g. the task customer name (the table left column) at each user keystroke, and the task duration (the table middle column) only when the user presses Enter or Tab. The declarative controller consists of specifying on which UI event the View should be synchronized with the Model, because the widget relation to the model is already set, similar to the drag-and-drop case.

Fig. 2. Fully declarative user interface featuring advanced interaction like drag and drop

A more advanced example of declaring interaction is drag and drop, described by its data semantics, for example a Task object can be dropped on a Line object. This information is sufficient to generate a functioning UI, because the meaning of the drop action is already governed by the View-Model connection declared for the respective container widget representing the Line. The widget representing the Line in turn contains widgets that represent Tasks. Therefore, in order to define drag and drop interaction declaratively, it is sufficient for the designer (possibly with developer help) to declare what Model objects are being dragged and on which object can they be dropped.

It is interesting to reflect that the declarative annotations that achieve the interaction exemplified above require very little code comparing to the procedural controller needed for achieving similar interaction. If we take the drag and drop example, such a procedural controller would need to define a drag and/or drop handler for each involved widget. Such a handler should do a translation between the graphical domain (widgets) to the abstract domain (Model objects), and then invoke the appropriate Model methods to implement the drag and drop semantics.

Towards a new, evolutionary UI design and development process based on declarative interaction

Based on such declarative controller definition techniques, the alternative proposed here to the traditional UI design and development process is the Declarative Interaction process, illustrated at the bottom of Figure 1 along with its potential emerging evolutions. Declarative interaction was inspired to the author through field observations and own experience [START_REF] Bogdan | IT Design for Amateur Communities[END_REF][START_REF] Bogdan | Makumba: the Role of Technology or the Sustainability of Amateur Programming Practice and Community[END_REF]. Early versions of declarative controllers were demonstrated in [START_REF] Bogdan | Makumba: the Role of Technology or the Sustainability of Amateur Programming Practice and Community[END_REF] and refined in [START_REF] Kis | Lightweight Low-Level Query-Centric User Interface Modeling[END_REF][START_REF] Kis | Generating Interactive Prototypes from Query Annotated Discourse Models[END_REF][START_REF] Kis | Declarative Setup-free Web Application Prototyping Combining Local and Cloud Datastores[END_REF][START_REF] Kis | Opportunistic Development of Data-Driven Interactive Applications[END_REF]. We have also shown a combination with MBUID approaches in [START_REF] Kis | Towards Fully Declarative High-Level Interaction Models: An Approach Facilitating Automated GUI Generation[END_REF].

The specific aspect of this process is that the designer, with input from users and customers, can work not just on the View, but also on the declarative Controller, and on the declarative parts of the Model as they are represented as annotations of the View on which the designer usually works. The developer plays a role in this process, by helping in structuring a convenient data Model, and possibly also to help express more complex interaction. However, the designer drives the iterations, changing the UI look and feel (View and Controller) while the developer only helps when relevant aspects of the data Model require adjustment. Early on, or in more advanced phases, online prototypes can be directly made using the target UI technology.

Once a final UI is decided upon, it can be tested and validated with users in regard to its usability and acceptance. After that engineers can take over the prototype and optimize it for Computer Science concerns (e.g. Security, Reliability, Scalability) using semi-automatic tools. Automatic processing (similar to model transformations) and analysis are possible thanks to the declarative nature of the interactive system representation. Since most of the design side uses such declarative representations, the results of the design phase are transferable to the engineering side, thus achieving evolutionary prototyping. In the engineering side, procedural code can be added. It is interesting to note that in this case developers work mostly on the system backend, which is why their work is termed "backend engineering".

Declarative Interaction and Conflicting UI properties

Design is a balancing act, a suite of tradeoffs that are made along the way between the needs and desires of various users and the institutions they may represent. Therefore conflicting UI design concerns are bound to occur. One way to address such conflicting concerns is the Participatory Design approach [START_REF]Design at Work[END_REF] of keeping the users involved at all stages of design, therefor ensuring that various conflicting qualities that users, designers and developers require are balanced in an acceptable way. Declarative Interaction supports the resolution of UI requirement conflicts by supporting Participatory Design through (1) facilitating equal-footing communication between users, designers and developers and (2) encouraging iterations until the late stages of the product design and implementation, thanks to its Evolutionary Prototyping nature.

Even in situations where Participatory Design is not suitable, the balanced power relation facilitated by Declarative Interaction between designers and developers is likely to achieve, through iteration, a good balance between the interactive system properties championed by designers and those guarded by developers. Especially the Interaction Evolution (Figure 1 left) iterations are also accessible to managers, letting them bring their own concerns. Therefore, iteration and the balance of power are the general process qualities that allow Declarative Interaction to support the resolution of conflicting UI design concerns.

A Case of Declarative Interaction

A European Non-Governmental Organization (NGO) with almost 100 locations developed their own systems for over 20 years: member database, document archives, summer course participant selection and management, virtual job fair, etc. All systems are tailor-made for the NGO rules and needs, ensuring greater user understanding and usability compared to general-purpose systems, if such systems are available at all. Users of such systems are 1000s of members and students of participant universities, creating 10000s of new data objects per year.

A major role in this success is played by the Makumba framework [START_REF] Bogdan | IT Design for Amateur Communities[END_REF][START_REF] Bogdan | Makumba: the Role of Technology or the Sustainability of Amateur Programming Practice and Community[END_REF]. It was designed with learning in mind, so that members have to learn two small declarative programming languages (a SQL variant for data retrieval, and HTML for data formatting). More advanced members can continue their learning path and "career" within the NGO by using Java for more complex application logic. Furthermore, a few production Makumba systems exist that use declarative SQL code for most of their application logic, including authentication and authorization, leaving just a few functions to be implemented in procedural Java code.

Reflecting on this long process, I believe that much of the success of Makumba in NGOs is due to the declarative nature of its languages. Declarative code is often small: if a declarative language suitable for a specific programming problem exists, the code will typically be more compact than the procedural correspondent. For an NGO this means less code to track and maintain. Declarative code is often intuitive to read, reducing the initial threshold that junior NGO volunteers have to face before they can contribute with code of their own. Once one makes the code work during development, declarative code is reliable to run, reducing the NGO maintenance costs. Because declarative code can be analyzed and transformed into other representations or technologies, it reduces technology lock-in for the NGO.

Another Makumba success factor stems from its facilitation of evolutionary prototyping, being therefore an early incarnation of Declarative Interaction. Systems are typically prototyped in HTML or directly in Makumba first, and are iteratively refined to become the production system. Systems are often prototyped starting from the user interface (and much of the system code is the user interface), which is intuitive and motivating for the developers, as they can work directly with the artifact that their fellow NGO members will use.

The first generation of Makumba technologies, based on Java Server Pages (JSP) and pre-Web 2.0 user interaction, is by now outdated. Kis [START_REF] Kis | Generating Interactive Prototypes from Query Annotated Discourse Models[END_REF][START_REF] Kis | Declarative Setup-free Web Application Prototyping Combining Local and Cloud Datastores[END_REF][START_REF] Kis | Opportunistic Development of Data-Driven Interactive Applications[END_REF] has explored the possibilities of combining data from a multitude of APIs into one interactive application, rather than data from a single relational database, thus using data as a "prototyping material". This has resulted in Endev [START_REF] Kis | Declarative Setup-free Web Application Prototyping Combining Local and Cloud Datastores[END_REF], a Javascript framework that is, however, not production-ready. One major reason is that, unlike Makumba, Endev does not optimize the number of queries sent to its data sources (a very complex problem when there are multiple data sources).

The current approach in modernizing Makumba recognizes that most organizations have one single main data source, and that the Makumba approach of binding data to elements of a HTML user interface has been taken by many other technologies such as AngularJS 1 . However, Angular still requires a lot of procedural Javascript code to be written, which breaks the principle of declarative development. Therefore the current approach is to develop 'plugins' for Angular and other suitable technologies to replace the current Makumba JSP layer, while keeping the declarative engines of Makumba: optimal SQL query combination, SQL query inlining for code re-use, authentication and authorization, etc. This approach is well under way and since Angular does much of the data binding that Makumba-JSP did, the resulting Angular 'plugin' is quite small and easy to maintain.

While the problem of Declarative Interaction has not yet been fully explored with Makumba, a number of experimental prototypes exist that allow fully declarative description of complex interaction such as Drag and drop, or simpler interaction such as form fill-in with UI update as the user types. One of the next steps considered is to describe declaratively various UI design pattern libraries (e.g. [START_REF] Boxes | Implementing a Pattern Library in the Real World: A Yahoo! Case Study[END_REF]) or other exemplary systems.

Declarative Interaction: Challenges for the future

There are many ways to achieve declarative interaction, the Makumba approach (using declarative queries to connect the UI to the data model) is just one. Exploring alternative declarative interaction approaches is thus one important future challenge. Is any user interaction possible to describe declaratively based on simple abstractions, familiar to designers and users? Addressing declaratively a wide palette of interaction patterns is probably the biggest challenge faces by researchers in the field.

While the impact on the human centered software engineering processes has already been considered, assessing further the impact on process is another important challenge. This includes considering further ways to empower the designers and users, but also to improve their communication with developers.

Most of this work was conceptualized within the classic Model-View-Controller paradigm. MVC is known to be sensitive to cross-cutting concerns, and an investigation of declarative interaction in relation to such concerns is therefore important. Alternatively, future work can consider other UI development approaches for declarative interaction.

Fig. 1 .

 1 Fig. 1. Traditional interactive system design and development process (top) and the new process facilitated by Declarative Interaction (bottom).


Danzhu Li
email: lidanzhu@icloud.com

Gerrit C Van Der Veer
email: gerrit@acm.org

The Evolution of Visual Art: From Painting to Interaction Design

Keywords: Interaction Design, Wearables, Stakeholders, Visual Art, Cocreation of Artistic Experiences

This paper focuses on the application of interactive technology in contemporary visual art, showing that current development has traces in history. We sketch an evolution, from the creation of primitive visual art to interaction design and wearable art. We will discuss how the artist communicates through his creation with the intended audience, and how new technology enables the art work to interact autonomously.

Introduction

Goal of our research is to attract attention from different stakeholders in visual arts, so that they discover their changing roles and appreciate the potential for unknown mutual collaboration and cooperation. In this way, we may achieve an ecological art environment that supports survival, co-creation, and development. The current paper focuses on the application of interactive technology in contemporary visual art, showing that current development has traces in history. We sketch an evolution, from the creation of primitive visual art to interaction design and wearable art. In section 2, we will show how visual art has always resulted in an artifact (a "thing") to communicate and to trigger understanding, experiences and behavior in an audience. In section 3, we will discover how in the new world this "thing" can be created to interact: Visual art is getting a true life of its own.

A Short Account of a Long History

Interactive technology is developing rapidly. The Internet of things promotes interaction design for diverse audiences and many platforms, more practical, more interesting and more approachable than ever before. As Weiser points: "The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it." [START_REF] Carlile | Ubiquitous Computing a.k.a[END_REF].

Early visual art works were intended to present images (drawings, sculptures) of important entities: gods, people, hunting. Such is prehistoric art: It is a mixed state of aesthetic and non-aesthetic factors, it does not only serve practical purposes but also shows aesthetic consciousness. Prehistoric art with its simple form and immature techniques attracts modern audiences and artists, partly because the content does not show too many ideas and values, which is also the goal of some streams in modern art [START_REF] Clottes | What Is Paleolithic Art?[END_REF].

Ancient Visitors of the Caves Understood and Experienced their Art

Prehistoric cave-or rock paintings represent the earliest forms of painting that survived, traced back to 40 thousand years ago. We experience a hint of the spiritual life of our ancestors, and we may imagine the intended audience (members of the same tribe, gods), who were supposed to (actively) interpret and understand the message as depicted. Figure 1 shows how the artist triggers his audience to see a depicted hand, where "she" (According to archaeologists, these are feminine handprints.) in fact, paints the space around the (invisible) hand -the audience will "fill in" the invisible.

The deer in Figure 2 may well represent something related to hunt, and tribal relatives of the artist will have known much more about the values, activities, and emotions related to the scene than modern viewers will ever be able to understand.

How our Ancestors Understood Hierarchy and Holiness

Many early historic paintings refer to religious belief, to the existence of a soul. and to prayers: after death people hope to go to heaven or be reborn. Ancient Egyptian murals are characterized by realism combined with deformation and decoration; hieroglyphs and images are used together, and the artist always maintains the readability (Figure 3). The picture composition is arranged with characters in a line, with different sizes according to the status hierarchy and carefully represented distances to get the image size in order. Other than this, there is a stylized regularity and unity in the form of expression, and some artistic techniques have been used continuously over many centuries to form a unique style of Egyptian art. The intended audience, whether human or god, will have understood the emotional intentions and the esthetics in relation to their interpretation of the meaning. Chinese early paintings, like early Egyptian art, use exaggeration to highlight the main characters, to distinguish their status hierarchy, e.g., see the Lo River map (Figure 4). In early Christian religious painting, the halo is used to distinguish between saint and man (Figure 5). The same technique can be found in Buddhist paintings (Figure 6) where the Buddha has a head halo and a back halo, which represents the highest level of this god. Some gods only feature a head light, indicating the differ-ence in rank. This style shows many expressive techniques in painting and sculpture, representing the meaning as well as the specific style and workmanship, which is related to a specific period. These characteristics often are used as the basis for dating.

From Painting to Writing -New Understanding Needed from the Audience

In a next stage of civilizations, series of images were used to represent spoken language, where the individual imagines were supposed to be named and the string of names was supposed to (actively) be interpreted by the audience as a spoken sentence. Examples may be found in Mayan texts (Figure 7), Egyptian hieroglyphics (see left bottom corner of Figure 3), and Sumerian cuneiform script. In each case, this type of script was used extensively for several centuries. However, in due time, the images lost pictorial details and developed into new type of "abstract" art styles, like Chinese calligraphy. Chinese characters are the only words in the oldest text that are still in use today. A different type of activity is triggered by the technique of panorama painting, displayed at a 360-degree angle, so that the audience can walk around and feel immersed in the visual representation of space. Like the Panorama Mesdag of Netherlands [START_REF] Mesdag | [END_REF].

The development of perspective drawing provided the suggestion of 3D images as rendered on a 2D surface, an early type of virtual reality, that was originally sometimes considered to be what we now would label photo-realistic. For example, (Figure 4) the perspective and composition of Chinese paintings are free and flexible. [12] Artists use this to break the limitation of time and space, aiming at a virtual reality in the viewer's mind. Later, artists took the liberty to play around and leave the interpretation of the suggested 3D work to the viewer (Figure10). In due time, the photorealistic rendering was sometimes labeled "trompe-l'oeil" -showing that the intended interpretation of "normal" painting was already beyond photo realistic, and in this way the trompe-l'oeil requires the viewer to appreciate that this is not just a precise rendering but a successful attempt to confuse the experience of reality. This shows that appreciation and interpretation develop and changes with the development of (art) history. From early 3D glasses to virtual reality, augmented reality, and so on, the audience of visual art changes in understanding, experience, and active participation.

Modern Times

In addition to the evolution of painting style and techniques, there is the development of color, material and composition. Developments are the result of artists applying new techniques, and of artists triggering their audience to give meaning and be active viewers to appreciate new types of experiences.

Visual Art Becomes an Acting Agent

Today, people can interact, talk, and touch art works in real space or by wearing equipment. Information and communication technology allows visual artists to develop active pieces of art. The art work can, in principle, be provided with sensors to be aware of the presence, the movements, the facial emotion features, and even the identity of individuals or groups of spectators. The art work could be programmed to react to spectator behavior or to trigger spectator behavior.

Experiences in the different contexts. From our analysis of museums, galleries, and international conferences we detect an amazing jump in the impact of technology on art. We will discuss some examples from the art exhibition at CHI 2016, San Jose: 'Breaking Andy Wall' (Figure11) is an interactive installation. When participants smash the canvas with the hammer, they can gradually break down the art piece.

Through the playful destruction and reconfiguration of iconic art pieces, this installation reconfigures relations between art objects and their audiences [14]. The authors of work "Source" are Jiang Xiaoyan and Liu Hanlu, the Digital Media Art Institute, Shanghai Conservatory of Music (Figure14). In the picture, objects on the wall (simulated umbrella surface) can move with the music of the GuZheng. When the audience strikes the strings, the points on the umbrella surface are gradually converging into lines, and then covering the three sides of the umbrella, demonstrating that music can be relaxing.

Art and technology

As the great French literature master Gustave Flaubert predicted, "the more time goes by, the more art will become scientific and conversely, the more science will become artistic. Both will unite at a summit after having been separated at the base". The development of contemporary art has been closely related to science and technology.

The exploration of the new art language was meant to be a combination of technology and science, interactive, and more intuitive. Thus, we see such as new media artists, audio-visual art, code artists and other new nouns of art have a significant attribute of this era.

Wearable Art -Interactive Textiles. Wearable devices are not just a hardware device supported through software, data exchange, and online interaction [START_REF] Gowrishankar | A strategy for material specific e-textile. interaction design in Smart textiles[END_REF]. Wearable devices may have powerful effect on our perception of life. Smart fabric in wearable devices is a very representative case. The trend is to make core computing modules smaller (to nanoscale units), and they are increasingly being used by artists. Philips Design gave (in 2007!) a glimpse of how will fashion look in 2020(Figure15):

The Bubelle Dress changes its look instantaneously according to wearer emotional state. It is made up of two layers, the inner layer contains biometric sensors that pick up a person's emotions and projects them in colors on the second layer, the outer textile, though limited to the sensor module and bulky looks [START_REF] Weir | Bubelle Emotion Sensing Dress[END_REF]. From participation to independent creation, will AI eventually replace artists? For most artists, the question is whether AI is liberating or constraining them. The intervention of artificial intelligence is only a small part of the development of visual arts.

It is difficult to replace human spontaneity and imagination and complex emotional systems. Artists are rich in inspiration and creativity. They try to co-create with audience. Artists have long experimented with the creation of multi-sensory experiences.

In addition to the use of wearable devices, they also attract visitors' attention with smell, touch, and so on, to communicate a richer user experience. Like (Figure18), where the group of pictures was taken at a gallery of the modern museum in Stockholm: The artist used a large amount of lavender in the exhibition hall. When visitors came close to the space, they smelled fragrances. And senses feel directly to relaxation, harmony, and reminiscent of the symbiosis of human beings and nature.

Whether artificial intelligence or the integration of senses, it's a challenge to artists. Because of the influence of computers and the internet on visual art, art creation is popular. Resource sharing and usability of interfaces and tools allow everyone to be an artist. Even people who don't have programming skills and art foundation can use tools such as Processing and Arduino to play art (there are lots of online courses and lectures). Faced to the challenge, do artists insist on their pursuit of art or make a change? How to make the vitality of art work last for a long time? How to continue and inherit the traditional art and techniques in the contemporary environment? How does the artist adapt to the contemporary environment? This is a problem for all contemporary artists. This is also a problem for all stakeholders. Thereupon: what is going on in the domain of visual art is the development of a new symbiotic relationships between cognitive systems, art, and technology. We followed examples of historical steps in visual art to interaction design. From ancient times to the present, technology and science have played a fundamental role, and people's understanding and application has been closely followed. Artists and their works will be more diverse, and the number of participants will increase. Stakeholders of current and future visual art should understand their new roles. Technology is still an alien domain for most artists. They should develop insight and learning ability for new techniques and paradigms, and consider blended creative patterns. It also requires stakeholders to study and develop components that are smaller, flexible and easy to use, so that more people will accept them. Universities should understand the importance of interdisciplinary collaboration and education. Galleries should be tolerant and encourage artists to innovate and experiment. Audiences should improve their understanding of contemporary art and become happy to co-create.

Figure 1 and 2 .

 2 Figure1 and 2. Lascaux Cave Paintings, France, Lascaux, ca 17000 BC[START_REF] Cave | [END_REF]

Figure 3 .

 3 Figure 3. Egyptian mural. ca. 1100 B.C. [4].

Figure 4 .

 4 Figure 4. Painting of the ode of the River Goddess, Gu Kaizhi, A.D. 348 -409 [5].

Figure 5 and 6 .

 56 Figure 5 and 6. Halos of Christian gods (Giotto di Bondone: Ognissanti Madonna. Italy. c. 1310) and Buddhist gods (Dunhuang Mural. China. ca. 538 AD). [6][7]

Figure 7

 7 Figure 7 Mayan text, Around the Christian era. [8]

  Figure 8 provides an impression of the development over time. Chinese calligraphy has independent aesthetic value, so it can be appreciated as a visual art. It is a technique which people learn by copying and creating their own style. The audience is supposed to appreciate the non-figural artistic qualities in relation to the meaning of the language.

Figure 8 .

 8 Figure 8. Examples of transformation of Chinese characters "Horse" over time [9]

Figure 9 and 10 .

 10 Figure 9 and 10. Han Xizai Evening Banquet, China, 937-975 AD [10]; John Gipkin, Bishop King Preaching at Paul's Cross before King James I. (1616) [13].

Figure 11 .

 11 Figure 11. Breaking Andy Wall, Interactive Art, Leo KangPace Beijing Gallery is exhibiting works of Team lab from Japan: Living Digital Forest: Lost, Immersed and Reborn (May 20 -Oct 10, 2017), e.g., Sketch Town (Figure12), a town that grows and evolves according the pictures drawn by children. The "town" in this work will be developed by all participants. After the 2-D drawings of the cars, buildings, UFOs, and spaceships are completed, they are scanned, become 3-D and enter a virtual 3-D townscape. Every component of the town has a role to play; for example, the fire trucks and cranes serve to protect the town. Children can interact with the final townscape by touching individual components to alter their behavior[START_REF]Sketch Town[END_REF].

Figure 12 .

 12 Figure12. Sketch Town, Co-Creation Art, Team lab, Japan, 2017[START_REF]Sketch Town[END_REF]

Figure 13 .

 13 Figure13. Life -two scenes from Hair, Interactive Art, Chen Yu, Beijing, 2016[START_REF] Yu | Life • Hair[END_REF]

Figure 14 .

 14 Figure 14. Source, Co-Creation Art, Jiang Xiaoyan and Liu Hanlu, Shanghai, 2017[START_REF] Jiang Xiaoyan | Source[END_REF]

Figure 15 and 16 .Figure 17 and 18 .

 161718 Figure 15 and 16. Bubelle Emotion Sensing Dress. Design group at Royal Philips Electronics. Netherlands. 2007 [19]; Fabric Strain Sensor, AdvanPro. Hong Kong [20]

Figure 19 .

 19 Figure 19. Mario Klingemann's Neurography, artificial intelligence creations. 2017[START_REF] Klingemann | Neurography-Artificial intelligence creations[END_REF]

Figure 20 . 4 New

 204 Figure 20. Most Ghosts Hold Grudges, Susanna Jablonski. Stockholm. 2017

  Technology enhances expressiveness and empowers the artist, and push some art pioneers unique in a short period. Some contemporary visual artists want to achieve interaction of art by means of wearables (VR glasses, sensor gloves, smart fabric, etc.) So where does art leads after that? As the different stakeholders of contemporary art constantly innovate, experiment and cooperate with new technology, the boundary between art and technology becomes increasingly blurred. The birth of AI gives many possibilities to art. Through deep learning and powerful and accurate algorithms, AI systems can be made to understand art, and even cultivate an artistic creativity. Figure18[23] shows AI art creations. Artist Mario Klingemann has a deep interest in human perception and aesthetic theory. He researches neural networks, code and algorithms. The advancements in artificial intelligence, deep learning and data analysis make him confident that in the near future "machine artists" will be able to create more interesting work than humans.

Acknowledgement

We thank Ernest Edmonds for his presentation on CHI 2017 that gave us inspiration to study the relation between art and technology, and we thank Elly Lammers who gave us advice and support.


Alistair Sutcliffe
email: a.g.sutcliffe@manchester.ac.uk

Conflict, Costs and Trade-Offs in User Interface Design

Keywords: Configuration, customisation, trade-offs, cost-benefit analysis

A framework for cost benefit analysis to inform design choices in configuration and customisation of user interfaces is presented. User costs of learning, UI complexity and operational usability are traded against benefits from better functional fit for the users needs. The design process is illustrated with case study experience from an eHealth application involving trade offs between conflicting requirements from two groups of users.

Introduction

The conflict between goals, needs and requirements from different stakeholders has received considerable attention in the Requirements Engineering (RE) community, where the conventional response has been to negotiate the conflicts to arrive at a common viewpoint [START_REF] Sommerville | Requirements engineering: processes and techniques[END_REF][START_REF] Robertson | Mastering the requirements process[END_REF]. Goal modelling [START_REF] Mylopoulos | From object-oriented to goaloriented requirements analysis[END_REF]van Lamsweerede, 2009) can make conflicts explicit, thereby supporting the negotiation process; however, resolution of conflicting requirements inevitably leads to compromises by some users. User interface (UI) properties, usually referred to as non-functional requirements in RE are a sub-set of the more general problem; for instance, the clash between usability, privacy and security in passwords is a well known design dilemma [START_REF] Braz | Designing a trade-off between usability and security: a metrics based-model[END_REF].

In HCI the requirements conflict-resolution process is an essential component of user-centred design (UCD) (Sutcliffe, 2002a). However, different user needs might be accommodated by different versions of the user interface, via a process of configuration or personalisation. While surface personalisation of UI features such as menu toolbars, display colours and layouts, and message terseness/verbosity, are standard components of all major operating systems, resolution of deeper functional differences between users is more problematic. Offering users choice of UI/application versions by configuration facilities imposes a cost on users when operating the configuration user interface, and most users accept the default version. The design dilemma is how to fit the requirements of diverse user groups while minimising the configuration cost and maximising the functional fit of the application to users' needs. This paper reports experiences in resolving requirements conflicts in user interfaces, approached through examining users' needs at a more fundamental level in the form of their values. Values have been explored in value-sensitive design [START_REF] Friedman | Value sensitive design[END_REF] and the related concept of worth can help to frame users' viewpoints as worth maps [START_REF] Cockton | Evolving and augmenting worth mapping for family archives[END_REF]. In Value-Based Requirements Engineering VBRE [START_REF] Thew | Value Based Requirements Engineering: method and experience[END_REF], users' values are made explicit by analysis with a reference taxonomy of values, motivations and potential emotional reactions. Making users' values explicit provides a richer context for negotiation and resolution of conflicts. The VBRE method has been applied to two case studies in health informatics. This paper first presents a framework for considering design trade offs for conflicting requirements and then describes the experience and lessons learned from the ADVISES and SAMS projects; which contribute to a discussion about different approaches and implications for conflicting requirements. UI design responses to conflicting requirements has been described by a variety of terms, e.g. personalization, customization, localization, configuration, adaptation. In this paper I use the term 'configuration' to refer to choice between alternative functions, which is close to the sense of adaptation of the user interface (Fischer 1994). Design of user interface features, such a colour, font size, menu options, which are more commonly referred to as personalization/customization are not with my remit.

A Trade-off framework for Assessing Conflict

The design response to conflicting requirements will depend on the number of user roles and the conflicting requirements demanded by those roles. As more requirements are added to the design, complexity increases, consequently the first design trade off is to consider a graded response from a single, complex product including a variety of functionality to meet each user's requirements, to separate product versions matched to each user role. A supplementary consideration is whether to provide configuration/customization facilities to enable the user to tailor a complex product to their own needs or to design separate product versions. These choices impose different costs on users and developers, summarized in figure 1. User costs are composed of learning to use the product and the configuration interface (if present), and then operating the configuration. Developer costs are the consequence of additional software development for versions and configuration facilities. Benefits accrue when the product exactly fits the user's requirements while minimizing developer and user costs. Clearly many product versions tailored exactly to individual users is the ideal, but this usually imposes prohibitive costs. The UI designer has to strive for a 'sweet spot' where development costs are constrained, while maximizing the user-system task fit, and minimizing user costs from learning and operating the UI. A further trade off emerges between increasing product complexity, which imposes learning and operating costs for the user to search and find the functions they require, in contrast to configuration facilities which enable users to select their required functions and hence reduce complexity of the operational UI. However learning and operating the configuration UI is an imposed cost.

There are further complexities the design trade off space that arise from the progression from simple configuration (i.e. by a selection menu at product set up time) to complex End User Development facilities that maximize user choice with increasing learning and operating costs. I have explored these questions in previous papers [START_REF] Sutcliffe | The Domain Theory: Patterns for knowledge and software reuse[END_REF][START_REF] Fischer | A framework for end-user development: Socio-technical perspectives and meta-design[END_REF]; in this paper I will only consider simple configuration, since this part of the trade off space relates to the forthcoming project experience.

ADVISES Experience

ADVISES is a decision-support system for academic researchers and National Health Service public health analysts who investigate epidemiology problems [START_REF] Sutcliffe | Experience with user-centred requirements engineering[END_REF]. The two distinct stakeholder communities had different goals. For academic researchers, understanding the generic causes of childhood obesity by statistical analysis of health records was a high-level goal. In contrast, the goal of public health analysts was local health management; for example, identifying where best to target interventions, such as promotion of healthy eating campaigns. Two academic research epidemiologists (both male, age 31, 52) and seven public health analysts (four male, three female, age range 27-41) were interviewed and participated in requirements workshops. VBRE augmented UCD techniques to investigate the users' workflows to explore how new decision-support tools might be used by academic epidemiologists as well as by public health professionals.

The key issues identified were the apparent contradiction between expected and actual collaboration among the stakeholders, which suggested requirements for better collaborative tools with trust-building measures, e.g. visualisation of workflows and research activities. Security and privacy of data emerged as an important value, in particular the addition of security features to customise data access to particular stakeholder roles. Collaboration, security and trust were shared values, but differences between the stakeholder emerged during design exploration of prototypes, concerning customisation, adaptability and security. These were addressed by adding requirements for data security on servers, configurable workflows to match systematic or more opportunistic processes, while creative values were supported by interactive visualisation for data analysis. Collaboration and trust were fostered by an iterative user-centred RE process to build up trust, and by implementing the system as a collaborative application.

The workflows for each stakeholder group were quite different; see Figure 2. The major functional requirements (goals) of the systems were for research and analysis support, namely database searches ranging from simple queries to complex associations between variables, leading to display of a detailed epidemiological data set in a context with map and graph overviews and functions to compare trends over time and different areas on maps. The researchers had a more sophisticated query investigation cycle and used more complicated statistical tests. In contrast, the public health analysts asked simpler questions directly related to spatial locations and used simpler statistical tests. Sociability, altruism and achievement motivations informed decomposition of stakeholder goals. For example achievement, altruism and systematic values led to a subgoal to record analytic procedures, enabling academic researchers to track their own work, while also supporting public health analysts in sharing analysis techniques and results with colleagues. Another value clash between the stakeholders was the desire by the researchers to increase the statistical rigor of the analysts' investigation. Not surprisingly the analysts saw this as an imposition into their area of competence.

Implementation

The system was implemented in C# using MS Silverlight for graphics and animating map displays for trend questions, so that successive displays gradually morphed into each other to enable users to see change over time within different map areas. A distributed architecture (Figure 3) was adopted and developed as a set of web services, with major class packages in the following functional areas:

 Dataset access: loads datasets from remote servers.  Map display: displays maps using MS Charting libraries. Map displays can be overlaid with point data (e.g. location of health clinics, sports facilities).  Charts and statistics display: runs basic statistical analysis scripts (R script calls) then displays range split histograms, box-and-whisker plots, etc., using MS Charting.  Dialogue management: handles the query interface, interactive query-bypointing and sliders.  Expert advisors: classes that implement the statistics and visualisation experts, with data set monitors to trigger advice. The prototype UI is illustrated in Figure 4. The statistics advisor was a direct response to the value clash between the users' over-rigorous analysis procedures. The resolution was to provide a statistical expert advisor which encapsulates the researchers' knowledge; however, use of the advisor was discretionary so the analysts could ignore it if they so wished. The visualisation expert embedded knowledge about which charts to select for particular data types as well as choice of colours and shading to optimise the legibility of displays. This was a consequence of an implicit value clash between the users and system designers who wished to improve display design with cognitive knowledge [START_REF] Ware | Information visualization: perception for design[END_REF][START_REF] Spence | Information visualisation[END_REF]. Fortunately both user groups were content with the visualisation expert which functioned non-obstrusively to configure the map-graph displays using a set of templates linked to frequent query types and their consequent data displays.

The original vision of ADVISES was designed to be a configurable system which could be adapted to other epidemiological applications, and in time to other e-health decision-support systems. This entailed developing adaptive data access and cleaning modules which could automatically adapt to new databases and data formats. However, it transpired that few external data sets have metadata description enabling such adaptation. Further configuration editors would have been necessary for tailoring output displays and the query interface. During the project it became clear that, technical difficulties notwithstanding, there was little appetite for developing more portable, configurable software since this served only the interests of the UK e-science programme, a remote stakeholder with less influence than the local, directly involved stakeholders (academic researchers and health analysts).

SAMS Experience

The SAMS (Software Architecture for Mental health Self management) project's main aim was to increase the proportion of dementia sufferers receiving an early diagnosis by detecting changes in their pattern of computer use [START_REF] Stringer | From click to cognition: detecting cognitive decline through daily computer use[END_REF]. At its core was a set of passive monitors that collect data as the user interacts routinely with the computer. This data is analysed to infer the stakeholders' cognitive health against a set of clinical indicators representing memory, motor control, use of language, etc. If the system detected potential problems, an alert message was sent to the user urging them to self-refer themselves to their GP for a check-up. There was a potential conflict between the clinical motivation to ensure that users responded to warning alert messages and users' need for privacy and self control.

The VBRE method was applied during interviews, scenario-storyboard requirements exploration sessions, and requirements analysis workshops. Requirements analysis was initiated with five workshops, conducted with a total of 24 participants (14 male, 10 female, age range 60-75). In the first session, the system aims, major components and operation were explained by presentation of PowerPoint storyboards illustrating design options (see Figure 5), for the alert-feedback user interface, such as choice of media (video, text, computer avatars), content (level of detail, social network) and monitoring (periodic feedback, alert-only, explicit tests). Discussion focused on privacy issues in monitoring computer use, data sharing and security, ethical considerations, emotional impact of alert messages, stakeholders' motivations and their likelihood of taking follow-up tests. Requirements issues raised in the workshops were explored further in 13 interviews presenting scenarios to illustrate similar design options with discussion on privacy, security and ethical issues. The scenarios used in both sessions were designed to test different design approaches that tacitly explored values, such as human-like presence in exploration, social networks (trust, sociability values) and explicitly probing issues of security and privacy.

Conflicts emerged in the values and requirements held by individual users as well as between end users and clinical-researcher stakeholders. End users expressed concerns over privacy and security arising from monitoring their computer use. Although they were reluctantly willing to share their data with the researchers for analysis, most participants insisted they should have control over their own data. Sharing data with their close kin/friends had to be under their control and the majority would not share information or the alert with their doctor. The majority were willing to allow monitoring of their computer use and e-mail text content, if it was anonymised to protect identity. Most participants expected to experience anxiety and fear if they received an alert message. Contact with a human expert or carer was cited as important support, with connections to support groups (e.g. the Alzheimer's Society) for reassurance (empathy) and as additional sources of information to motivate people to take follow-up tests. Users had conflicting values (privacy, efficacy, altruism) which impacted on system reliability and accuracy. While these concerns were not UI properties they did influence non-functional requirements and design of the feedback UI. Users' motivations for self control over their own health care, demanded a reliable and accurate system that detected early signs of dementia. Signs of change and usual behaviour patterns in the recorded data might indicate dementia, but they could have many other causes, such as mental health problems, e.g. depression, and not pathological causes such as mood changes. Teasing apart the signal of potential pathology from the noise of normal variation was part of the research problem. The user implications were to avoid false positive alarms. Furthermore, even true positive indications were unlikely to be 100% accurate, so potentially disturbing messages had to be delivered sensitively. This posed a further requirements dilemma. On one hand, the feedback messages needed to urge users to self refer themselves to their doctors for a check-up, but on the other, messages should not alarm people unnecessarily. The 'fear of diagnosis' problem implies complex persuasive UI design which is part of our continuing research.

Privacy and security were the most common values, with implications for controls over any data sharing, encryption, secure transmission and depersonalised data for research. These values clashed with users' motivations for monitoring so they could manage their own health (efficacy, empowerment), the desire for self control, and altruism by participating in the research which might help research on dementia. Self control was prioritised by implementing a user control to 'stop recording', and information visualisation so users could view summaries of their own activity.

Trust in the SAMS system was closely related to security, but it also involved accuracy of system information and diagnosis as well as organisational trust in the healthcare professionals. Trust-building was helped by a co-design consultation processes that involved users in the research and its results. The value clash between the need for privacy and continuous recording of users' activity resulted from the need to record as much data as possible to improve the fidelity of the analysis. This improved the effectiveness of SAMS as a research tool, and its subsequent version as a healthcare self-management system, aligned with users' self-efficacy and altruism (help research) values. The privacy goal also clashed with the researchers' motivation to record as much data as possible for research purposes. Data security was a shared concern for all stakeholders.

Implementation

To resolve the privacy clash, a UI function was provided so users could turn off data recording at their discretion. The system then prompted users to turn the recording back on after set time intervals of 5 and 10 minutes. If users did not comply after three reminders this was visible to the researchers from recording log files. They had the choice to phone the user to ask them to re set the recording. Data security was ensured by encryption of the recorded data and secure transmission to the university's server. Data depersonalisation also protected user privacy.

Preferences between users for different styles of feedback UI was addressed by providing a limited number of options which users could select when the system was set up, e.g. verbosity and tone of messages (empathetic/terse); delivery modality (text only, speech, speech plus avatar) and information provision (on/off). The latter choice allowed users access to visualisations and graphs of their recorded data on demand, with a limited set of display options of the quantity of data and summarisation. Choices were limited by the cost of configuration and developing different UI displays. To date only a limited implementation of the feedback UI has been attempted, backed up by human intervention when the system detects potential problems. The persuasive UI design with its inherent conflict between the designer's goal of persuading people to take a course of action and possibly infringing personal freedom is still to be resolved.

Discussion and Conclusions

Conflicting UI properties and, more generally, conflicting user requirements, are inherent in many systems. This paper has reported some experiences in trying to make these conflicts explicit so they can be resolved by negotiation or design. Conflicts may appear as explicit differences in stated requirements; however, frequently different viewpoints between users are tacit and need to be analysed in terms of values and motivations. Methods such as VBRE (Thew and Sutcliffe 2017) and Value Sensitive Design (Friedman, 2008) help in this endeavour.

If negotiation fails to resolve requirements conflicts, then a design response is necessary. Configuration at design time or adaptation at runtime are the usual choices. Configuration has the advantage of user participation, so they are aware of their choice and can pick design options that match their needs [START_REF] Sutcliffe | PC-RE: a method for personal and contextual requirements engineering with some experience[END_REF]. However, configuration involves user effort and most users do not use customisation features provided by operating systems, and resent having to spend time choosing configuration options. Adaptation via an intelligent monitoring and automated change saves the user effort, but the changes are chosen by the designer and the change may produce inconsistency in the UI and induce usability problems [START_REF] Fischer | User modeling in human-computer interaction[END_REF]. Apart from specialised areas such as recommender systems [START_REF] Bonhard | Accounting for taste: using profile similarity to improve recommender systems[END_REF], manual adaptation or configuration has been preferred.

However, configuration imposes learning and operational costs on users. Furthermore, the configuration options are provided by designers, and this may limit the fit between the users' needs and the design options offered. In the ADVISES system we did not implement most configuration facilities because of constraints on developer resources. This decision was a trade-off between the perceived user demand for configuration, which was estimated to be low, and the considerable software development effect necessary. ADVISES implemented a resolution of clashes between user groups by giving users control over which facilities they chose to use, in particular the statistics advisor. This choice was a compromise since it failed to satisfy the researchers' wish to enforce statistical rigor in the public health analysts' work, although it did preserve the freedom of the analysts to control their own workflow.

In SAMS the value clashes between users' desire for privacy and their self-efficacy/empowerment motivation for healthcare self-management was partially resolved by provision of a UI control to temporarily halt data recording. The potential clash between the outcome of the monitoring where emotive messages had to be conveyed has not been resolved. This is an ongoing research issue concerning persuasive technology [START_REF] Fogg | The behavior grid: 35 ways behavior can change[END_REF] where the designer or system owner's goal, i.e. to persuade the use to take a particular course of action, conflicts with ethical concerns that technology should not control people's behaviour by explicit or covert means.

In the perspective of the trade off framework in figure 1, in the ADVISES project the design choice was to increase complexity of the UI and impose more operational costs on users who had to search for the functions they needed. However, these costs were mitigated by providing limited configuration in the ability to turn the statistics advisor off. Avoiding product versions for the two user groups contained development costs. In SAMS the complexity of the monitoring UI was not an issue; however, the value analysis demonstrated than configuration at run time was vital by giving users the ability to turn off monitoring. Configuration of the persuasive users interface was not resolved during the project, although in future research we expect to provide configuration to match the style of persuasion to the user profile. This exposes the ethical dimension of user v. system control that needs to be added to the figure 1 framework.

Conflicts in the user interface may be overt in the form of different tasks, workflows or functional requirements owned by different user groups, as was the case with the researchers and public health analysts in ADVISES. In this case provision of tools to fulfil both sets of tasks is the answer. Harder to resolve are conflicts involving clashes between user values or non-functional requirements. These have to be refined into design choices which may partially satisfy one or more stakeholder groups; but as our experience has demonstrated, conflicts can often pose deep-seated irreconcilable dilemmas.

Figure 1 .

 1 Figure 1. A trade off framework for considering user and developer costs with potential benefits.

Figure 2 .

 2 Figure 2. Workflows for the research and public health analyst user stakeholders

Figure 3 .

 3 Figure 3. System architecture of the implemented modules of ADVISES (UML package format)

Figure 4 .

 4 Figure 4. ADVISES user interface showing the query results in map and graph displays

Figure 5 .

 5 Figure 5. Design options mock up illustrating avatar explaining feedback information display

Acknowledgements

This work was partial funded by EPSRC Project ADVISES (Adaptive Visualisation Tools for e-Science Collaboration and SAMS (Software Architecture for Mental Health Self Management). The author wishes to thanks colleagues and participants in both projects for their contributions towards the production of this paper.