Senior Member, IEEE Luca Turchet
email: luca.turchet@unitn.it

Mathieu Lagrange

Cristina Rottondi

Gyorgy Fazekas

Senior Member, IEEE Nils Peters

Jan Ostergaard

Frederic Font

Tom Backstrom

Senior Member, IEEE Carlo Fischione

Member, IEEE György Fazekas

G Fazekas

J Østergaard

T Bäckström

The Internet of Sounds: Convergent Trends

Keywords: Insights and Future Directions Music, Audio, Internet of Things, Internet of Musical Things, Internet of Audio Things, machine listening, music information retrieval, semantic audio, embedded systems

HAL is

I. INTRODUCTION

R ECENT developments in networking infrastructures, em- bedded systems and sound processing algorithms have opened opportunities for the integration of a variety of musical and non-musical practices within the new contexts provided by computing and networking research. In the last two decades the Internet of Things (IoT) has made its inroads in the field of Sound and Music Computing, leading to the emergence of novel paradigms such as the Internet of Musical Things (IoMusT) [START_REF] Turchet | Internet of Musical Things: Vision and Challenges[END_REF] and the Internet of Audio Things (IoAuT) [START_REF] Turchet | The internet of audio things: state-of-the-art, vision, and challenges[END_REF].

From a computer science perspective, IoMusT refers to the networks of computing devices embedded in physical objects (Musical Things) dedicated to the production and/or reception of musical content. Musical Things, such as smart musical instruments [START_REF] Font | SOURCE: a Freesound Community Music Sampler[END_REF], [START_REF] Turchet | Smart Musical Instruments: vision, design principles, and future directions[END_REF] or wearables serving a musical purpose (such as smart watches, smart bracelets, or headsets for virtual/augmented reality) [START_REF] Migicovsky | Moveosc -smart watches in mobile music performance[END_REF]- [START_REF] Turchet | Touching the audience: musical haptic wearables for augmented and participatory live music performances[END_REF], are connected by an infrastructure that enables multidirectional communication, both locally and remotely. Similarly, the IoAuT refers to the networks of computing devices embedded in physical objects (Audio Things) dedicated to the production, reception, analysis and understanding of audio in distributed environments. Networked Audio Things, such as nodes of wireless acoustic sensor networks [START_REF] Bertrand | Applications and trends in wireless acoustic sensor networks: A signal processing perspective[END_REF] or networked systems for interactive sonification [START_REF] Turchet | Interactive sonification and the iot: the case of smart sonic shoes for clinical applications[END_REF], can communicate both locally and remotely in order to, for example, gather meaningful information about the environment.

Both the IoMusT and IoAuT fields intersect the domains of embedded audio [START_REF] Vignati | A comparison of real-time linux-based architectures for embedded musical applications[END_REF], networks [START_REF] Maier | The tactile internet: vision, recent progress, and open challenges[END_REF], acoustic sensor networks [START_REF] Bertrand | Applications and trends in wireless acoustic sensor networks: A signal processing perspective[END_REF], [START_REF] Sheng | Wireless acoustic sensor networks and edge computing for rapid acoustic monitoring[END_REF], ubiquitous and pervasive computing [START_REF] Satyanarayanan | Pervasive computing: Vision and challenges[END_REF], [START_REF] Lazzarini | Ubiquitous Music Ecologies[END_REF], machine listening [START_REF] Virtanen | Computational analysis of sound scenes and events[END_REF], and human-computer interaction [START_REF] Rogers | Interaction design: beyond humancomputer interaction[END_REF]. Along the same lines, both fields lead to the emergence of novel ecosystems that form around sound-based technologies and that impact a large variety of stakeholders. In addition, IoMusT and IoAuT share common challenges related to privacy, security, interoperability and standardization. Furthermore, several systems, discussed in this paper, signal trends toward the convergence of approaches in system design, implementation and usage, in both the musical and non-musical domains.

A parallel development is the rapid rise of voice operated devices, such as smart speakers, televisions, mobile phones and even microwave ovens. For example, in 2019, the sales of smart speakers rose to 146.9 Million 1 . Such devices offer hands-free operation of information services and physical actuators, such as home automation. Many smart speaker manufacturers already offer systems with a network of interacting devices, which could be called voice operated IoT devices. A recent project, the Open Voice Network, generalizes this concept and attempts to develop a standard for communicating between interoperable voice operated IoT devices 2 .

All the trends reported above are converging towards a novel research area that we coin as the "Internet of Sounds" (IoS). The emergence of the IoS field as a discipline in its own right is not only witnessed by an increasing corpus of literature on the several and diverse underlying topics, but also by the creation of a community of academics, practitioners and industrial representatives who are gathering around a dedicated annual event, the "International Workshop on the Internet of Sounds" (arrived in 2022 to the third edition 3 ). In November 2022 the IoS community, with the aim of sharing knowledge about and foster research on IoS topics, launched the "Internet of Sounds Research Network" initiative 4 , which at the time of this writing gathers more than 90 partner institutions worldwide and more than 200 researchers have subscribed to the mailing list. In March 2023, the IEEE Communication Society accepted the proposal of the IoS community to form an "IEEE Emerging Technology Initiative on the Internet of Sounds" 5 . Moreover, the Journal of the Audio Engineering Society has promoted the field with a dedicated special issue published in 2021 6 . Furthermore, in recent years, funding bodies have supported a variety of IoS projects in both the musical 7 and non-musical 8domain, which indicates the timeliness and importance of this field. Following these trends, IoS courses at master degree and PhD levels are taught in different universities 9 .

In this paper, we present a survey of the IoS field aiming at organizing the knowledge accumulated in previous studies to build a foundation for future IoS works. To this end, we offer a review of studies related to the analysis, representation and interconnection of sound-related information, in both musical and non-musical domains. We also provide an overview of hardware and software enabling technologies for the IoS, with a particular emphasis on system architecture paradigms and applications to realistic use cases. Based on the review of tools and applicative results, we identify and discuss open challenges of this field. Our aim is not only to bridge existing research areas and communities and foster cross-domain collaborations, but also to ensure that IoS-related challenges are tackled within a shared, pluralist and system-level perspective.

We believe that the IoS has the potential to foster new opportunities for the IoT industry, paving the way to novel services and applications that are capable of exploiting the interconnection of the digital and physical realms, especially in the Smart Home and Smart City contexts. Nevertheless, for IoS technologies to emerge and be widely adopted by end users, a number of computational and human-related challenges need to be addressed.

While we are aware that in other contexts the term "audio" refers to a macro-category for musical, speech, and environmental sounds, it is worth noticing that in the specific context of this paper, we highlight the difference between the terms "music", "audio", and "sound". With "music", we exclusively refer to musical data, with "audio" we refer solely to the domain of non-musical auditory data, whereas with "sounds" we mean the union of both music and audio. Though speech, audio, sound and music are all closely related, speech and speech interfaces are such large areas of study on their own (e.g., [START_REF] Furui | Digital speech processing, synthesis, and recognition: Synthesis, and recognition[END_REF]- [START_REF] Bunz | The internet of things[END_REF]) that we choose to mostly exclude them from this work and focus on music, audio, and sound.

The remainder of this article is organized as follows. Section II introduces the conceptual basis of the IoS field. Sections III surveys works and technologies related to the IoS. Section IV discusses the main research challenges ahead of us on the IoS landscape. Finally, Section V outlines a research agenda for this area, while in Section VI we provide summarizing conclusions and final remarks.

II. THE SCIENTIFIC FIELD OF THE INTERNET OF SOUNDS

The IoS field addresses musical and non-musical domains in networked contexts. We see the IoS as the union of the two paradigms of the IoMusT and the IoAuT. As described in [START_REF] Turchet | The internet of audio things: state-of-the-art, vision, and challenges[END_REF], these two fields are intersecting. Both fields have been envisioned in [START_REF] Turchet | Internet of Musical Things: Vision and Challenges[END_REF] and [START_REF] Turchet | The internet of audio things: state-of-the-art, vision, and challenges[END_REF] as sub-fields of the general IoT field. Therefore, the IoS is a specialization of the IoT, where one of the prime objectives is to enable processing and transmission of musical and non-musical data and information.

In the proposed vision, the IoS enables the connection of digital and physical domains by means of appropriate information and communication technologies, fostering novel applications and services based on musical and non-musical information. The IoS enables the integration and cooperation among heterogeneous devices with different sensing, computational, and communication capabilities and resources, in musical and non-musical contexts as well as in co-located and remote settings.

The IoS has strong connections with and could be seen as a subfield of the Internet of Media Things (IoMT), which in turn is a subfield of the IoT. The IoMT is defined as a network of Things capable of sensing, acquiring, actuating, or processing media or metadata [START_REF] Alvi | Internet of multimedia things: Vision and challenges[END_REF]. The IoS differentiates from the IoMT for its focus on sound-based applications, whereas the IoMT also deals with other multimedia aspects, such as video. Figure 1 illustrates the positioning of the field IoS with respect to its composing sub-fields of the IoAuT and IoMusT, and to the IoT and IoMT fields.

The most critical difference between IoS and IoT is the nature of the acquired, processed and transmitted information, which in the case of the IoS is sound and other sound-related content. Moreover, the IoS requires dedicated devices able to sample, process and/or synthesize sound (especially in realtime contexts) as well as specific network architectures (for instance to support a multidirectional stream of audio packets from/to nodes that is continuous and periodic). In contrast to IoT and IoMT, the IoS may pose stringent requirements and challenges related to the collection, analysis, and communication of sound-related information. For instance, to enable geographically distant musicians to play together in real-time, the network infrastructure must ensure latencies at the order of milliseconds, to ensure no perceptible delays. Similarly, a distributed array of microphones in a wireless acoustic sensor network (WASN) deployed in an open environment might need to be synchronized tightly with low latency communications to detect audio events in real-time. Current IoT hardware, protocols and systems are insufficient to tackle those challenges. Moreover, the IoS demands a new set of analytic tools specific to the sound processing domain, which should be able to process large amounts of sound-related data and extract meaningful information given tight latency and energy constraints. This entails specific challenges in the areas of realtime signal processing and Machine Learning (ML). Furthermore, current data models (e.g., ontologies) devised for the representation of the IoT and IoMT domains are not adequate to describe the knowledge related to IoS ecosystems. This hampers the creation of ecosystems where heterogeneous IoS devices can communicate through a common, interoperable framework [START_REF] Noura | Interoperability in internet of things: Taxonomies and open challenges[END_REF].

Internet of Sounds

A. IoS definitions

Following the definitions of the IoMusT and the IoAuT reported In [START_REF] Turchet | Internet of Musical Things: Vision and Challenges[END_REF] and [START_REF] Turchet | The internet of audio things: state-of-the-art, vision, and challenges[END_REF] respectively, we provide a definition of a Sound Thing as "a networked computing device, equipped with sensors and/or actuators, with the capabilities to acquire, process, exchange, or generate sound or sound-related information". With "sound-related information" we refer to "data sensed and processed by a Sound Thing, and/or exchanged with a human or with another Sound Thing". We define the IoS as "the ensemble of Sound Things, network infrastructures, protocols and representations of sound-related information that enable services and applications for the communication of sound-related information in physical and/or digital realms".

Furthermore, similarly to what the Web of Things 10 represents for the Internet of Things [START_REF] Guinard | From the internet of things to the web of things: Resource-oriented architecture and best practices[END_REF], we use the term "Web of Sound Things" to refer to approaches taken to provide an Application Layer that supports the creation of IoS applications. 10 https://www.w3.org/WoT/ Just like the general IoT domain [START_REF] Boley | Digital ecosystems: Principles and semantics[END_REF], [START_REF] Mazhelis | Defining an internet-of-things ecosystem[END_REF], the IoS may be structured into ecosystems. An IoS ecosystem consists of IoS technologies (hardware and software platforms as well as standards) and communities of stakeholders utilizing them. From the technological perspective, the four core components of an IoS ecosystem are:

(i) Sound Things. As defined above, Sound Things are networked devices utilized to control, generate, or track responses to sonic content. Sound Things are entities that can be used in a musical or non-musical context to produce sound-related content or to observe phenomena associated to sound-based experiences. They can be connected to a local and/or remote network and act as sender and/or receiver. At hardware level, they may be equipped with sensors (in particular microphones), actuators (in particular loudspeakers), and a variety of wireless and wired connectivity options. At software level, they encompass programs that not only enable the collection, analysis, reception, and transmission of sound-related information, but also provide context-awareness and proactive capabilities. Key factors are interoperability and synchronization. The IoS vision predicts that in the future, several and new kind of musical and non-musical devices will be connected to the Internet for the users to free themselves from several constraints, such as geographical locations and synchronous presence of every stakeholder. This could have a transformative effect on how humans conduct sound-based activities and interact with sound-based devices to better comprehend the environment or perform creative tasks in a more convenient way.

(ii) Network. The IoS network infrastructure supports multi-directional wireless or wired communication between Sound Things. The interconnection of Sound Things may happen over local and/or remote networks and is achieved by means of dedicated hardware and software technologies, as well as standards and protocols that regulate the communication. Some IoS applications focusing on real-time, such as music live performance or synchronized wireless acoustic sensor networks, put particular constraints on communications. In such use cases, the connectivity infrastructure should ensure communications with low latency, high reliability, high perceptual quality, and tight synchronization between the nodes. Typically, these requirements are not present in the vast majority of IoT applications.

(iii) Storage and datasets. Most IoS ecosystems strongly rely on online storage of sound-related information, which is one of the main differentiating aspects with respect to IoT. This storage, centralized or distributed, can be of different types and refers to musical and non-musical information. One type is represented by online sound repositories, which are entities in evolution: users can update the repository by changing its content, e.g., uploading new sounds or deleting existing records. Examples of this category are Freesound 11 or Jamendo 12 . A second type consists of sound corpora. These are essentially static databases that users can download and that typically contain audio files together with pre-extracted audio features and metadata annotations. This is the case of the MTG-Jamendo dataset [START_REF] Bogdanov | The mtg-jamendo dataset for automatic music tagging[END_REF] for automatic music tagging, or FSD50K [START_REF] Fonseca | Fsd50k: an open dataset of human-labeled sound events[END_REF], a dataset for tagging of sound events. In music datasets, however, it is very common that copyright limitations do not allow the audio to be shared as part of a dataset. In that case, links to sound files are normally provided together with pre-extracted audio features. An example of this category is AcousticBrainz 13 . The third type is represented by test beds, i.e., platforms to facilitate diverse research through providing collections of datasets with meta-level annotations. The Open Multitrack Testbed 14 [START_REF] De Man | The Open Multitrack Testbed[END_REF], for example, can be used to search multitrack recordings with a specific set of instrumentation or stems, and provides a browsing and searching interface with metadata filters related to specific devices used in music production.

(iv) Applications and services. Different kinds of applications and services can be created on top of the connectivity infrastructure, which target a variety of stakeholders. Such applications and services may have an interactive or a noninteractive nature. To establish interactive applications, realtime computations have a particular importance. Analogously to the IoT field, the IoS can leverage Web Application Programming Interfaces (APIs) and Web of Things architectures devised to serve sonic purposes. Services can be exposed by Sound Things via Web APIs. Applications are part of a higher layer in the Web of Sound Things architecture letting users interact with content or Sound Things directly.

Figure 2 depicts the main technological components of an architecture supporting a generic IoS ecosystem. The data flow exchanged via wireless and/or wired links between such components can be grouped into: 13 https://acousticbrainz.org/ 14 http://multitrack.eecs.qmul.ac.uk/ 1) Captured sound streams: some Sound Things are equipped with microphones to collect data from the user (e.g., a networked music performance system [START_REF] Cáceres | Jacktrip: Under the hood of an engine for network audio[END_REF]) or from the environment (e.g., nodes of WASNs [START_REF] Bello | SONYC: A System for Monitoring, Analyzing, and Mitigating Urban Noise Pollution[END_REF]); 2) Features computed on analyzed sounds: some Sound Things have the capability of performing real-time or offline analysis of the sounds captured from microphones (e.g., a smart musical instrument [START_REF] Turchet | Real-time hit classification in a Smart Cajón[END_REF]), while online storage and datasets can provide the features upon queries; 3) Control messages: beyond sound or sound-related information, the components of an IoS ecosystem may exchange messages to control the behaviors of each other (e.g., data sent to control sonic shoes used in rehabilitation applications [START_REF] Turchet | Interactive sonification and the iot: the case of smart sonic shoes for clinical applications[END_REF] or used to query online sound repositories [START_REF] Font | SOURCE: a Freesound Community Music Sampler[END_REF]); 4) Sound streams for reproduction: some Sound Things encompass loudspeakers which enable to reproduce sound that is synthesized (e.g., by smart speakers [START_REF] Turchet | Voice-based interface for accessible soundscape composition: composing soundscapes by vocally querying online sounds repositories[END_REF] or smart musical instruments [START_REF] Turchet | On making physical the control of audio plugins: the case of the Retrologue Hardware Synthesizer[END_REF]), played back (e.g., from online sound repositories [START_REF] Skach | Embodied interactions with e-textiles and the internet of sounds for performing arts[END_REF]), or received from other Sound Things as a real-time flow (e.g., in networked music performances [START_REF] Rottondi | An overview on networked music performance technologies[END_REF]).

B. IoS stakeholders

The IoS field impacts a large variety of stakeholders, which has important implications at societal and economical levels. As far as the IoMusT is concerned, the stakeholders include amateur and professional musicians, performers, composers, conductors, studio producers, live sound engineers, audience members, students, teachers, schools of music, record labels, publishers, musical instruments manufacturers, concert venues, and musical services providers. Regarding the IoAuT, stakeholders may include all actors involved in ecoacoustics

Stakeholder Benefit

Amateur and professional musicians (performers, composers, students and teachers, sound producers, live sound engineers)

• Novel kinds of individual and collaborative musical activities via networked musical devices: new forms of musical expression, pedagogy, interaction with musical content, collaboration, remote participation, and remote control. • Novel forms of monetization via blockchain technology: new forms of revenue streams for artists, music distribution without intermediaries.

Audiences

• Enjoyment of new artistic forms based on networked interactions between performers, between performers and audience members, as well as between audience members. • New forms of revenue streams specific for audiences via blockchain technology.

Doctors, patients and the healthcare sector

• Novel forms of sound-based therapies in networked contexts (e.g., tele-rehabilitation).

Citizens at large

• Possibility for policy makers of defining measures to reduce acoustic pollution in cities via the monitoring of the urban soundscape through WASNs. • Real-time monitoring of dangerous situations in cities (e.g., gunshots) and remote surveillance.

• Real-time synchronized acoustic information provided to users at large indoor or outdoor spaces (such as hospitals or stadiums) via a distributed network of loudspeakers. • Low-latency and highly reliable voice communications (via phone or videoconferencing systems).

Wildlife and environment

• Possibility for policy makers of defining measures to safeguard wildlife via ecoacoustics monitoring methods leveraging WASNs. • Possibility for policy makers of defining measures to improve the quality of the urban environment by considering monitoring methods leveraging WASNs. • Environmental sustainability: reduced pollution by zeroing the travels of musicians in need to meet for collaborative musical activities, as well as the travels of patients in need to go to hospitals for the soundbased therapies. • Real-time monitoring and predictions of volcanoes evolution based on vibration and sound analysis to alert citizens.

Manufacturing industry and machine operators

• Sound-based predictive maintenance and automatic sound-based anomaly detection in manufacturing machines or other systems handled by human operators (e.g., large motors of ships).

Audio service providers

• Availability of novel technologies enabling the creation of new types of sound-based services, such as context-aware recommendation of musical content.

[37], smart city [START_REF] Bello | SONYC: A System for Monitoring, Analyzing, and Mitigating Urban Noise Pollution[END_REF] or smart homes contexts who are using audio-based services. In addition, relevant stakeholders are Audio Things manufactures and audio-based service providers. Furthermore, common to both fields there are stakeholders such as telecommunication companies, institutions and regulatory bodies.

All such stakeholders will form ecosystems around IoS technologies, which will allow them to interact and avail themselves of innovative services. Sound Things will be able to support the activities of such stakeholders, thanks to their embedded intelligent capabilities (such as context-awareness and proactivity [START_REF] Perera | Context aware computing for the internet of things: A survey[END_REF], [START_REF] Lefford | Context aware intelligent mixing systems[END_REF]). Table I summarizes the potential benefits for different stakeholders brought about the implementation of the IoS paradigm.

C. Types of interactions

Different kinds of interaction can occur within IoS ecosystems, which depend on the following aspects.

Entities: Sounds and sound-related information can be exchanged between i) machines (e.g., nodes of WASNs), ii) humans (e.g., collaborative music making using a networked music performance system), iii) machines and humans (e.g., queries from a smart musical instrument to an online sound repository).

Temporal aspects: interactions between the entities above can be i) real-time (e.g., via networked music performance systems or via smart instruments that repurpose the results of the analysis of sounds at the moment in which they are generated) or non real-time (e.g., offline analysis of sounds in online repositories); ii) synchronized (e.g., nodes of a WASNs sharing the same clock or musical devices sharing the same tempo) or asynchronized (e.g., musical performances not needed of tight synchronizations).

Spatial aspects: interactions may be i) co-located, when entities share the same physical location, or ii) remote, when entities are geographically displaced.

Content: sound and sound-related information in the IoS may be i) musical or ii) non-musical, which lead to radically different interactions between the entities.

Directionality: interactions between entities may be i) one-to-one, ii) one-to-many, iii) many-to-one, or iv) many-to-many.

III. RELATED WORK

In this section we review the state of the art in relation to both the musical and non-musical domains, with a focus on the most recent studies. The review is not meant to be exhaustive, rather we aim to describe the results of various application domains that lead to the emergence of the IoS field. We first review studies specific to the IoMusT, secondly those specific to the IoAuT, and then those that are relevant to both fields. We survey both hardware and software systems, along with application and services. In Figures 3,4, 5, 6 and 7, we provide schematic diagrams of IoS architectures supporting various applications and services, detailing the main software and hardware components and their integration.

A. Relevant works in the IoMusT

The IoMusT research field originates from the integration of many lines of existing research including networked music performance systems [START_REF] Rottondi | An overview on networked music performance technologies[END_REF], [START_REF] Gabrielli | Wireless Networked Music Performance[END_REF], ubiquitous music [START_REF] Lazzarini | Ubiquitous Music Ecologies[END_REF], new interfaces for musical expression [START_REF] Jensenius | A NIME Reader: Fifteen Years of New Interfaces for Musical Expression[END_REF], music information retrieval [START_REF] Burgoyne | Music information retrieval[END_REF], human-computer interaction [START_REF] Rowland | Designing connected products: UX for the consumer Internet of Things[END_REF], Musical XR [START_REF] Turchet | Music in extended realities[END_REF], participatory art [START_REF] Hödl | Design implications for technology-mediated audience participation in live music[END_REF], IoT [START_REF] Borgia | The Internet of Things vision: Key features, applications and open issues[END_REF] and aspects of semantic audio [START_REF] Fazekas | Semantic Web and Semantic Audio Technologies[END_REF] combining web and audio technologies [START_REF] Sandler | Semantic web technology for new experiences throughout the music productionconsumption chain[END_REF], [START_REF] Fazekas | An overview of Semantic Web activities in the OMRAS2 project[END_REF].

In the following we survey the essential IoMusT components and underlying technologies.

1) Networked music performances: In essence, Networked Music Performances (NMP) systems are real-time audio/video streaming applications aimed at supporting remote musical interactions among performers placed in different physical locations. Nowadays, several experimental and commercial NMP solutions are available (see, e.g., [START_REF] Cáceres | Jacktrip: Under the hood of an engine for network audio[END_REF], [START_REF] Turchet | Elk Audio OS: an open source operating system for the Internet of Musical Things[END_REF]- [START_REF] Carôt | Distributed network music workshop with soundjack[END_REF]). The ongoing Sars-CoV-2 pandemic has significantly fostered their development, due to the sudden need to support the daily activities of musicians and music schools while adhering to social distancing rules enforced during lockdowns. Literature from the past two decades also reports a wide range of demonstrations of networked musical performances (some recent examples include [START_REF] Olmos | Exploring the role of latency and orchestra placement on the networked performance of a distributed opera[END_REF]- [START_REF] Bosi | Unlocking remote music performance during the lockdowns: a networked concert with jacktrip[END_REF]).

To achieve performative conditions as similar as possible to those experienced by musicians in traditional in-presence settings, the mouth-to-ear latency perceived by the performers must be extremely low: according to several studies appeared in the last decade [START_REF] Chafe | Effect of temporal separation on synchronization in rhythmic performance[END_REF]- [START_REF] Rottondi | Feature-based analysis of the effects of packet delay on networked musical interactions[END_REF] it should not exceed 20-30 ms. This is the amount of time taken by a sound wave to propagate in air 8-10 meters (which is usually considered the maximum physical displacement tolerated by a musical ensemble to synchronously perform together in absence of additional reference tempo cues). Above 30ms, latency becomes clearly perceivable and typically has a negative impact on the performance quality, as it makes the maintenance of tempo stability and of the desired musical interplay progressively harder, due to the fact that perceiving the counterpart as "late" generates a tendency towards tempo deceleration. For this reason, research efforts have recently been devoted to the integration of artificial metronomes capable of providing adaptive audio cues to the remote musicians [START_REF] Battello | An adaptive metronome technique for mitigating the impact of latency in networked music performances[END_REF], [START_REF] Hupke | Effect of a global metronome on ensemble accuracy in networked music performance[END_REF].

The overall mouth-to-ear latency includes multiple contributions introduced by different stages of the audio acquisition, processing, transmission and reproduction processes. Due to the packet-switching approach adopted in Internet-based networks, audio data are subdivided in chunks and packetized, then each packet is routed individually from source to destination through the telecommunication network infrastructure. It follows that consecutive packets may experience different delays, as some latency components (e.g., the queueing time in intermediate routers) exhibits variations along time. The variation of the interarrival times between consecutive packets is named "jitter". Jitter excursions and packet losses may cause audio portions to arrive too late (or not arrive at all) to be reproduced at the receiver side, thus causing audio artifacts (note that conventional packet retransmission mechanisms cannot be adopted in NMP applications, as they would further increase the mouth-to-ear latency). By introducing a delay, the variance of the jitter can be reduced through a combination of jitter buffers and forward error correction codes [START_REF] Sørensen | Delay minimization in real-time communications with joint buffering and coding[END_REF]. Another source of audio impairments is the drifting effect caused by the imperfect synchronization of local clock oscillators, which leads to buffer over/underruns due to the deviation between the number of audio samples generated by the sender and the number of those reproduced by the receiver during a given time period [START_REF] Werner | Unison: A novel system for ultra-low latency audio streaming over the internet[END_REF], [START_REF] Ferguson | Trans-europe express audio: testing 1000 mile low-latency uncompressed audio between edinburgh and berlin using gps-derived word clock, first with jacktrip then with dante[END_REF]. To mitigate the impact of such artifacts without introducing a delay, robust audio codecs designed for ultralow-latency streaming applications over imperfect networks can be used. These include audio coders where packet loss concealment is integrated (such as OPUS [START_REF] Valin | Definition of the opus audio codec[END_REF] and [START_REF] Pflug | Robust ultra-low latency soft-decision decoding of linear pcm audio[END_REF]) or multiple-description audio coders, where audio frames are encoded into several redundant packets which individually yield a desired quality, and where any combination of packets improves the quality [START_REF] Arean | Multiple description perceptual audio coding with correlating transform[END_REF]- [START_REF] Østergaard | Low delay robust audio coding by noise shaping, fractional sampling, and source prediction[END_REF].

2) Musical Things: Various types of Musical Things have been proposed by the IoMusT research community (e.g., [START_REF] Font | SOURCE: a Freesound Community Music Sampler[END_REF], [START_REF] Thorn | Telematic wearable music: Remote ensembles and inclusive embodied education[END_REF], [START_REF] Turchet | On making physical the control of audio plugins: the case of the Retrologue Hardware Synthesizer[END_REF], [START_REF] Keller | The handy metaphor: Bimanual, touchless interaction for the internet of musical things[END_REF], [START_REF] Turchet | Examples of use cases with smart instruments[END_REF]), along with frameworks to connect them (e.g., [START_REF] Fraietta | Rapid composition for networked devices: HappyBrackets[END_REF]- [START_REF] Centenaro | Towards a 5G Communication Architecture for the Internet of Musical Things[END_REF]).

The so-called Smart Musical Instruments are one of the most prominent instances of Musical Things [START_REF] Turchet | Smart Musical Instruments: vision, design principles, and future directions[END_REF]. This is an emerging class of musical instruments characterized by sensors, actuators, wireless connectivity, and embedded intelligence. Such instruments are the result of the integration of various technologies including IoT, sensor-and actuator-based augmented instruments, embedded acoustic and electronic instruments, NMP systems, as well as methods for sensor fusion, audio pattern recognition, and semantic audio. Smart musical instruments are envisioned to have advanced intelligence capabilities, such as context-awareness, adaptation, and proactivity. Nevertheless, up to now, only a few musical instruments exist in both industry and academia which encompass such features.

Examples from industrial research are Smart Acoustic Guitar by HyVibe and the Sensus Smart Guitar developed by Elk [START_REF] Turchet | Examples of use cases with smart instruments[END_REF].

Examples in academic research are the Smart Cajón reported in [START_REF] Turchet | Real-time hit classification in a Smart Cajón[END_REF], the SOURCE sampler detailed in [START_REF] Font | SOURCE: a Freesound Community Music Sampler[END_REF], or the Smart Mandolin described in [START_REF] Turchet | Smart Mandolin: autobiographical design, implementation, use cases, and lessons learned[END_REF].

A number of innovative applications associated to such instruments are also emerging. The authors of [START_REF] Turchet | An ubiquitous smart guitar system for collaborative musical practice[END_REF] propose a smart guitar system that uses the instrument as a hub for collaborative music making over a local wireless network. Thanks to this system, performers using musical apps on mobile devices produce sounds by wirelessly controlling the instrument's sound engine. At the same time, the player can not only actually play and control other parts of the instrument's sound engine, but can also send messages to the connected mobile devices changing the configuration of the app. In a different vein, the system described in [START_REF] Turchet | Cloud-smart musical instrument interactions: Querying a large music collection with a smart guitar[END_REF] reports a smart guitar application enabling the use of distributed intelligence, via cloud computing and edge computing paradigms, for recreational music making and learning contexts. Thanks to direct internet connectivity and embedded processing, the Results of anomaly detection algorithms

Alert system and countermeasures

Hardware:

-Microphone -Embedded system -Wireless module

Software: -Acoustic features extraction algorithms -Detection of anomalous events

Machine to be monitored Fig. 7. Diagram of a networked, sound-based anomaly detection system for the monitoring of manufacturing machines.

instrument sends requests of desired musical pieces to online music repositories and reproduces the downloaded music. In particular, the search is performed using musical features, such as tempo and chords, which are extracted by the instrument's capabilities, rather than utilizing conventional textbased search such as the title of the song or the artist's name.

Wearable devices utilized for musical purposes are another category of Musical Things [START_REF] Thorn | Telematic wearable music: Remote ensembles and inclusive embodied education[END_REF], [START_REF] Skach | Embodied interactions with e-textiles and the internet of sounds for performing arts[END_REF]. For instance, headsets for virtual or augmented reality can be considered as Musical Things if used in networked musical applications and in conjunction with other Musical Things (see e.g., [START_REF] Turchet | Examples of use cases with smart instruments[END_REF], [START_REF] Men | Lemo: Supporting collaborative music making in virtual reality[END_REF]). However, this line of research has thus far received remarkably little attention, despite its great potential for performance, composition, and education [START_REF] Turchet | Music in extended realities[END_REF], [START_REF] Loveridge | Networked music performance in virtual reality: Current perspectives[END_REF]. On the one hand, Musical Haptic Wearables, are an emerging class of wearable devices embedding haptic stimulation, tracking of gestures and/or physiological parameters, and wireless connectivity features. On the other hand, such devices were devised to en-hance communication between performers, as well as between performers and audience members by leveraging the sense of touch in both co-located and remote settings [START_REF] Turchet | Co-design of Musical Haptic Wearables for electronic music performer's communication[END_REF], [START_REF] Turchet | Musical Haptic Wearables for synchronisation of visually-impaired performers: a co-design approach[END_REF]. They were devised to enrich musical experiences of audiences of music performances by integrating haptic stimulations, as well as provide novel capabilities for creative participation thanks to embedded sensor interfaces [START_REF] Turchet | Touching the audience: musical haptic wearables for augmented and participatory live music performances[END_REF].

3) IoMusT connectivity systems: Recent endeavors in IoMusT research have explored the creation of dedicated systems to interconnect Musical Things, paving the way to the creation of IoMusT ecosystems. Some scholars have proposed frameworks that allow one to exchange information within artistic installations [START_REF] Matuszewski | A web-based framework for distributed music system research and creation[END_REF], [START_REF] Bown | Supporting creative practice in wireless distributed sound installations given technical constraints[END_REF] or performance settings leveraging the real-time dimension [START_REF] Bown | Facilitating creative exploratory search with multiple networked audio devices using happybrackets[END_REF]- [START_REF] Vieira | Sunflower: an environment for standardized communication of iomust[END_REF].

Other authors have proposed preliminary architectures based on Semantic Web technologies to foster interoperability across heterogeneous Musical Things. The semantically-enriched IoMusT architecture reported in [START_REF] Turchet | Towards a Semantic Architecture for Internet of Musical Things applications[END_REF] relies on a semantic audio server, embedded audio systems, and edge computing techniques. In particular, the SPARQL 15 (SPARQL Protocol and RDF Query Language) Event Processing Architecture described in [START_REF] Roffia | Dynamic Linked Data: A SPARQL Event Processing Architecture[END_REF] was used as an interoperability enabler allowing multiple prototypes of Musical Things to cooperate. However, Semantic Web technologies are typically not suitable for dealing with real-time aspects of IoMusT applications, as the Semantic Web stack is oriented towards scenarios where information evolves at a low rate. To cope with this issue, Viola et al. improved the architecture reported in [START_REF] Turchet | Towards a Semantic Architecture for Internet of Musical Things applications[END_REF] by using CoAp, a lightweight IoT protocol for machine-to-machine communication [START_REF] Viola | C Minor: a Semantic Publish/Subscribe Broker for the Internet of Musical Things[END_REF].

Such architecture has been further improved and extended, leading to the Musical Semantic Event Processing Architecture (MUSEPA), a semantically-based architecture designed to meet the IoMusT requirements of low-latency communication, discoverability, interoperability, and automatic inference [START_REF] Turchet | Semantic Web of Musical Things: achieving interoperability in the Internet of Musical Things[END_REF]. MUSEPA uses at its core the Internet of Musical Things Ontology, an ontology dedicated to the representation of knowledge related to the IoMusT domain [START_REF] Turchet | The internet of musical things ontology[END_REF].

4) Music Information Retrieval: The wider fields of Music Informatics and Semantic Audio are both concerned in part with the extraction of information from audio signals, signifying the relevance of these fields to both IoAuT and IoMusT and the broader IoS and IoMT domains. Semantic Audio goes beyond typical concerns of Music Informatics in its use of explicitly structured data and applications outside the music domain as discussed in Section IV-C. As a field related to both, Music Information Retrieval (MIR) has seen an increase in its multidisciplinarity, and it is now commonly thought to cover research and techniques for music understanding and modelling that use information processing methodologies [START_REF] Serra | Roadmap for music information research[END_REF]. This includes music generation as well as cognitive or musicological analyses of different representations of music.

Core research in MIR, however, still focuses on tasks such as key and chord recognition [START_REF] Pauws | Musical key extraction from audio[END_REF], [START_REF] Fujishima | Realtime chord recognition of musical sound: a system using Common Lisp Music[END_REF], tempo and beat tracking [START_REF] Davies | Casual tempo tracking of audio[END_REF], the detection of musical note onsets [START_REF] Bello | A tutorial on onset detection in music signals[END_REF], [START_REF] Turchet | Hard real time onset detection for percussive sounds[END_REF], automatic music transcription [START_REF] Klapuri | Signal processing methods for the automatic transcription of music[END_REF], classification [START_REF] Tzanetakis | Musical genre classification of audio signals[END_REF], and description (also known as captioning) [START_REF] Choi | Towards music captioning: Generating music playlist descriptions[END_REF], [START_REF] Manco | Muscaps: generating captions for music audio[END_REF] as well as music emotion recognition [START_REF] Barthet | Music Emotion Recognition: From Content-to Context-Based Models[END_REF]- [START_REF] Turchet | Music emotion recognition: intention of composers-performers versus perception of musicians, non-musicians, and listening machines[END_REF]. A large body of research considers musical audio in these tasks to support search, retrieval and interaction use cases. This makes MIR especially relevant for the increasing number of interconnected IoS devices such as smart speakers, networked media players, set top boxes, general purpose intelligent assistants [START_REF] Azvine | The intelligent assistant: An overview[END_REF] as well as more specific devices such as smart instruments [START_REF] Turchet | Smart Musical Instruments: vision, design principles, and future directions[END_REF].

Conventional key and chord estimation techniques typically share a similar signal processing pipeline. A time-frequency representation such as a spectrogram or constant-q-transform is obtained first from the audio followed by aggregation steps, e.g., a pitch class profile or chroma feature calculation for every time frame. The objective is to fold energies from different octaves into a feature that is robust to timbre variations. The features are finally pooled over time and compared with templates for each chord or key. The best matching template determines the label. ML techniques, such as a hidden Markov model [START_REF] Sheh | Chord segmentation and recognition using EM-trained hidden Markov models[END_REF] may be used to smooth transitions or impose a musicological model to improve accuracy over raw labels extracted from the signal directly. More recently, endto-end feature learning approaches have been proposed for these tasks [START_REF] Korzeniowski | End-to-end musical key estimation using a convolutional neural network[END_REF], [START_REF] O'hanlon | Fifthnet: Structured compact neural networks for automatic chord recognition[END_REF], where typically a convolutional neural network frontend is applied to process a time-frequency signal representation into increasingly higher level features [START_REF] Choi | A tutorial on deep learning for music information retrieval[END_REF]. This is followed by dense layers and softmax classification to determine the label. A comprehensive survey of the development of chord recognition is provided in [START_REF] Pauwels | 20 years of automatic chord recognition from audio[END_REF].

Typical approaches to determine the onset of musical notes involve the calculation of an onset detection function (ODF), e.g. using time-domain energy fluctuation or zero crossing rate and frequency-domain techniques such as spectral difference or phase deviation [START_REF] Bello | A tutorial on onset detection in music signals[END_REF]. The ODF is then smoothed and its peaks are picked to find note onsets. The time-frequency representation of the signal may be pre-processed to reduce the effects of instrument playing techniques such as vibrato [START_REF] Böck | Maximum filter vibrato suppression for onset detection[END_REF], while different acoustical features may be fused in order to emphasise less salient notes in the signal [START_REF] Tian | Design and evaluation of onset detectors using different fusion policies[END_REF]. Neural networks have also been employed in this task [START_REF] Schlüter | Improved musical onset detection with convolutional neural networks[END_REF], where a convolutional frontend followed by dense layers is used to compute an ODF. While deep learning represents the state of the art in terms of accuracy, in IoS applications with hard real-time and energy constraints, signal processing methods are still relevant [START_REF] Turchet | Hard real time onset detection for percussive sounds[END_REF].

Automatic music transcription (AMT) is concerned with analysing an acoustic signal to extract parameters of the sound corresponding to traditional music notation or other symbolic representation that allows reproducing the piece using musical instruments or synthesizers [START_REF] Klapuri | Signal processing methods for the automatic transcription of music[END_REF]. This is relevant to IoS as this allows advanced search, for instance, by matching melodic patterns, or advanced interaction through altered resynthesis. AMT is often divided into sub-problems such as note onset detection and multi-pitch estimation, however, Non-Negative Matrix Factorisation (NMF) and Neural Networks have been predominant in the field in recent years [START_REF] Benetos | Automatic music transcription: An overview[END_REF]. These aim to jointly estimate the spectro-temporal characteristics of musical notes. NMF considers a non-negative time-frequency signal representation as the product of a dictionary and an activation matrix which are iteratively updated to minimise the divergence between e.g. a spectrogram and the matrix product. Neural networks for AMT use one or more timefrequency representations as input and aim to estimate note onsets, pitch and duration. A notable example is Google Brain's Onsets and Frames Network [START_REF] Hawthorne | Onsets and frames: Dual-objective piano transcription[END_REF]. State-of-the-art approaches aim to capture short and long-term characteristics of the signal, e.g. the temporal evolution of note spectra as well as dependencies between musical notes. For example, the model proposed in [START_REF] Hawthorne | Sequence-to-sequence piano transcription with transformers[END_REF] represents audio features and language-like dependencies jointly.

The association of semantic labels such as genre or mood with audio signals is also very prominent in MIR. Instrument, genre or emotion recognition may be considered separate tasks, or part of a multi-label classification problem known as automatic music tagging. In the IoS domain, these labels facilitate retrieval. Music Emotion Recognition (MER) is conventionally treated as a supervised ML problem. Acoustic features corresponding to cognitive or psychological factors may be selected, then models are trained to predict either categorical mood labels (e.g. happy vs. sad) or a continuous representation of moods (e.g. arousal and valence). Features may also be selected adaptively [START_REF] Saari | Genre-adaptive semantic computing enhances audio-based music mood prediction[END_REF] or using a data-driven approach [START_REF] Saari | Semantic models of musical mood: Comparison between crowd-sourced and curated editorial tags[END_REF], [START_REF] Baume | Selection of audio features for music emotion recognition using production music[END_REF]. A comprehensive survey of MER can be found in [START_REF] Barthet | Music Emotion Recognition: From Content-to Context-Based Models[END_REF]. Similar pipelines have been dominant in instrument and genre recognition too [START_REF] Tzanetakis | Musical genre classification of audio signals[END_REF], but state-of-theart results are now obtained using end-to-end deep learning models. These tasks are increasingly considered part of a broader music tagging problem [START_REF] Choi | Transfer learning for music classification and regression tasks[END_REF], [START_REF] Choi | The effects of noisy labels on deep convolutional neural networks for music tagging[END_REF]. Convolutional and recurrent networks are combined in [START_REF] Choi | Convolutional recurrent neural networks for music classification[END_REF] for predicting labels corresponding to genre, mood, instrument and musical era. A musically motivated front-end focusing on timbral and temporal features simultaneously is proposed in [START_REF] Pons | End-to-end Learning for Music Audio Tagging at Scale[END_REF], while features extracted from a convolutional network trained on tags is shown to perform well across a range of MIR tasks in [START_REF] Choi | Transfer learning for music classification and regression tasks[END_REF]. Generating full sentence descriptions of a music piece may be considered an extension of the tagging problem. This involves the use of an acoustic model and a large language model [START_REF] Manco | Muscaps: generating captions for music audio[END_REF].

In the broader context of interconnected IoS devices, MIR can facilitate new mechanisms for search and retrieval as well as new interactions with large music collections on the Web. Processing audio queries can be handled for instance by extracting features locally in the IoS device using MIR techniques, and using these features in the retrieval process. A relevant application is demonstrated in [START_REF] Turchet | Cloud-smart musical instrument interactions: Querying a large music collection with a smart guitar[END_REF] in the context of a smart guitar, comparing keyword search with both cloud and edge computing approach to retrieve content using tempo, chords, key and tuning features. Earlier applications for handheld IoS devices (e.g., tablets) demonstrate how MIR facilitates music learning in an IoS context [START_REF] Barthet | Music recommendation for music learning: Hotttabs a multimedia guitar tutor[END_REF], [START_REF] Pauwels | Exploring real-time visualisations to support chord learning with a large music collection[END_REF], while both retrieval and repurposing of sounds is exemplified in [START_REF] Xambo | Live repurposing of sounds: Mir explorations with personal and crowdsourced databases[END_REF].

B. Relevant works in the IoAuT

Similarly to the IoMusT, the IoAuT research field is positioned at the confluence of different disciplines. Beside the IoT and networking, these include wireless acoustic sensor networks, sound and music computing, sonic interaction design, semantic audio, artificial intelligence, and human-computer interaction.

1) Acoustic Sensor Networks: As the impact of human activities on the health of humans and the natural environment is getting more and more important, there is an urge to reliably gather information about our environment that allows us to take action. To tackle this issue, one approach is to foster on the advent of low-cost microphones based for example on the MEMS technology [START_REF] Picaut | Lowcost sensors for urban noise monitoring networks-a literature review[END_REF] to monitor our environment. In order to produce reliable, interpretative and privacy preserving information using this kind of technology, it has to be done at scale, on the edge, and in a efficient manner. The audio modality has many advantages. Its is contactless, non invasive, omnidirectional, provides features of relatively low bandwidth and can operate without light. Those advantages lead to the widespread of application scenarios in the recent years, mostly for surveillance and monitoring.

Gathering information about our environment using the acoustic modality requires a complex processing chain. The pipeline usually consists in a microphone, a processing unit for computing low-rate features, and another processing unit that, from those features, predicts high-level attributes such as the presence of sources of interest. Lastly, the audio, the features and the high-level attributes may be consumed via gateways or stored using storage facilities. Some constraints have to be met when placing key components over the graph that constitutes the network of sensors and servers. For example, the microphone must obviously be located in the sensor. For the others, the design choices leading to the correct network architecture is task-specific [START_REF] Gontier | An efficient audio coding scheme for quantitative and qualitative large scale acoustic monitoring using the sensor grid approach[END_REF] depending on the correct balance between privacy, energy harvesting, security and reliability for the project at hand. Traditional acoustic array processing allows for spatio-temporal processing of the surrounding sound field, and is frequently used for detection and localization of sources, analyses of signals, enhancement and noise reduction, etc. In Acoustical Sensor Networks (ASNs), the wireless microphone sensors can be placed at arbitrary locations and is not restricted to a fixed grid. This provides a high degree of flexibility but also introduces challenges such as synchronization issues between the sensors, artefacts due to audio coding and transmission errors, potentially unknown and time-varying network topology and quality, and energy restrictions in the case that the sensors are battery driven.

Predicting high level attributes from those sensors is done using machine listening software components that detect sound events and classify them into application specific ontologies. The now well-established DCASE challenge 16 provides a unique venue fostering academic and industrial efforts towards effective detection or classification models [START_REF] Stowell | Detection and classification of acoustic scenes and events[END_REF], [START_REF] Politis | Overview and evaluation of sound event localization and detection in dcase 2019[END_REF] for standardized tasks that can serve as reference when considering real application scenarios. The challenge allows the community to compare different inference techniques in a standardized setting geared towards replicability and fairness of comparison. A companion workshop is held that allows the challenge participants to describe and discuss their design choices. Besides striving for more and more powerful inference models, there is a number of important issues to address in order to ensure that the society takes full profit of the technology at hand. This section describes some of these challenges and discusses recent approaches taken to improve them.

a) Urban audio: Urban environments are designed by humans for humans but their acoustic quality are most of the time neglected. IoS technologies are envisioned to play a role in improving the quality of urban environment by many means [START_REF] Steele | The sensor city initiative: cognitive sensors for soundscape transformations[END_REF]. Passive monitoring to allow better enaction is under research in many major cities in the world [START_REF] Bello | SONYC: A System for Monitoring, Analyzing, and Mitigating Urban Noise Pollution[END_REF], [START_REF] Ardouin | An innovative low-cost sensor for urban sound monitoring[END_REF]. Active solutions where sound displays are played at specific places are also investigated [START_REF] Lavia | Sounding brighton: Practical approaches towards better soundscapes[END_REF]. At the scale of housing, similar potentials are also envisioned.

The urban environment, as designed by humans, should provide security and safety to its citizen. In order to do so, urban planners needs to balance many factors. Noise mitigation and urban soundscape quality improvement are now recognized as important, but citizens and policy-makers currently lack quantitative and qualitative information about it. European regulations request that large cities publicly display a noise emission map to allow citizens to have quantitative information about their exposure to noise. As of today, these maps are built using sound propagation techniques similar to ray-tracing in image synthesis and are only based on some estimates of the number and the speed of emitting sources (e.g., cars, trains, planes) [START_REF] Kephalopoulos | Common noise assessment methods in europe (cnossos-eu)[END_REF]. While those noise maps have their interests, they typically lack time resolution, display only sound pressure levels, and are, by essence, only predictive and not tied to direct measurements. In order to better describe the acoustic environment and its potential impact on the quality of life of the citizens, it is necessary to characterize it in terms of presence of sources.

For those reasons, in several countries around the world, innovative projects like SONYC in New York City [START_REF] Mydlarz | The life of a New York City noise sensor network[END_REF], DYNAMAP in Rome [START_REF] Bellucci | LIFE DYNAMAP project: The case study of Rome[END_REF], SONORUS in Antwerp [START_REF] Botteldooren | Early stage sound planning in urban re-development: The Antwerp case study[END_REF], StadtLärm in Jena [START_REF] Abeßer | A distributed sensor network for monitoring noise level and noise sources in urban environments[END_REF], or CENSE in Lorient [START_REF] Ardouin | An innovative low-cost sensor for urban sound monitoring[END_REF] have deployed acoustic sensor networks with various designs. Measuring the sound environment in urban areas is far from trivial because of the diversity of sound sources but the use of sensors opens a wide range of improvements. First, continuous monitoring allows citizens to interact with regulations administration more efficiently [START_REF] Bello | SONYC: A system for monitoring, analyzing, and mitigating urban noise pollution[END_REF]. Second, noise maps produced by sound propagation techniques and ASNs have different biases. Considering both source of information helps reduce estimation biases in order to produce more reliable estimates [START_REF] Lesieur | Meta-modeling for urban noise mapping[END_REF]. Third, information about sources of interest, vehicle, humans, animals are very useful for prediction high-level attributes of the sound environment such as its pleasantness [START_REF] Gontier | Estimation of the perceived time of presence of sources in urban acoustic environments using deep learning techniques[END_REF].

Despite the potential of acoustic sensor networks for urban sound monitoring, these technologies currently face multiple challenges. In particular, meeting citizen's privacy constraints [START_REF] Gontier | An efficient audio coding scheme for quantitative and qualitative large scale acoustic monitoring using the sensor grid approach[END_REF], designing reliable networks, efficiently adapting source detectors to new monitoring areas without requiring a significant amount of human labor [START_REF] Cartwright | TriCycle: Audio representation learning from sensor network data using selfsupervision[END_REF], [START_REF] Gontier | Polyphonic training set synthesis improves self-supervised urban sound classification[END_REF], all those challenges are now under active research.

b) Ecoacoustics: Along the study of vocalizing animals at an individual level, a field of research called bioacoustics, a new field have recently emerged that is termed ecoacoustics [START_REF] Sueur | Ecoacoustics: the ecological investigation and interpretation of environmental sound[END_REF]. The purpose of ecoacoustics is to gather high-level information about given species living in a ecological niche through the use of passive network of microphones. The quantities of interest here are typically the number of animal calls in order to monitor for example the impact of human activities or climate change on the behavior of a specific species.

Most of those applications require the acquisition of some metrics at an hour rate, often for very long periods of time, ideally up to several years. For example, the study of the impact of light pollution on bird vocalizations [START_REF] Van Doren | High-intensity urban light installation dramatically alters nocturnal bird migration[END_REF] requires a high level of precision in time. Those requirements, along the fact that most ecosystem under study are rather remote with low access to power and wireless networking, brings strong constraints on the design of the network of sensors.

Among the most drastic are the autonomy of the sensors, autonomy meaning here low dependence to energy harvesting and maintenance [START_REF] Lostanlen | Energy efficiency is not enough: Towards a batteryless internet of sounds[END_REF].

In more densily populated areas with cell phone coverage, citizen science approaches such as Merlin BirdId 17 Warblr18 or birdnet 19 allows, based on advanced machine listening techniques [START_REF] Stowell | Automatic large-scale classification of bird sounds is strongly improved by unsupervised feature learning[END_REF], [START_REF] Kahl | Birdnet: A deep learning solution for avian diversity monitoring[END_REF], the citizen to easily geotag bird calls. Indeed, the cell phones are equipped with the necessary hardware needed to record, identify, and send bird species identification reports. While this kind of approach has numerous benefits socially speaking [START_REF] Rovithis | Design recommendations for a collaborative game of bird call recognition based on internet of sound practices[END_REF], it has strong limitations for scientific investigations due to the sampling bias induced by considering only voluntary contributions. As such, remote places such as off-shore underwater areas require specific, reliable dedicated hardware [START_REF] Davis | Long-term passive acoustic recordings track the changing distribution of north atlantic right whales (eubalaena glacialis) from 2004 to 2014[END_REF] to perform long-term studies [START_REF] Baumgartner | Persistent near real-time passive acoustic monitoring for baleen whales from a moored buoy: System description and evaluation[END_REF].

2) Audio Things: Various IoT devices designed to provide or collect sonic information to/from users have been proposed while others are currently under study [START_REF] Lostanlen | Energy efficiency is not enough: Towards a batteryless internet of sounds[END_REF]. An illustrative example is represented by sonic shoes devised for clinical or sport training applications [START_REF] Turchet | Interactive sonification and the iot: the case of smart sonic shoes for clinical applications[END_REF], [START_REF] Iber | Mind the steps: Towards auditory feedback in tele-rehabilitation based on automated gait classification[END_REF]. Another example of Audio Things concern the analysis of environmental sounds. Various embedded systems have been utilized for this purpose, using both real-time and non-real-time architectures. Relevant examples concern environmental monitoring [START_REF] Vandendriessche | Environmental sound recognition on embedded systems: From fpgas to tpus[END_REF], ecoacoustics (e.g., birds monitoring) [START_REF] Sethi | Robust, real-time and autonomous monitoring of ecosystems with an open, low-cost, networked device[END_REF]- [START_REF] Kiarie | Raspberry pi based recording system for acoustic monitoring of bird species[END_REF] and urban sounds [START_REF] Bello | SONYC: A System for Monitoring, Analyzing, and Mitigating Urban Noise Pollution[END_REF]. Smartphones are also increasingly used for similar purposes [START_REF] Kukushkin | Automatic acoustic classification of feline sex[END_REF] along with drones [START_REF] Fu | The chirocopter: A uav for recording sound and video of bats at altitude[END_REF], [START_REF] Zualkernan | An aiot system for bat species classification[END_REF].

In addition, Audio Things with real-time sound analysis capabilities have been employed for remote anomaly detection in manufacturing or machine operation contexts [START_REF] Michailidis | Implementing remote audio as a diagnostics tool for maritime autonomous surface ships[END_REF], [START_REF] Iber | Auditory augmented reality for cyber physical production systems[END_REF]. Moreover, voice-based interfaces such as smart speakers have been utilized to access cloud-based repositories of non-musical sounds [START_REF] Turchet | Voice-based interface for accessible soundscape composition: composing soundscapes by vocally querying online sounds repositories[END_REF].

Hearing aids are another prominent category of Audio Things. Next-generation AI-powered wireless hearings aids will be connected to computing IoS devices and thereby enabling functionalities such as audio streaming from your smart TV, hands-free communications via the smart phone, and real-time translation of the received speech signal into any desired language [START_REF] Hsu | Starkey's AI Transforms Hearing Aids Into Smart Wearables[END_REF]. In addition, hearing aids will be equipped with non-acoustic sensors such as cameras [START_REF] Michelsanti | An overview of deep-learning-based audio-visual speech enhancement and separation[END_REF], EEG sensors [START_REF] Christensen | Toward eeg-assisted hearing aids: Objective threshold estimation based on ear-eeg in subjects with sensorineural hearing loss[END_REF], bone conduction and skin vibration sensors [START_REF] Quatieri | Exploiting nonacoustic sensors for speech encoding[END_REF], etc., in order to obtain enhanced speech quality and intelligibility in noisy environments [START_REF] Shen | Multi-channel wireless hybrid active noise control with fixed-adaptive control selection[END_REF].

C. Relevant works common to both IoMusT and IoAuT

The IoAuT and IoMusT may share some technologies enabling their applications. The following surveys some among the most prominent of them.

1) Embedded systems for sound-based applications: Researchers have proposed a number of operating systems for the IoT (such as RIOT [START_REF] Baccelli | Riot: An open source operating system for low-end embedded devices in the iot[END_REF]). However, these are not specifically conceived bottom-up to handle sound and are insufficient for most IoS applications, which usually have requirements in terms of low-latency of sensor and sound signal processing.

Lately, there has been an increasing attention towards the development of embedded systems dedicated to digital sound processing, where a variety of audio software runs on single board computers such as the Raspberry Pi or the Beaglebone [START_REF] Silva | Evaluation of classical machine learning techniques towards urban sound recognition on embedded systems[END_REF]. This endeavour is usually referred to as "embedded audio". Notable examples, conceived for the makers community, are Axoloti 20 and Prynth [START_REF] Franco | Prynth: A framework for self-contained digital music instruments[END_REF]. According to the results reported in [START_REF] Meneses | A comparison of open-source linux frameworks for an augmented musical instrument implementation[END_REF], the state-of-the-art in this space is represented by Bela [START_REF] Mcpherson | An environment for Submillisecond-Latency audio and sensor processing on BeagleBone black[END_REF]. This platform is based on the BeagleBone Black single-board computer, which is extended with a custom expansion board featuring inputs and outputs for audio, sensors and actuators.

In industrial contexts, the state-of-the-art in embedded audio today is arguably represented by the Elk Audio OS, a Linux-based, open-source operating system developed by the company Elk [START_REF] Turchet | Elk Audio OS: an open source operating system for the Internet of Musical Things[END_REF]. Elk Audio OS is optimized for ultra-lowlatency and high-performance sound and sensor processing on embedded hardware, as well as for handling wireless connectivity to local and remote networks. Differently from other systems, it supports a variety of single board computers. To achieve latencies below 1 ms Elk Audio OS (as well as Bela) uses the Xenomai real-time kernel extension, which according to different studies [START_REF] Vignati | A comparison of real-time linux-based architectures for embedded musical applications[END_REF], [START_REF] Brown | How fast is fast enough? Choosing between Xenomai and Linux for real-time applications[END_REF] is the best-performing of the hard real-time Linux environments.

2) Semantic audio: Semantic Audio is an interdisciplinary field providing techniques to extract meaningful and structured information from audio. The field is situated in the intersection of semantic technologies concerned with helping machines understand data through structured knowledge representations that facilitate automated information processing [START_REF] Pollock | Adaptive Information: Improving Business Through Semantic Interoperability[END_REF], and information extraction from audio through the use of signal processing and ML techniques. The objective of semantic audio is often to facilitate interaction with audio in human terms. This is achieved by providing high-level meaningful control in complex scenarios such as audio production [START_REF] Fazekas | Semantic Web and Semantic Audio Technologies[END_REF].

Semantic Audio methods find application in several IoSrelated scenarios, from intelligent audio and music processing and production [START_REF] Lefford | Context aware intelligent mixing systems[END_REF], [START_REF] Moffat | Approaches in intelligent music production[END_REF]- [START_REF] Fazekas | Intelligent editing of studio recordings with the help of automatic music structure extraction[END_REF], to online music distribution [START_REF] Sandler | Semantic web technology for new experiences throughout the music productionconsumption chain[END_REF], [START_REF] Fazekas | An overview of Semantic Web activities in the OMRAS2 project[END_REF], to auditory scenes classification tasks [START_REF] Stowell | Detection and classification of acoustic scenes and events[END_REF], to ecoacoustics [START_REF] Stowell | Automatic acoustic detection of birds through deep learning: the first bird audio detection challenge[END_REF] as well as speech technology, including speaker, gender or language identification [START_REF] Doukhan | An open-source speaker gender detection framework for monitoring gender equality[END_REF], [START_REF] Draghici | A study on spoken language identification using deep neural networks[END_REF].

An important endeavor in the Semantic Audio field has been that of defining a variety of ontologies to represent knowledge related to the musical and non-musical domains. Regarding musical contexts, examples include the Music Ontology21 (MO) [START_REF] Raimond | The music ontology[END_REF], [START_REF] Raimond | The music ontology specification[END_REF], a high-level ontology for representing the music domain, in particular modelling the music valuechain from production to consumption [START_REF] Fazekas | An overview of Semantic Web activities in the OMRAS2 project[END_REF]. Since MO is arguably the most comprehensive ontology frameworks for the music domain [START_REF] Fazekas | Knowledge representation issues in audiorelated metadata model design[END_REF], several extensions have been proposed for specific sub-domains. These include the Studio Ontology [START_REF]The Studio Ontology Framework[END_REF], a framework comprising of a set of modular ontologies that represent artefacts and technical workflows in music production. The Audio Effect Ontology 22 represents audio effects [START_REF] Wilmering | The audio effects ontology[END_REF] and their applications [START_REF]AUFX-O: Novel Methods for the Representation of Audio Processing Workflows[END_REF] in music production workflows. The Audio Features Ontology 23 is concerned with providing a structures schema for information extracted from audio signals i.e., descriptors representing specific characteristics of sound signals [START_REF] Allik | An ontology for audio features[END_REF] linked to information extraction tools. Broader extensions of these ontologies encompass music theoretical domains [START_REF] Rashid | A music theory ontology[END_REF], ethno-musicology [START_REF] Proutskova | From Music Ontology towards Ethno-Music-Ontology[END_REF], musical archives [START_REF] Page | Realising a layered digital library: Exploration and analysis of the Live Music Archive through Linked Data[END_REF], [START_REF] Wilmering | Bridging fan communities and facilitating access to music archives through semantic audio applications[END_REF], as well as musical instruments [START_REF] Kolozali | Knowledge representation issues in musical instrument ontology design[END_REF], [START_REF] Kolozali | Automatic ontology generation for musical instruments based on audio analysis[END_REF] and mobile devices for music making [START_REF] Thalmann | The mobile audio ontology: Experiencing dynamic music objects on mobile devices[END_REF]. This and the following extensions are particularly relevant in the IoS context. The Internet of Musical Things Ontology 24 represents knowledge related to IoMusT ecosystems, including network connectivity and types of Musical Things [START_REF] Turchet | The internet of musical things ontology[END_REF], while the Smart Musical Instruments Ontology represents knowledge associated to the family of smart musical instruments [START_REF] Turchet | The smart musical instruments ontology[END_REF]. Various applications have been devised which leverage these ontologies at their core (see e.g., [START_REF] Turchet | SMIF: a format for the offline exchange of Smart Musical Instruments configuration and data[END_REF]- [START_REF] Thalmann | The semantic music player: A smart mobile player based on ontological structures and analytical feature metadata[END_REF]).

Concerning the non-musical domain, relevant ontologies include the Audio Set Ontology, which represents knowledge related to general auditory events [START_REF] Gemmeke | Audio set: An ontology and human-labeled dataset for audio events[END_REF]. The Audio Commons Ontology 25 is a high level ontology binding several audio related ontologies together, which was designed to facilitate the integration of audio content repositories on the Web as well as content consumption by software agents [START_REF] Ceriani | Audio commons ontology: a data model for an audio content ecosystem[END_REF]. Ontologies in the broader sound related domains include EBU Core26 [START_REF]EBU Core Metadata Set Specification v1.10[END_REF] and the MPEG-21 media contract ontology [START_REF] Rodriguez-Doncel | Overview of the mpeg-21 media contract ontology[END_REF], while specifically for the sound domain, an early ontology has been proposed in [START_REF] Nakatani | Sound ontology for computational auditory scence analysis[END_REF], with similar structures incorporated into later works [START_REF] Gemmeke | Audio set: An ontology and human-labeled dataset for audio events[END_REF], [START_REF] Ceriani | Audio commons ontology: a data model for an audio content ecosystem[END_REF], [START_REF] Lobanova | Lexical and perceptual grounding of a sound ontology[END_REF]. Ontology-aware models for sound classification have been proposed in [START_REF] Jiménez | Sound event classification using ontology-based neural networks[END_REF], [START_REF] Sun | An ontology-aware framework for audio event classification[END_REF], while an instrument recognition model using hierarchical structures is presented in [START_REF] Garcia | Leveraging hierarchical structures for few-shot musical instrument recognition[END_REF].

3) Sound-based repositories: Several online repositories exist which feature APIs that allow the integration of their content with IoAuT and IoMusT applications. Specially relevant are those that feature content under open licenses that allow their reuse. On the audio side, Freesound [START_REF] Font | Freesound technical demo[END_REF] is the biggest repository with Creative Commons licensed audio. On the music side, Jamendo and the Free Music Archive 27 represent the most important sources containing Creative Commons music tracks. Other repositories with a broader purpose but also incorporating significant amounts of open sound content accesible through an API include the Internet Archive28 and Europeana 29 . Of relevance is also the recent Audio Commons Initiative [START_REF] Font | Audio commons: bringing creative commons audio content to the creative industries[END_REF], which focused on development of technologies and tools to improve re-usability of Creative Commons sound content, including sound and music analyzers and ontologies for a uniform conceptualization of sound repositories.

A slightly different type of sound repositories which are also relevant for IoAuT and IoMusT applications are those featuring static content that can be downloaded and used offline. These include scientific research datasets such as the MTG-Jamendo and FSD50k mentioned above [START_REF] Bogdanov | The mtg-jamendo dataset for automatic music tagging[END_REF], [START_REF] Fonseca | Fsd50k: an open dataset of human-labeled sound events[END_REF], and other collections of reusable sound content available online. While these resources do not allow online interaction such as that offered by online APIs, they are very relevant for research purposes while developing applications in the IoS domain.

For example, the music information retrieval community (see Section III-A4) maintains a broad range of public datasets 30,31 that come with audio. 4) Web-based digital sound applications: One of the most recent and widely adopted among the technologies for musical and non-musical sound applications on the web is represented by the Web Audio API [START_REF] Smus | Web Audio API: Advanced Sound for Games and Interactive Apps[END_REF]. Such a technology enables realtime sound synthesis and processing on web browsers simply by writing JavaScript code. It is a World Wide Web Consortium (W3C) proposed standard 32 and represents a promising basis for the creation of distributed audio applications such as those envisioned in the IoS.

Several projects have demonstrated how sound-based applications can be integrated into the web browser via the Web Audio API. A large amount of these projects have a musical nature (e.g., [START_REF] Roberts | Gibber: Live coding audio in the browser[END_REF]- [START_REF] Lan | Glicol: A graph-oriented live coding language developed with rust, webassembly and audioworklet[END_REF]), even including the implementation of MIR audio analysis techniques in the web browser [START_REF] Correya | Audio and music analysis on the web using essentia. js[END_REF], and a full DAW 33 . Another strand of projects have focused on both musical and non-musical sounds, such as the Freesound Explorer 34 . Conversely, other projects have only focused on non-musical sounds, such as the online real-time sound effects synthesis platform described in [START_REF] Bahadoran | Fxive: A web platform for procedural sound synthesis[END_REF].

In recent years, Web Audio technologies have been employed in embedded systems, thus bridging the realm of audio applications leveraging the web with that of smart objects. For instance, in [START_REF] Matuszewski | Toward a Web of Audio Things[END_REF] the authors describe a system comprising Raspberry Pi platforms, each running a Web Audio application. Such an application could exploit various libraries previously built for mobile based applications (e.g., for synchronization purposes [START_REF] Lambert | Synchronisation for distributed audio rendering over heterogeneous devices, in html5[END_REF]), with the purpose of implementing a distributed architecture for musical performances. Another example of system integrating web-based digital audio technologies and embedded audio was proposed in [START_REF] Skach | Embodied interactions with e-textiles and the internet of sounds for performing arts[END_REF] for body-centric sonic performances. The system consisted of sensor-and actuator-equipped jacket and trousers enabling the interactive manipulation of musical and non-musical sounds retrieved from online sound repositories.

5) Networking: In Section II, we have seen that the realtime communication of audio-related information poses stringent requirements in terms of latency (delay from the transmission to the reception of packets), jitter (sudden variations of the latency) and reliability (probability of successful packet reception) [START_REF] Baratè | Advanced experience of music through 5g technologies[END_REF]- [START_REF] Rinaldi | On the exploitation of 5G multi-access edge computing for spatial audio in cultural heritage applications[END_REF]. Moreover, in some use cases the data rate is low, such as to detect crashes from accidents or simple acoustic signals, whereas in other use cases, the data rates are very high, such as when the acoustic signals carry complex information and need to be quantized at very high sampling rates. Communication protocols such as those used in cellular wireless communications, local area network communications (e.g., Wi-Fi) and internet connectivity have not been traditionally designed or intended for the use cases of internet in Internet of Sounds. However, in the recent year, there have been significant research efforts to design and introduce general purpose protocols that can potentially support these "low-latency and high-reliability applications". These protocols work both at the network core level (Internet connectivity), and at the wireless access level (wireless local area and cellular wireless networks), as we briefly survey below. For a deeper discussion, see [START_REF] Jiang | Low-latency networking: Where latency lurks and how to tame it[END_REF].

At the network core level, the Internet protocol (IP) includes lower layers protocols which greatly determine the delay, jitter, and reliability performance. One of the most prominent set of layers in this regard is Ethernet, and within Ethernet, the Time Sensitive Networking (TSN) [START_REF] Finn | Introduction to time-sensitive networking[END_REF]. TSN is a set of protocols that has been introduced to support applications such as professional audio and video. TSN aims at supporting packet (level 2 frames) loss ratios from 10 -9 to 10 -12 . TSN divides the traffic in two main categories: Class A and Class B. In Class A, the end-to-end latency that can be supported is up to 2ms, but for packets with limited bandwidth and very few hops in the path source-to-destination. In Class B, the end-to-end latency that can be supported is up to 2ms, but for packets with relatively limited bandwidth and up to 7 hops in the path source-to-destination. TSN requests to set up a contract between the source of traffic and the network operator, which ensures zero queueing loss and high packet synchronization. One issue with the adoption of TSN is that it is a service that will have to be required by the packet source when installing or maintaining the Internet service, usually at additional costs on top of the Internet connectivity. When the TSN cannot be used, we can resort to fall-back options, such as using UDP protocols without bothering which lower lever protocol is implemented. However, this fall back options greatly limits the bit rates, and introduces higher packet loss probabilities, and higher jitter compared to TSNs.

At the wireless access level, we have two main technologies to meet the requirements of the Internet of Sounds: wireless local area networks, and cellular networks. For wireless local area networks, we can use TSN to achieve the communication requirements that are suitable for audio services. Here, TSN offers the same pros and cons as in Ethernet for Internet. For the cellular networks technologies, in the recent years there have been major efforts to define the 5G standards. 5G is already being implemented and offers very good communication services for audio applications. This is possible, among others, thanks to three features: at the physical layer, and the network architectural level, and at the network management level:

1) At the physical layer, one of the major novelty is the introduction of the millimeter waves communications [START_REF] Shokri-Ghadikolaei | Millimeter wave cellular networks: A MAC layer perspective[END_REF], which make it possible to transit very high data rates from the audio source at some wireless devices up to the base station and back, with rates of the order of Giga bits per second and latency below 1 ms.

2) At the network architectural level, one major novelty is the concept of edge computing [START_REF] Shi | Edge computing: Vision and challenges[END_REF], where the idea is to place computational resources near the wireless access points (base stations) so that wireless devices that need to perform complex computations can devolve them to near located edge computing nodes (MEC -Multi-Access edge computing), with major delay gains as compared to sending these computations to the cloud. Moreover, the edge nodes can store content near the wireless devices, so that the delay to fetch such content may be limited. 3) Finally, the third major novelty of 5G at the network management level is the concept of network slicing [236], which allows to transmit simultaneously on the physical wireless network several classes of traffic each having quite different requirements in terms of latency and reliability. In practice, network slicing allows to use sets of communication protocols that are tailored to the specific uses cases, such as audio or other real-time services. The future composition of TSN and 5G protocols (when the source is a wireless device that needs to be connected to the Internet) will therefore allow to arguably meet most of the communication requirements of the Internet of Sounds. Recent research in this space has proposed dedicated architectures to interconnect distributed musicians wireless links along with analysis of simulations under different conditions as well as real-world applications of such architectures [START_REF] Centenaro | Towards a 5G Communication Architecture for the Internet of Musical Things[END_REF], [START_REF] Rinaldi | On the exploitation of 5G multi-access edge computing for spatial audio in cultural heritage applications[END_REF], [START_REF] Carôt | The world's first interactive 5G music concert: Professional quality networked music over a commodity network infrastructure[END_REF], [START_REF] Dürre | In-depth latency and reliability analysis of a networked music performance over public 5G infrastructure[END_REF].

6) Synchronization: In several IoS applications the distributed nodes need to be synchronized in time. Nevertheless, the accuracy of such a synchronization shall depend on the application at hand. To keep a satisfactory level of synchronization between the nodes, developers focus shall be devoted to the control of two quantities: 1) the local time of each node and 2) the delay, i.e., the amount of time needed by the node to record, synthesize or playback an audio signal once the request to do so have been received. A complicating factor is that IoS nodes could slightly differ even in presence of the same hardware and software, and even minimal differences in parameters such as sampling rates in the long run can result in clock drifts and therefore cause synchronization issues [START_REF] Wu | Clock synchronization of wireless sensor networks[END_REF]. Moreover, even if different devices in the network would initially share the same clock, they need a re-synchronization procedure from time to time.

In general, quality of service is ensured minimizing the two quantities: σ t , the variance of the difference between the local time of each node, and σ d , the variance of the difference between the delays of each node. To show the importance of such quantities, in the following we describe three usecases with growing requirements in terms of synchronization accuracy:

1) In WASN, synchronization is typically needed to interpret some high-level behaviors happening across different nodes. In this case, σ t and σ d shall remain below the second. 2) On contrary, distributed playback systems that operate over IP [START_REF] Hildebrand | Aes67-2013: Aes standard for audio applications of networks-high-performance streaming audio-over-ip interoperability[END_REF], such as RAVENNA [START_REF] Holzinger | Realtime linear audio distribution over networks: A comparison of layer 2 and 3 solutions using the example of ethernet avb and ravenna[END_REF] or Dante [START_REF] Sheu | Realization of an ethernetbased synchronous audio playback system[END_REF],

reducing σ t below the millisecond is critical as the human auditory system is highly sensitive to phase delays. In this case, σ d is not a strong issue since the nodes are simple playback systems that are not in charge of audio processing or synthesis. 3) Smartphone [START_REF] Arango | The smartphone ensemble. exploring mobile computer mediation in collaborative musical performance[END_REF] orchestras, laptop [START_REF] Trueman | Plork: The princeton laptop orchestra, year 1[END_REF], [START_REF] Wang | Stanford laptop orchestra (slork)[END_REF] orchestras, or any NMP systems involving synchronization [START_REF] Migicovsky | Moveosc -smart watches in mobile music performance[END_REF] represent a much more challenging case. They have the same requirements as distributed playback systems but have to face much more stress on σ d as the nodes of the network have to process and synthesize audio before rendering using a wide diversity of hardware platform. The latter calls for software based solutions [START_REF] Schnell | Of time engines and masters -an api for scheduling and synchronizing the generation and playback of event sequences and media streams for the web audio api[END_REF] which, however, are known to be inherently limited in terms of precision [START_REF] Turchet | Technical performance assessment of the Ableton Link protocol[END_REF]. Different systems and protocols have been developed by the IoT community to minimize σ d as well as to address the issue of establishing an accurate, network-wide notion of time, which is crucial for scenarios demanding precise temporal coordination [START_REF] Li | Global clock synchronization in sensor networks[END_REF], [START_REF] Sivrikaya | Time synchronization in sensor networks: a survey[END_REF]. Various systems have attempted to improve the weakness of the Network Time Protocol, which is widely used in sensor networks [START_REF] Mills | Internet time synchronization: the network time protocol[END_REF]. A particular focus has been placed on minimizing synchronization errors between nodes of wireless sensor networks (see e.g., the Flooding Time-Synchronization Protocol [START_REF] Maróti | The flooding time synchronization protocol[END_REF], PulseSync [START_REF] Lenzen | Optimal clock synchronization in networks[END_REF] and variations of it [START_REF] Yildirim | Time synchronization based on slowflooding in wireless sensor networks[END_REF]).

Synchronization issues are particularly relevant to NMPs occurring in both wide and local area networks and in both wired and wireless networks [START_REF] Rottondi | An overview on networked music performance technologies[END_REF], [START_REF] Gabrielli | Wireless Networked Music Performance[END_REF], [START_REF] Bartlette | Effect of network latency on interactive musical performance[END_REF]. Several NMP systems have been devised for Wireless Local Area Networks (WLANs), typically leveraging Wi-Fi and using protocols to exchange musical messages between devices such as MIDI or Open Sound Control [START_REF] Turchet | The internet of musical things ontology[END_REF], [START_REF] Wright | Open sound control: an enabling technology for musical networking[END_REF]- [START_REF] Wang | The scalability of WiFi for mobile embedded sensor interfaces[END_REF]. Synchronization aspects in WLANs have been addressed by various studies in the literature [START_REF] Turchet | Internet of Musical Things: Vision and Challenges[END_REF], [START_REF] Rottondi | An overview on networked music performance technologies[END_REF], [START_REF] Brandt | Time in distributed real-time systems[END_REF]- [START_REF] Dannenberg | O2: A network protocol for music systems[END_REF]. An example is represented by the approach based on HTML5 proposed in [START_REF] Lambert | Synchronisation for distributed audio rendering over heterogeneous devices, in html5[END_REF] to synchronize mobile based applications leveraging the Web Audio framework. To date, the most widely adopted synchronization protocol for musical applications within Wi-Fi based WLANs is Link, a de-facto standard developed by the company Ableton [START_REF] Goltz | Ableton link-a technology to synchronize music software[END_REF]. However, recent research has assessed the limits of such a protocol in supporting a large number of nodes [START_REF] Turchet | Technical performance assessment of the Ableton Link protocol[END_REF].

Moreover, a significant body of research has investigated at technical and perceptual level the use of metronomes in NMPs distributed over the Internet. For instance, dedicated hardware that broadcast GPS reference time over the network has been proposed in [START_REF] Hupke | Effect of a global metronome on ensemble accuracy in networked music performance[END_REF], [START_REF] Oda | The global metronome: Absolute tempo sync for networked musical performance[END_REF], [START_REF] Hupke | Impact of source panning on a global metronome in rhythmic networked music performance[END_REF], while adaptive metronomes have been proposed in [START_REF] Battello | An adaptive metronome technique for mitigating the impact of latency in networked music performances[END_REF], [START_REF] Battello | Experimenting with adaptive metronomes in networked music performances![END_REF].

IV. CHALLENGES

In this section we discuss open questions that currently hinder the development of the IoS. In particular, we describe the challenges that are common to both the IoMusT and the IoAuT, as well as those that are specific to such fields.

A. Embedded audio

As the IoS emerges, sound-specific operating systems are required on embedded hardware to ease development and portability of IoS applications. Most of current embedded systems specific to sound processing offer a little range of connectivity options and scarce hardware-software methods supporting advanced ML algorithms. In the IoS vision, the connectivity component of embedded systems is crucial to devise advanced applications leveraging edge computing techniques while seamless accounting for privacy and security aspects.

Current challenges in this area include i) platform independence (i.e., the operating system should support a number of hardware platforms as great as possible, to foster software portability); ii) low-latency audio and sensor processing (especially for IoMusT applications); iii) support wireless connectivity options, in particular the latest generation of cellular network; iv) development of methods able to optimize the consumption of resources (e.g., memory, processing, power); v) enabling ML (primarily inference, but also adaptation procedures or re-training) on edge devices by dedicated hardware accelerators.

B. Machine listening

The recent availability of large amounts of audio recordings has fostered research on the use of ML methods to gather both low and high level information about various aspects of musical and non-musical contexts. These endeavors fall within the remits of the so-called field of "Machine Listening".

Concerning musical contexts, the field of MIR investigates computational methods to extract information from musical content [START_REF] Burgoyne | Music information retrieval[END_REF], [START_REF] Widmer | Getting closer to the essence of music: The con espressione manifesto[END_REF]. One of the major branches of this area deals with the analysis of audio signals captured by microphones, from recordings of single musical instruments to recordings of large ensembles. One of the challenges concerning MIR in the context of the IoS paradigm regards the real-time aspect and the use of embedded systems to perform the computations. To date, the majority of MIR research has focused on offline methods analyzing large datasets of audio recordings [START_REF] Alonso-Jiménez | Tensorflow audio models in essentia[END_REF]. The availability of good MIR techniques for real-time contexts, especially on embedded systems, is scarce. This is due not only to the stringent requirement on the processing time to report the wanted information, but also to the fact that pre-and post-processing of the audio signal, typically involved in offline methods, cannot be performed. An area of application of embedded real-time MIR is interactive systems for musical performance [START_REF] Fiebrink | The wekinator: a system for real-time, interactive machine learning in music[END_REF] such as smart musical instruments [START_REF] Turchet | Smart Musical Instruments: vision, design principles, and future directions[END_REF], where the extracted information is repurposed, with unperceivable latency for the human player, into digital sounds. Current real-time MIR methods and related datasets are severely affected by temporal inaccuracies. Therefore, novel temporally accurate datasets are needed to advance the state of the art in this space, along with innovative information retrieval algorithms conceived for real-time usage.

Along the same lines, the development of ML approaches to predict the future evolution of an audio signal in realtime, based on past audio samples, would find application in NMP scenarios as an alternative approach for packet loss concealment [START_REF] Verma | A deep learning approach for low-latency packet loss concealment of audio signals in networked music performance applications[END_REF]. Nowadays, recovery mechanisms included in audio codec implementations are adopted when a packet carrying audio data gets lost or is received too late to be reproduced in due time, but they introduce additional processing delays which add up to the overall perceived mouthto-ear latency. Moreover, in a long term and more "visionary" scenario, ML algorithms could even become capable of learning to anticipate what the musician will play ahead of time, thus being able to reproduce a predicted version of the audio stream before the real audio data generated by a remote performer is received. Such advanced artificial intelligence capabilities would definitively eliminate any delay-induced usability limitation.

Regarding non-musical contexts, as described in Section III-B1 several efforts have been conducted to extract meaning from soundscapes (i.e., sonic environments), particularly in urban areas [START_REF] Silva | Evaluation of classical machine learning techniques towards urban sound recognition on embedded systems[END_REF] and wildlife monitoring. While those two application areas are now well established, one recent and promising field of application for ASNs is the surveillance of industrial processes for security and quality control. The main scientific challenge here is that, due to the complexity of the production chain, all possible abnormal behaviors can hardly be modeled. Alternatively, one has to numerically define what is a normal behavior and raise an issue when the recorded behavior significantly departs from it. Quantifying and thresholding this level of difference between the observed data the model of normality the system have in memory is, at this time, a very open research question.

New tasks have been proposed in the DCASE challenge to foster research on this topic [START_REF] Kawaguchi | Description and discussion on DCASE 2021 challenge task 2: Unsupervised anomalous sound detection for machine condition monitoring under domain shifted conditions[END_REF] and the body of knowledge is gaining momentum, as demonstrated by a systematic research study done by considering 7 journals and 16 conferences relevant to the field [START_REF] Nunes | Anomalous sound detection with machine learning: A systematic review[END_REF]. Within those publication venues, the study shows that the number of publications on this topic is approximately doubling each year since 2015.

C. Semantic Audio

With the proliferation of "smart" devices featuring advanced information interaction capabilities and access to large amounts of data, there is increasing demand for such features in the context of IoS. This calls for the use of state-of-theart audio and music analysis and processing techniques in IoS devices provided by Semantic Audio and MIR (see Sections III-A4 and III-C2). These fields, however, are increasingly dominated by deep learning solutions [START_REF] Choi | A tutorial on deep learning for music information retrieval[END_REF], [START_REF] Deng | Deep learning: methods and applications[END_REF], [START_REF] Kim | One deep music representation to rule them all? a comparative analysis of different representation learning strategies[END_REF].

Although the available computing power is fast increasing in IoS and edge computing devices, with more and more innovative AI accelerators proposed [START_REF] Feldmann | Parallel convolutional processing using an integrated photonic tensor core[END_REF], decoding large neural networks still presents a problem in certain applications, particularly where computational, bandwidth or energy resources are limited, or where hard real-time criteria need to be observed, e.g., for efficient interaction between human and machine [START_REF] Turchet | Hard real time onset detection for percussive sounds[END_REF]. There are currently no clear guidelines and established methodologies for optimizing applications given specific technical criteria and user requirements. For example, in [START_REF] Turchet | Cloud-smart musical instrument interactions: Querying a large music collection with a smart guitar[END_REF] edge and cloud computing solutions for semantic audio analysis are painstakingly compared with traditional keyword based search to determine the trade-off between these techniques with respect to user requirements. Similar experiments would be needed to establish the optimal solution for most applications involving MIR or Semantic Audio techniques in IoS applications.

Test-beds providing diverse annotated data and platforms to easily test algorithms in different application contexts would help breaking the barriers in applying advanced audio analysis and processing techniques in IoS. Advanced methodologies for optimizing neural networks for specific hardware would also be beneficial to overcome these barriers. Particularly in the context of IoAuT, model compression [START_REF] Han | Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding[END_REF] and quantization [START_REF] Krishnamoorthi | Quantizing deep convolutional networks for efficient inference: A whitepaper[END_REF] have been noted as a promising way forward in [START_REF] Turchet | The internet of audio things: state-of-the-art, vision, and challenges[END_REF]. Further challenges in edge computing in IoS are discussed in Section IV-D.

Data interoperability is another substantial barrier in creating complex IoS applications. The ontologies discussed in Section III-C2 provide a mechanism for shared conceptualizations that alleviate the problem of matching the "meaning" of data and metadata items in complex networked applications with multiple stakeholders and solution providers. There are ontologies ripe for use in the audio and music domains, and their utilities have been demonstrated across diverse IoS relevant applications [START_REF] Sandler | Semantic web technology for new experiences throughout the music productionconsumption chain[END_REF], [START_REF] Fazekas | An overview of Semantic Web activities in the OMRAS2 project[END_REF], [START_REF] Wilmering | Bridging fan communities and facilitating access to music archives through semantic audio applications[END_REF], [START_REF] Turchet | SMIF: a format for the offline exchange of Smart Musical Instruments configuration and data[END_REF]- [START_REF] Thalmann | The semantic music player: A smart mobile player based on ontological structures and analytical feature metadata[END_REF], [START_REF] Stables | Safe: A system for extraction and retrieval of semantic audio descriptors[END_REF]. However, it can be observed that no ontology is optimal for any given application, since trade-offs are necessary for viable shared conceptualisation [START_REF] Fazekas | Knowledge representation issues in audiorelated metadata model design[END_REF]. As a solution to this problem, the development of flexible ontology frameworks that allow solution-specific components to be plugged into higher-level models that include only foundational classes and relations is proposed in [START_REF] Fazekas | Knowledge representation issues in audiorelated metadata model design[END_REF], while a layered approach to information modelling is proposed in [START_REF] Page | Realising a layered digital library: Exploration and analysis of the Live Music Archive through Linked Data[END_REF] in the music and audio domains.

A further related challenge is the efficiency of ontologybased solutions. This can be divided into efficiency challenges related to data exchange, and challenges related to storage, querying and inference. In the context of data exchange, i.e., the syntax used for representing semantically enhanced information, verbose XML-based formats such as RDF/XML 35 have been criticized, with Turtle 36 and later JSON-LD 37 emerging as broadly supported solutions. In the context of IoS, a further consideration is the optimal granularity at which semantics should be embedded in an application. While binary and very concise exchange formats such as Open Sound Control [START_REF] Freed | Features and future of open sound control version 1.1 for NIME[END_REF] with weak underlying semantic models are popular and necessary in some IoS contexts, data efficient semantic information exchange formats are also emerging. This includes binary serialization formats for semantically enhanced data such as Header Dictionary Triples (HDT/RDF 38 ) [START_REF] Martínez-Prieto | Exchange and consumption of huge rdf data[END_REF].

Regarding querying and complex machine-to-machine communication, the Constrained Application Protocol [START_REF] Shelby | The constrained application protocol (CoAP)[END_REF] have been proposed as a lightweight IoT protocol using semantic technologies [START_REF] Roffia | Dynamic Linked Data: A SPARQL Event Processing Architecture[END_REF]. This has been demonstrated to cope with efficiency constraints in a musical context in [START_REF] Viola | C Minor: a Semantic Publish/Subscribe Broker for the Internet of Musical Things[END_REF] and [START_REF] Turchet | Semantic Web of Musical Things: achieving interoperability in the Internet of Musical Things[END_REF].

Inference remains an ongoing challenge since reasoning with very large knowledge bases is known to be computation- 35 http://www.w3.org/TR/rdf-syntax-grammar/ 36 https://www.w3.org/TR/turtle/ 37 https://json-ld.org/ 38 https://www.rdfhdt.org/ ally expensive. Solutions to this problem include restricting the expressivity of knowledge representation, e.g., through the use of a specific profile of an ontology language such as OWL EL 39 which allows reasoning in polynomial time. Deep learning-based solutions for reasoning with ontologies are also emerging [START_REF] Liu | Deep learning-based reasoning with multi-ontology for IoT applications[END_REF]. The granularity at which semantic representation is used is another important consideration in the IoS. For example, using an ontology to annotate large blobs of homogeneous data, instead of individual data items, may be a good trade-off in some IoS applications. Further semantic audio related challenges in the context of interoperability and storage are discussed in Sections IV-I and IV-H.

D. Audio detection on the edge

Edge computing is a distributed computing paradigm that brings computation and data storage closer to the sources of data. In ASNs, there are numerous advantages in concentrating most of the processing on the sensors. Computational inference of quantities of interest on the edge reduces the bandwidth required to transmit the data, and most of the time reduces privacy issues. The latter arises because a greater level of control can be performed since the data transferred is usually more compact.

To do so, the sensor has to embed computational components which must be carefully chosen to wisely balance energy consumption and processing capabilities. While sensing and features computation are relatively low consumption processes, powerful state of the art machine listening detectors based on deep learning models have to be simplified to fit memory and energy consumption requirements [START_REF] Cerutti | Compact recurrent neural networks for acoustic event detection on low-energy lowcomplexity platforms[END_REF]. Even with such compact embedded software systems, sensors are heavily dependent on a reliable and continuous source of energy. That means that the sensor has to be wired or has to carry powerful batteries to overcome the non avoidable down times of energy harvesting components like solar panels.

Resorting to embedded power storage reduces the ease of maintenance and increases the dependency on polluting materials. Those issues call for innovative solutions such as batteryless designs [START_REF] Lostanlen | Energy efficiency is not enough: Towards a batteryless internet of sounds[END_REF]. While this kind of hardware potentially allows simpler and more reliable networks designs, it also opens an interesting challenges in sparse network managment to be able to produce reliable estimates of quantities of interest from sources of information that are only intermittently able to record, process and transmit.

E. Networking architectures

To date, actual IoS deployments over 5G networks remain scarce, and very limited statistical results are available on the actual latency and reliability of 5G networks for IoS scenarios [START_REF] Centenaro | Towards a 5G Communication Architecture for the Internet of Musical Things[END_REF], [START_REF] Dürre | In-depth latency and reliability analysis of a networked music performance over public 5G infrastructure[END_REF]. There is the need to design, implement and evaluate novel networking architectures exploiting the new capabilities offered by 5G to support IoS services and applications. Research should address fundamental questions such as how to design 5G slices dedicated to low-latency and highreliable transmissions, which computations can be offloaded Furthermore, IoS research could address emerging computing paradigms such as in-network computing. To date only a handful of IoS studies have investigated such avenue [START_REF] Wu | Accelerating industrial iot acoustic data separation with in-network computing[END_REF].

F. Networked music performances

For what concerns NMP solutions, the adoption of 5G wireless transmission technologies would ensure access delay to the backbone telecommunication infrastructure below 10 ms. Such figure is compliant to the typical latency requirements of NMP while fostering flexibility and portability of NMP setups, which are nowadays constrained by the need of leveraging cabled Ethernet connections for local area network connectivity (note that Wi-Fi wireless connections are typically avoided for NMP purposes, as they introduce too high jitter). As further step, involving Internet service providers and network operators in the development of commercial NMP services would allow the adoption of prioritization criteria for NMPgenerated audio/video data (e.g., in the context of Software Defined Networking design approaches), thus providing to the users adequate Quality of Service guarantees in terms of latency and jitter.

Video conferencing has advanced dramatically with the introduction of ultra-reliable low-latency communication (UR-LLC) in 5G and Nvidia's radical new idea, where only key points in the video are transmitted and then Deep Neural Networks (DNNs) reconstruct artificial video representations, which have very high similarity with the original video [283]. The challenge would be to exploit similar tricks for the audio content, where key points in the acoustic scenes are transmitted, and then DNNs would be constructing artificial representations of the significant parts of the original acoustic scenes.

A challenge is to change the current paradigm where users interact with their Sound Things via individual interfaces and networks, into a more sound service oriented and network driven paradigm, where any sound technology can be seamlessly accessed via a common IoS software layer. In principle, users only see the sound services and not the underlying Sound Things that automatically organize themselves, analyze the acoustic scenes, and exchange information with each other.

G. Distributed Machine Learning over Networks

Machine learning is a research area that is increasingly gaining attention to design, optimize and manage sound related systems. Although ML is well established within computer vision and speech and text analysis, ML faces new major challenges when it comes sounds and internet. In a IoS ecosystem, all the involved units generate distributed, heterogeneous and imbalanced data that cannot be easily collected timely due to the bandwidth of the communication protocols, the economic unsustainability to deploy high bandwidth communications, and privacy concerns.

Unfortunately, to achieve the highly desirable ML-based targets for IoS services, we cannot simply apply existing ML.

The prominent successes of ML are largely in the domains of images, speech, where the availability of large datasets and popular platforms that can provide vast amounts of dedicated computational and communication resources in centralized setups such as data centers. Such assumptions challenge the distributed, networked, and real-time nature of the Internet of Sounds [START_REF] Li | Learning iot in edge: Deep learning for the internet of things with edge computing[END_REF]. On the one side, ML methods and algorithms are not yet mature for being used in the Internet of Sounds domain. On the other side, it is not clear up to which extent the ML predictions can be realistically used in a Internet of Sounds data communication architecture. Specifically, in the Internet of Sounds, we face two prominent challenges: unbalanced data sets and distributed data sets, as we survey below:

• The first challenge for ML-based IoS systems is that the predictions have to be done at locations or times from which noisy or sparse/partial measurements are available. ML achieves impressive performance mostly on the data that contains no missing values and have balanced classes. In IoS datasets (for both time-series data or statistical data), missing data commonly exists due to the distributed devices connected over communication networks. Internet of Sound data suffer from a number of unpredictable impairments since the hardware and software of the involved units will be produced by different vendors with different standards and different costs, memory and computation [START_REF] López | An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics[END_REF]. Datasets may also be imbalanced, which may increase the risk of having the so-called bias in the training pipeline. Despite recent advancements in connectivity solutions for IoT, the existing methods cannot properly address how to handle heterogeneous types of data. A recent survey [START_REF] Huang | Deep prototypical networks for imbalanced time series classification under data scarcity[END_REF] showed that deep learning methods achieve good performance, among other alternative approaches, for most time-series classification tasks with missing data. Unfortunately, the existing methods are usually limited to non-sparse and balanced datasets, which we may rarely see in cyber-physical systems such as the Internet of Sound. • The second challenge for ML in Internet of Sound is that the datasets are distributed and must be connected over a public communication network, including via Edge computing [START_REF] Li | Learning iot in edge: Deep learning for the internet of things with edge computing[END_REF], [START_REF] Park | Wireless network intelligence at the edge[END_REF]. In these solutions, ML services will have to be solved by distributed algorithms, where the heavy coordination and computation procedures of ML are much hindered by bandwidth limitations, latency and message loss. The computational capability and storage of the Internet of Sound devices challenges the use of heavy pre-trained models. The communication networks enforce inference with partial knowledge. The challenge here is to support self adaptive ML based applications composed of edge and cloud modules. A promising direction to address these issue is the design of novel wireless and wired communication protocols tailored for distributed ML [START_REF] Hellström | Wireless for machine learning[END_REF]. devices to content stored in cloud repositories that can provide satisfactory user experiences when working with audio in realtime and consuming audio from the cloud, is one of the basic challenges that can be addressed with the adoption of faster network technologies such as 5G. Moreover, interoperability between different cloud content providers is also an open challenge. In a rich IoS ecosystem, different processes running in different devices and under different platforms should be able to interact with storage solutions using common protocols and APIs.

In addition to that, the traceability and authentication of audio content distributed in the IoS ecosystem is a substantial challenge that needs to be addressed for successful deployments of IoS applications. To that end, blockchainbased technologies have already been envisioned to verify the integrity of original audio content and to enable its secure distribution, as well as have been implemented as a proofof-concept within the paradigm of IoAuT [START_REF]Storage and authentication of audio footage for IoAuT devices using distributed ledger technology[END_REF]. However, challenges remain open, including the traceability of audio content usage in the IoS ecosystem.

I. Interoperability and standardization

The result of our survey of the IoS field reveals that, to date, active research on IoS-related themes is rather fragmented, typically focusing on individual technologies or single application domains in isolation. Ad-hoc solutions exist that are well-developed and substantial. However, their adoption remains low due to the issues of fragmentation and weak interoperability between existing systems. Such a fragmentation is potentially detrimental for the development and successful adoption of IoS technologies, a recurring issue within the more general IoT field [START_REF] Borgia | The Internet of Things vision: Key features, applications and open issues[END_REF].

Within the IoS, different types of musical and non-musical devices targeting a variety of stakeholders are utilized to generate, detect, or analyze sonic content. Those devices need to be able to dynamically discover and spontaneously interact with heterogeneous computing, physical resources, as well as digital data. Challenges related to their interconnection include the need for ad-hoc protocols and interchange formats for sound-related information that have to be common to the different Sound Things, as well as the definition of common APIs specifically designed for IoS applications. Semantic technologies, such as Semantic Web [START_REF] Berners-Lee | The semantic web[END_REF] and knowledge representation [START_REF] Sowa | Knowledge representation: logical, philosophical, and computational foundations[END_REF] have arguably the potential to become a viable solution to enable interoperability across heterogeneous Sound Things. However, to date, only a few ontologies exist for the representation of the knowledge related to IoS ecosystems.

The Moving Picture Experts Group was a working group of ISO/IEC and has since 1988 been instrumental in driving the development and standardization of audio compression, synchronization, processing, transmission, and file formats for a wealth of applications [292]. Recently new organizations such as the Moving Picture, Audio and Data Coding by Artificial Intelligence has been formed in order to further push the technology towards artificial intelligence data coded standards [START_REF]Picture, audio and data coding by artificial intelligence (MPAI)[END_REF]. Standards targeting the combination of communications and audio for personal area networks include Bluetooth LE [START_REF]Bluetooth LE[END_REF], [START_REF] Cao | Enabling technologies for wireless body area networks: A survey and outlook[END_REF]. A challenge would be to further develop the standardizations and interoperability within the intersection of communications and audio beyond personal area networks.

A common operating system for Sound Things can be considered as a starting point for achieving interoperability. As surveyed in Section III-C1, recent technological advances in the embedded audio field have led to the creation of platforms that are suitable for IoS applications. Their wide adoption and deployment is therefore expected to enable advanced IoS ecosystems and has the potential to foster interoperability within and even between the musical and non-musical sides of the IoS.

J. Quality of Service

The desire to distribute audio and audio-related data across IoS devices imposes application specific service requirements of the IoS architecture.

In general, the reliability of the data delivery is defined under the umbrella term Quality of Service (QoS). For telecommunication application, the telecommunication experts of the International Telecommunication Union defines QoS as the "[t]otality of characteristics of a telecommunications service that bear on its ability to satisfy stated and implied needs of the user of the service" [START_REF]ITU-T, E.800 -Definitions of terms related to quality of service[END_REF]. This definition also holds for the IoS in which users could be humans or machines, e.g., in form of sound creators or listeners. For instance, for a telematic musical event with co-located musicians and audiences, the need to exchange audio signals or parametric control data over the network without perceivable artifacts is crucial. Perceivable artifacts can be caused by packet loss, transmission delay, and limited bandwidth. Packet loss may occur when utilizing an unreliable data transport protocol, like the User Datagram Protocol (UDP), or when the audio stream via a reliable transport mechanism, like the Transmission Control Protocol (TCP), gets interrupted. Without proper mitigation or error concealment, audible artifacts such as spectral distortion, drop-outs, pops, or crackles reduce the audio quality and consequently affect the listening experience. An end-to-end low-latency transmission chain enables remote musicians to perform as if they were in close proximity. It has been shown that a round trip delay as low as 40 ms already starts affecting the ability to perform together (see also Section III-A1) and a latency of above 150 ms will degrade a communication scenarios [START_REF]1305 -Effect of delays on telemeeting quality[END_REF]. Especially in wireless networks, transmission latency is often time-variant which needs to be compensated by additional audio buffers. This adds delay for real-time audio applications. The bandwidth limitations of many IoS network protocols such as Bluetooth, or Wi-Fi demand perceptual audio signal compression. The processing latency caused by the signal encoding and signal decoding contributes to the overall transmission latency. Low-latency audio codecs have a coding delay as low as 5 ms [START_REF] Lutzky | MPEG-4 AAC-ELD v2-the new state of the art in high quality communication audio coding[END_REF], [START_REF] Schnell | LC3 and LC3plus: The new audio transmission standards for wireless communication[END_REF]. Generally, parametric control data (e.g., MIDI, Open Sound Control) require relatively little bandwidth compared to audio data. When transmitted together, the QoS has to ensure that control data and audio data are kept synchronized across the IoS network. Many IoS devices are hardware-constrained and have limited capabilities to connect. Further, mobile IoS devices need to be resourceful with their limited battery power which may lead to throttled data processing and transmission. At the same time, network conditions may change over time and can become unstable. Supporting the high QoS requirements across time-varying network conditions and across devices that may have different processing and network capabilities poses a cross-disciplinary challenge for the IoS.

K. Ethics, privacy, security and sustainability

Notwithstanding the numerous societal, economic, and artistic benefits that the IoS promises, the ubiquitous nature and increased autonomy of Sound Things raise concerns about the ethical compliance of the associated services. We need to incorporate ethics in the IoS so that the services provided do not infringe on the ethical rights of its beneficiaries.

Nevertheless, the study of ethics in music and audio technology is not yet well established, with only a few authors dealing in recent years with the importance of related ethical dilemmas. Researchers in the musical domain have questioned the practices of music streaming services in monitoring users and inducing behaviours, warning about the associated risks [START_REF] Drott | Music as a technology of surveillance[END_REF], [START_REF] Morreale | my library has just been obliterated": Producing new norms of use via software update[END_REF]. Other authors have examined the ethical dimensions of the field of MIR, arguing that such technology is not valueneutral but is influenced by design choices, and so has ethically relevant implications [START_REF] Holzapfel | Ethical dimensions of music information retrieval technology[END_REF]. In the field of New Interfaces for Musical Expression, some authors have discussed political issues inherent in new musical instruments [START_REF] Morreale | A nime of the times: Developing an outward-looking political agenda for this community[END_REF]. Similar statements can be made for the audio-technology domain.

All these research strands are also relevant to the IoS field, although to the authors' best knowledge no investigations have been conducted yet on such topics at the scale of IoS. Ethical research in IoT has identified major issues such as privacy, security, transparency, trust, social equity, social equality, and responsibility according to law. Research has also identified the factors that can increase the acceptance of IoT and proposed a set of guidelines to interact with IoT from a social perspective [START_REF] Baldini | Ethical design in the internet of things[END_REF]. There is a strong need to follow some policies that support social issues also in the IoS in order for it to be socially acceptable and undertaken in the public interest. Nevertheless, the IoS is distinguished from the general IoT field for its sound-specific focus. While some technical and societal challenges are common to the two fields, IoS poses music-and audio-specific challenges that cannot be simply addressed by using the same methods and tools of IoT.

Sound Things have the ability to automatically collect, analyze, and exchange personal data related to their users, and consequently, they can expose users to breaches of security and privacy, which are common to other IoT subfields [START_REF] Bäckström | Speech coding, speech interfaces and iot-opportunities and challenges[END_REF], [START_REF] Weber | Internet of things: Privacy issues revisited[END_REF]. Research in this domain is already emerging, in particular in the area of speech, where recent works have addressed the interpretation of legal frameworks [START_REF] Nautsch | The GDPR & Speech Data: Reflections of Legal and Technology Communities, First Steps Towards a Common Understanding[END_REF], the challenge of quantifying privacy [START_REF] Nautsch | The Privacy ZEBRA: Zero Evidence Biometric Recognition Assessment[END_REF] and by organizing challenges, such as VoicePrivacy [START_REF] Tomashenko | Introducing the VoicePrivacy Initiative[END_REF], to research and develop solutions to particular problems.

It is important to consider that users have an intuition and can be assumed to be competent to gauge their level of privacy and security only when it relates to human capabilities [START_REF] Brüggemeier | Perceptions and reactions to conversational privacy initiated by a conversational user interface[END_REF], [START_REF] Bäckström | Intuitive privacy from acoustic reach: A case for networked voice user-interfaces[END_REF]. However, where computer systems go beyond human, to super-human capabilities, users generally cannot be expected to have an intuition with respect to their security and privacy when they interact with the system. As an analog, it is important that passengers of an airplane feel safe (i.e. that they have an intuition of safety), but it is more important that they actually are safe, though we cannot reasonably expect passengers to be competent in evaluating whether they are safe or not. Similarly, it is thus important that regulations and guidelines are developed to govern the security and privacy of IoT. Such guidelines are already available or in development for voice interfaces, e.g. [START_REF]Guidelines 02/2021 on virtual voice assistants[END_REF], [START_REF]Privacy guidelines and capabilities unique to voice[END_REF].

The range of problems related to, and approaches for preserving privacy in IoS and IoT is vast. For example, data intentionally shared can unintentionally leak related information or, as a form of function creep, data can be used in unexpected ways. Moreover, users' ignorance can lead to breaches, they can be tricked or coerced to share information, or their hardware or service providers can be hacked [START_REF] Bäckström | Privacy in speech interfaces[END_REF]. As a general policy, it is therefore important to develop products and services using the privacy-by-design approach, where the design and development process includes, from the beginning, impact assessment of security and privacy.

Application of privacy and security standards thus has to be included in all parts of systems design, including the acoustic environment, acoustic and hardware design of device, edge software, communication links, device interaction, cloud services as well as user interface and service design. One approach to ensure the privacy of information consists of the definition of privacy policies. Sound Things could be equipped with machine-readable privacy policies, so that when they come into contact they can negotiate privacy policies for before communicating [START_REF] Roman | Securing the Internet of things[END_REF], [START_REF] Zarazaga | Acoustic fingerprints for access management in ad-hoc sensor networks[END_REF].

Taking advantage of distributed computational resources in an IoS context seems to bear the risk of giving away control over shared data. For instance, transmitting audio signals to a cloud service for acoustic signal enhancement exposes the audio signals to the cloud provider and to the algorithm developer and could potentially enable unwanted data usage. There are well-established data encryption methods for protecting data from unwanted access while stored and while transmitted over the internet (e.g., HTTPS, MQTT). However, protecting data during the actual processing remains challenging. Ensuring privacy during data processing is the field of secure signal processing (SSP) and privacy preserving ML respectively. SSP allows mathematical operations to be executed on encrypted data without the need for prior decryption. The foundation of SSP is homomorphic encryption, which is a special class of data encryption methods. Here, one or more specific mathematical operations performed on the encrypted data corresponds to mathematical operation performed on the original data. The result of the operation remains encrypted and cannot be decrypted without proper credentials.

In the audio context, homomorphic encryption has been used for audio analytics (e.g., privacy-preserving speaker verification and identification [START_REF] Pathak | Privacy-preserving speaker verification and identification using gaussian mixture models[END_REF]) as well as signal processing tasks (e.g., convolution [START_REF] Yakubu | Secure audio reverberation over cloud[END_REF]). Besides the audio input data, it has been shown that even the actual algorithm can remain encrypted, which protects both the data provider and the algorithm developer from unwanted information sharing with the platform provider. Homomorphic encryption is also applied to ML methods, such as logistic regression [START_REF] Crockett | A Low-Depth Homomorphic Circuit for Logistic Regression Model Training[END_REF], XGBoost [START_REF] Meng | Privacy-Preserving XGBoost Inference[END_REF], or even DNNs [START_REF] Qaisarahmadalbadawi | Towards the alexnet moment for homomorphic encryption: Hcnn, the first homomorphic cnn on encrypted data with gpus[END_REF]. The main challenge of secure signal processing using homomorphic encryption is the computational complexity. For instance, the inference on the privacy preserving ML model in [START_REF] Meng | Privacy-Preserving XGBoost Inference[END_REF] is 400 times slower than its conventional version.

In a different vein, it is paramount to be aware of the adverse impact of current IoS technology on the environment in terms of greenhouse gases emissions, pollution and soil consumption. To date, research on sustainability aspects of the IoS is limited. The study reported in [START_REF] Gabrielli | Towards a sustainable Internet of Sounds[END_REF] has recently provided a survey of the environmental issues produced by current information and communication technology and related these to the use cases that the IoS envisions. On the basis of this survey, the authors identified some key aspects to reduce the footprint of IoS services and products and then provided suggestions to make advancements in IoS environment-aware.

L. Blockchain technologies

Blockchain is an emerging technology that is impacting several industries, including the creative industries and those operating in the IoT [START_REF] Fernández-Caramés | A review on the use of blockchain for the internet of things[END_REF]- [START_REF] Uddin | A survey on the adoption of blockchain in iot: Challenges and solutions[END_REF]. The IoS vision requires, above all, IoT features such as decentralization, seamless authentication, transparency, data integrity privacy, and self-maintenance, as well as the musical domain's feature such as efficient handling of copyrights and speed of royalties payment. Such features can be brought by blockchain, but its integration in the IoS has not been investigated thus far.

Recently the integration of IoMusT and blockchain technology has been proposed in [START_REF] Turchet | Blockchain-based Internet of Musical Things[END_REF], where several examples of use cases have been provided. For instance, in the "cover song identification" scenario, authors envisioned a Sound Thing, equipped with microphones and machine listening algorithms dedicated to the identification of the cover of songs, which is used by an inspector of a National Rights Society at a music venue during a live concert. The Sound Thing automatically verifies whether the played songs match the titles declared by musicians in the list of music pieced to be sent to the National Rights Society. As soon as the match is verified the composers of those music pieces are immediately rewarded thanks to the use of a smart contract deployed on the blockchain underlying the system.

V. IOS: A RESEARCH AGENDA

The above sections have shed some light on the current challenges that prevent the IoS field to flourish. In this section, we collect together the different aspects and open questions that need to be answered in order to create, design, use and finally evaluate IoS systems. This may be seen as a roadmap that we hope would be addressed in the ongoing research of the emerging IoS community.

1) To progress the design of embedded platforms specific for the IoS, including the integration of methods for low-latency processing, and the support to the latest generation of cellular networks; 2) To progress the design of reliable, autonomous sensing devices to better monitor and understand our environment through the audio modality; 3) To progress the design of Sound Things, with new solutions for the analysis of sound-related information based on the edge computing paradigm and the most advanced machine listening approaches; 4) To advance the current connectivity infrastructure, with the implementation of novel interoperable protocols for the exchange of sound-related information; 5) To define standards (e.g., for protocols, shared ontologies, formats, APIs) that will allow one to reduce fragmentation and facilitate interoperability among Sound Things as well as the services they offer including audio processing and storage; such endeavors could entail the creation of dedicated Web of Sound Things architectures; 6) To create advanced IoS ecosystems, in both the musical and audio domain, which include enabling technologies and communities of users interacting with them; 7) To investigate ethical concerns and define appropriate measures to address them; this includes the definition of principles for an Ethical IoS that can inform design, development, and evaluation of IoS ecosystems, their hardware and software components, and the interactions of stakeholders; moreover, this entails tackling the challenges of i) privacy and security of personal data, with a "privacy by design" approach; ii) sustainability aspects at all levels, from production to distribution; iii) inclusiveness and accessibility, conceiving systems for various categories of users; 8) To explore the integration of haptic feedback and motion tracking mechanisms in those IoS scenarios where gestural data play a fundamental role (e.g., in the context of remote music teaching) or when enhanced immersion in artificial soundscapes is required; 9) To devise methods capable of minimizing the need for the user to configure the Sound Things: the Sound Things should self-configure and automatically adapt to changing environments and use cases. 10) To ease the adoption by the industry of IoS related technologies by providing end-to-end systems that are easy to deploy and to adapt to new needs; 11) To investigate novel network communication protocols for the IoS (e.g., TSN over 5G) that will be compatible with the network slicing concept introduced in 5G; 12) To establish distributed machine learning tasks (training and inference) that are robust to the data missing and data imbalance problems and that can run over optimized or ad-hoc communication protocols so that the computations will be reliable and with limited latencies; 13) To develop cognitive processing methods to fuse acoustic signals from ad-hoc microphone arrays into a single high-quality audio stream; 14) To design processing frameworks that allows dynamic distribution of audio processes within the network with service guarantees e.g., to take advantage of both lowlatency edge computing and scalable cloud processing; 15) To devise new temporally accurate datasets and methods for the real-time analysis and classification of musical and non-musical sonic content on embedded systems; 16) To integrate the blockchain into the IoS, and create effective and efficient applications based on such integration such as secure audio distribution and traceability.

VI. CONCLUSION In this paper we introduced the paradigm of the Internet of Sounds, highlighting its unique characteristics that differentiate it from to the IoT, and identifying the major challenges and requirements that need to be addressed for reaching full potential. We introduced a definition for such a new subfield of the IoT, and placed it in the context of neighbouring fields. In the IoS paradigm, which merges under a unique umbrella the emerging fields of the Internet of Musical Things and the Internet of Audio Things, heterogeneous devices dedicated to musical and non-musical tasks can interact and cooperate with one another and with other things connected to the Internet to facilitate sound-based services and applications that are globally available to the users. We presented a vision for this emerging research field, which is rooted in different lines of existing research including sound and music computing, Internet of Things, machine listening, semantic audio, artificial intelligence, and human-computer interaction. The IoS relates to wireless networks of intelligent devices dedicated to musical and non-musical purposes, which enable, in both co-located and remote scenarios, various forms of interconnection among different stakeholders.

The IoS vision not only offers several unprecedented opportunities, but also brings technological and non-technological challenges that both academic and industrial research will need to address in upcoming years. The realization of the proposed IoS vision would ultimately benefit society, by providing several novel possibilities, which include musical interactions between geographically displaced performers and audiences, a widespread use of ambient intelligence systems employed to monitor environments in smart cities, and in general novel ways of interacting with sounds across the network (such as sound-based therapies involving remotely connected users).

We recognize that substantial standardization efforts are also necessary. Just like for the general IoT field, the success of the IoS strongly relies on standardization requirements, which are currently unmet. The definition of standards specific to the IoS (for platforms, formats, protocols, and interfaces) will allow to achieve interoperability between heterogeneous systems. Issues related to security and privacy of information, which are also common to the IoT, need to be addressed, especially for IoS systems deployed for the masses. Moreover, research will need to address the challenge of how to design systems capable of supporting rich interaction paradigms that enable users to fully exploit the potentials and benefits of the IoS.

Fig. 1 .

 1 Fig. 1. A schematic representation of the relation between the Internet of Sounds (IoS), the composing fields of Internet of Audio Things (IoAuT) and Internet of Musical Things (IoMusT), and the parent fields of the Internet of Media Things (IoMT) and Internet of Things (IoT).

Fig. 2 .

 2 Fig. 2. Schematic representation of a general architecture supporting IoS ecosystems

Fig. 3 .-Fig. 4 .Fig. 5 .

 345 Fig. 3. Diagram of a 5G architecture supporting low-latency, highly-reliable musical interactions between two geographically displaced musicians.

Fig. 6 .

 6 Fig. 6. Diagram of a wireless acoustic sensor network for wildlife monitoring.

  This article has been accepted for publication in IEEE Internet of Things Journal. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2023.3253602 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

  39 http://www.w3.org/TR/owl-profiles This article has been accepted for publication in IEEE Internet of Things Journal. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2023.3253602 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ from local devices to MEC servers, what is the interplay between MEC and cloud servers, and how to optimize the position of MEC servers even in a dynamic fashion.

  H. StorageStorage is an important component of the IoS ecosystem with relevant open challenges. Fast access from embedded This article has been accepted for publication in IEEE Internet of Things Journal. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2023.3253602 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

  This article has been accepted for publication in IEEE Internet of Things Journal. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2023.3253602 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

TABLE I POTENTIAL

 I BENEFITS FOR DIFFERENT STAKEHOLDERS BROUGHT ABOUT THE IMPLEMENTATION OF THE IOS PARADIGM.

https://audiomostly.com/2022/call/cfiwis/

https://internetofsounds.net/

https://www.comsoc.org/about/committees/ emerging-technologies-initiatives/internet-of-sounds

https://www.aes.org/journal/online/JAES V69/10/JAES V69 10 PG706.pdf

E.g., https://cordis.europa.eu/project/id/749561

E.g., https://cordis.europa.eu/project/id/956369

E.g., https://www.audiolabs-erlangen.de/fau/professor/peters/teaching/ 2021s AIoT

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

This article has been accepted for publication in IEEE Internet of Things Journal. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2023.3253602 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

http://www.w3.org/TR/sparql11-query/

http://dcase.community This article has been accepted for publication in IEEE Internet of Things Journal. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2023.3253602 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

https://merlin.allaboutbirds.org

https://www.warblr.co.uk

https://birdnet.cornell.edu This article has been accepted for publication in IEEE Internet of Things Journal. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2023.3253602 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

http://www.axoloti.com/

http://musicontology.com/

https://w3id.org/aufx/ontology/1.0

https://w3id.org/afo/onto/1.1

http://purl.org/ontology/iomust/internet of things/0.1

https://w3id.org/ac-ontology/aco

https://www.ebu.ch/metadata/ontologies/ebucore/

https://freemusicarchive.org

https://archive.org

https://www.europeana.eu This article has been accepted for publication in IEEE Internet of Things Journal. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2023.3253602 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

https://www.ismir.net/resources/datasets/

http://www.audiocontentanalysis.org/data-sets/

https://www.w3.org/TR/webaudio/

https://www.soundtrap.com

http://labs.freesound.org/fse/

This article has been accepted for publication in IEEE Internet of Things Journal. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2023.3253602 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ IEEE INTERNET OF THINGS JOURNAL, VOL. XX, NO. XX, XXX 20XXX