Lionel Tidjon

Senior Member, IEEE Foutse Khomh

The Different Faces of AI Ethics Across the World: A Principle-Implementation Gap Analysis

Keywords: AI, Ethics, Ethical Principles, Principle Implementation, Gap Analysis, EthicsOps Human augmentation, Bias evaluation, Explainability By Justification, Reproducible operations, Displacement strategy, Pratical Accuracy, Trust by privacy, Data risk awareness

Artificial Intelligence (AI) is transforming our daily life with several applications in healthcare, space exploration, banking and finance. These rapid progresses in AI have brought increasing attention to the potential impacts of AI technologies on society, with ethically questionable consequences. In recent years, several ethical principles have been released by governments, national and international organisations. These principles outline high-level precepts to guide the ethical development, deployment, and governance of AI. However, the abstract nature, diversity, and context-dependency of these principles make them difficult to implement and operationalize, resulting in gaps between principles and their execution. Most recent work analysed and summarized existing AI principles and guidelines but they did not provide findings on principle-implementation gaps and how to mitigate them. These findings are particularly important to ensure that AI implementations are aligned with ethical principles and values. In this paper, we provide a contextual and global evaluation of current ethical AI principles for all continents, with the aim to identify potential principle characteristics tailored to specific countries or applicable across countries. Next, we analyze the current level of AI readiness and current implementations of ethical AI principles in different countries, to identify gaps in the implementation of AI principles and their causes. Finally, we propose recommendations to mitigate the principle-implementation gaps.

INTRODUCTION

AI achieved successful results in several domains such as healthcare [START_REF] Kourou | Machine learning applications in cancer prognosis and prediction[END_REF], automotive [START_REF] Kuutti | A survey of deep learning applications to autonomous vehicle control[END_REF], and aviation [START_REF] Gui | Flight delay prediction based on aviation big data and machine learning[END_REF]. The powerful and profound impact of AI on society led to several debates about the principles and values that must guide its implementation and use. In recent years, several ethical AI principles have been released by governments, national, and international organisations. Some of these principles are generic while others are tailored to a specific environment or context. Most generic principles have been designed by international organisations, with the aim to provide abstract and inclusive principles, that will lead the implementation and use of AI in member countries or companies. Contextual principles have been designed by national organizations and governments to take into account the specific AI factors in a given country. For example, no robotics principle is provided for a country with no robotics industry. As a consequence, countries around the world are having different principles tailored to their context in addition to generic principles from the international organizations to which they are affiliated. This diversity and context-dependency of ethical AI principles make them difficult to implement and operationalize; resulting in gaps between the stated principles and their execution.

Recently, researchers have analysed and summarized AI principles proposed on some continents [START_REF] Floridi | Ai4people-an ethical framework for a good ai society: Opportunities, risks, principles, and recommendations[END_REF]- [START_REF] Fjeld | Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for ai[END_REF]. However, to the best of our knowledge, none of these studies examined the gaps between the principles and their implementation. This paper aims to fill this gap in the literature by analyzing ethical AI principles around the world and the possible gaps

• The authors are with Polytechnique Montréal, Montréal, QC H3C 3A7,

Canada. E-mail: {lionel.tidjon, foutse.khomh}@polymtl.ca between the principles and their implementations. We also investigate the root causes of principle-implementation gaps and propose mitigation strategies. The analysis of ethical AI principles around the world is important for understanding the characteristics of ethical principles tailored to specific countries, and identifying similarities between principles from different countries, as well as similarities between global principles from different continents and those from international organizations such as the Organization for Economic Cooperation and Development (OECD), the Global Partnership on AI (GPAI), and the Group of Twenty (G20).

In this paper, we provide a contextual and holistic evaluation of current ethical AI principles and their implementations by analyzing a set of 100 documents and websites containing 100 AI principles from 29 countries in the 6 continents and 121 implementations of these principles from 14 countries in the 6 continents (e.g., guidelines, standards, tools). The documents are selected based on their reliability, recency, and diversity. Results show that the most cited principle in the countries is Transparency. The principles that cover most countries (i.e., global principles) are Privacy, Transparency, Fairness, Security, Safety, Responsibility, Accountability, Explainability, Well-being, Human Rights, and Inclusiveness. Results from the implementation analysis show that United States and United Kingdom are more favorable for investment, business, and research in AI with a good growth stability between 2020 and 2021. They also show the fast growing countries in terms of AI readiness and the current implementation tools (e.g., laws, standards, checklists, training, softwares). The principleimplementation gaps include lack of effective implementation tools and lack of practical training courses on AI. Recommendations for gap mitigation include inclusiveness and diversity of team roles, education and awareness on ethical values, and use of EthicsOps. EthicsOps is the continuous planning, execution, and monitoring of ethics principles in a project lifecycle (including AI project). It ensures that team roles are diverse (i.e., no race and gender discrimination), business requirements are pro-ethically aligned (i.e., meet ethics laws and standards, ethics governance guidelines), ethical controls are enforced (e.g., checklists), and that ethics by design practices are applied [START_REF] Commission | Ethics by design and ethics of use approaches for artificial intelligence[END_REF] during the project lifecycle. Governments, national, and international organisations can leverage these results to guide their AI initiatives and conduct ethical AI implementations adapted to their context.

The rest of this paper is structured as follows. Section 2 reviews the relevant literature. Section 3 describes the methodology of our study. In Section 4, we evaluate recent ethical AI principles tailored to specific countries and those that are more generic. Section 5 presents an analysis of current implementations of AI principles. In Section 6, we provide recommendations to mitigate the gaps between AI principles and their implementation. Section 7 discusses threats that could affect the validity of the reported results. Section 8 concludes the paper and outlines some perspectives.

RELATED WORK

Floridi et al. [START_REF] Floridi | Ai4people-an ethical framework for a good ai society: Opportunities, risks, principles, and recommendations[END_REF] proposed five ethical principles to lead the development and adoption of AI for the benefit of society and presented 20 concrete recommendations to assess, develop, incentivise, and support good AI. However, the proposed recommendations are high-level and mainly focus on AI ethics governance. This work does not evaluate the application of AI ethics in practice (e.g., country AI readiness analysis, principle-implementation gap analysis).

Jobin et al. [START_REF] Jobin | The global landscape of ai ethics guidelines[END_REF] performed an in-depth analysis of AI guidelines and observed that they are structured around five global principles (transparency, justice and fairness, nonmaleficence, responsibility, and privacy). They also identified two important actions to be undertaken by the global community: [START_REF] Kourou | Machine learning applications in cancer prognosis and prediction[END_REF] translating principles into practice and [START_REF] Kuutti | A survey of deep learning applications to autonomous vehicle control[END_REF] seeking harmonization between AI ethics codes (soft law) and legislation (hard law). However, they have not analyzed ethical AI implementations.

Fjeld et al. [START_REF] Fjeld | Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for ai[END_REF] proposed a visualization of AI principles classified by themes (e.g., International Human Rights, Privacy) and sectors (multi-stakeholder, inter-governmental, private, government, civil society) to provide a high-level snapshot of the current state of AI ethics governance. Similarly to [START_REF] Floridi | Ai4people-an ethical framework for a good ai society: Opportunities, risks, principles, and recommendations[END_REF], [START_REF] Jobin | The global landscape of ai ethics guidelines[END_REF], no analysis of ethical AI implementations and principle-implementation gaps was conducted.

Hagendorff et al. [START_REF] Hagendorff | The ethics of ai ethics: An evaluation of guidelines[END_REF] evaluated 22 guidelines and found that diversity is lacking in the AI community, political abuse of AI systems (e.g., fake news, election fraud, automated propaganda) is also missed by the guidelines, and artificial general intelligence is not discussed. Our work extends [START_REF] Hagendorff | The ethics of ai ethics: An evaluation of guidelines[END_REF] with local and global analysis of AI principles (countries, continents), country AI readiness analysis, principle-implementation gap analysis and mitigation.

STUDY METHODOLOGY

The goal of this study is to understand the landscape of AI principles around the world, through a contextual and holistic analysis of the characteristics of the different ethical AI principles proposed by countries from different continents, in order to identify potential principleimplementation gaps, their root causes, and formulate recommendations for gap mitigation. The perspective of this study is that of governments, national, and international organisations, which can leverage our findings to guide their AI strategy in developing and implementing AI principles adapted to their context. The context of this study is a set of 100 documents and websites containing 100 AI principles from 29 countries in the 6 continents (e.g., fairness, transparency) and 121 implementations of these principles (e.g., guidelines, standards, tools) from 14 countries in the 6 continents, gathered online from reports of national/international organizations and websites. The data sources are selected based on their reliability, recency, and diversity. We address the following research questions:

RQ 1 : What are the characteristics of ethical AI principles across countries and continents? This research question aims to help identify the countries (resp. continents) that released a high number of principles, the principles on which countries (resp. continents) place a greater emphasis (i.e., the most cited principles) and the principles shared across different countries (resp. continents). The common principles found between continents are compared to those from existing international organisations (i.e., OECD, GPAI, G20) to identify missing in the existing principles. RQ 2 : What is the current level of AI readiness in countries as well as the current implementations of the ethical AI principles? This research question seeks to identify the level of preparedness of countries based on scoring indexes [START_REF] Shearer | Government ai readiness index[END_REF], [START_REF] Nettel | Government ai readiness index[END_REF] to implement the necessary changes required by the adoption and development of AI technologies. We also aim to understand how ethical AI principles are implemented in these countries. RQ 3 : What are the gaps between ethical AI principles and their implementations as well as their root causes? This research question aims to identify any gap that may exist between the stated AI principles and their implementation. We also aim to identify the root causes of these gaps, in order to formulate recommendations for their mitigation. RQ 4 : What are potential solutions for principleimplementation gap mitigation? This research question aims to propose some recommendations, to mitigate gaps between ethical AI principles and their implementation.

Data collection

Three selection criteria have been defined for data collection: reliability, recency, and diversity. The reliability criteria ensures that the data can be trusted or is provided by a national-artificial-intelligence -policy/ 2021 Government South America trusted sources. More than 350 reports and websites from companies, national, and international organizations have been inspected to collect AI principles and proposed implementation tools. For each candidate report, we assessed the trustworthiness of the institutions behind the report. An institution is considered to be trustworthy if it is officially recognised by an official state (e.g., The White House) and an international organisation (e.g., UNESCO).

The recency criteria ensures that the data source contains recent information about ethical AI principles. From the 350 collected reports and websites, we have selected 100 relevant reports and websites published between 2016 and 2022. To select a report, we first read the document/website and check if it contains a clear reference to ethical principles (e.g., Our principles, We follow these principles), or to the implementation of principles (e.g., law and standard references, reference links for ethical AI softwares, training courses on AI ethics). Reports that did not contain ethical AI principles nor implementations were ignored.

The diversity criteria is used to ensure that we collect data from different countries and continents. The final set of reports analyzed in this paper covers countries from Europe, North America, South America, Oceania, Africa, and Asia. The references of these reports can be found in our replication package [START_REF]Aiethics[END_REF].

Data extraction

In the hundred reports and websites, we have manually selected 100 ethical AI principles randomly, representing a confidence level of 90% with an error margin of 8.25%. Each principle contains 2 to 15 keywords including Safe, Safety, Transparent, Transparency, Robust, Robustness, Ex- plainable, Explainability, Fair, Fairness, Secure, Security, Responsible, Responsibility, Inclusive, and Inclusiveness. Using Microsoft Excel, we have built regular expressions to extract these string keywords. The following regular expression was used to extract each principle keyword: *Safe* (Safety), *Transparen* (Transparency), *Robust* (Robustness), *Explainab (Explainability), *Fair* (Fairness), *Secur* (Security), *Responsib* (Responsibility), and *Inclusive* (Inclusiveness). For each principle, we extracted and recorded the following information: the name of the report's provider (e.g., US Department of Defense, IEEE), the title of the report, the location where the report is published, the number of principles contained in the report, the contained principles, the reference to get the principles, the date when the principle was released, the type of institution (e.g., government, international, company, university), and continent. An example of recorded information is shown in Table 1. The complete dataset is available in our replication package [START_REF]Aiethics[END_REF].

We have extracted 521 occurrences of AI principle keywords from 29 countries: Canada, United States (US), United Kingdom (UK), China, Australia, Singapore, United Arab Emirates (UAE), Spain, Switzerland, Chile, Colombia, Sweden, Belgium, Israel, Amsterdam, New Zeland, India, Germany, Hong Kong, South Korea, South Africa, Finland, Japan, France, Netherlands, Norway, Russia, Ireland, and Italy. Table 3 presents the distribution of these principles across the countries. The 521 occurrences are grouped using 33 distinct keywords: Fairness, Transparency, Accountability, Safety, Security, Robustness, Explainability, Interpretability, Well-being, Human Oversight, Human Rights, Sustainability, Equity, Reliability, Privacy, Justice, Autonomy, Human Dignity, Responsibility, Solidarity, Beneficence, Non-maleficence, Non-discrimination, Contestability, Human-centric, Inclusiveness, Trustworthy, Democracy, Governance, Bias, Integrity, Controllability, and Accuracy.

We also extracted 121 AI ethics implementations from the hundred reports and websites (see Table 2), representing a confidence level of 95% with an error margin of 8.91%. In

Data Processing and Analysis

We processed the extracted data using word frequency analysis. The word frequency analysis is performed by counting the number of AI principle occurrences per country and per continent. The aim being to identify potential characteristics such as the countries (resp. continents) that released a high number of principles, the principles on which specific countries (resp. continents) place a greater emphasis and the principles shared across different countries (resp. continents). Then, findings for the defined research questions are presented through point clouds using RStudio Integrated Development Environment (IDE). Point clouds show small and large circles depending on the number of occurrences of the principle; thus helping to identify the principle emphasis in the country (resp. continent).

EVALUATION OF ETHICAL AI PRINCIPLES

In this section, we analyze the processed data and provide answers to our first research question (i.e., RQ1). We organise our analysis in two parts: [START_REF] Kourou | Machine learning applications in cancer prognosis and prediction[END_REF] we perform a contextual analysis of ethical AI principles; by identifying and analyzing AI ethical principles defined for specific countries, and (2) We perform a holistic analysis by considering ethical AI principles that are generic, i.e., principles stated at a global scale (e.g., continent-level).

Contextual evaluation

To identify the characteristics of ethical AI principles across countries (RQ1), we have formulated two sub-research questions: What are the countries that released a high number of principles ? and What are the most cited principles in the countries as well as the principles shared across different countries ? have not yet released any national ethical AI principles. However, Some of the missing countries are part of (or in the process of becoming member of) international organizations such as the Organization for Economic Cooperation and Development (OECD), which will be described later.

In South America, Colombia is the first country to release a national ethical AI principles, followed by Chile. Other South American countries such as Argentina and Brazil do not have national ethical AI principles, but they are in a partnership with OECD. Chile, Costa Rica, Colombia, and Mexico are also OECD members. In Europe, UK is the first country to publish national ethical AI principles, followed by Germany. France appears in the third position followed by Belgium. In Asia, China is the first country with the highest number of principles and Japan occupies the second position. Next come South Korea and Singapore in third and fourth position, respectively. In Oceania, Australia is the first country with the highest number of principles followed by New Zealand. Other countries such as Nauru did not release national principles. In Africa, South Africa is the q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q North America Europe Oceania Asia South America Africa

Country

Principles value q q q q q 0 25 50 75 100

Fig. 1: Number of occurrences per principle per country first to have national principles via the Research ICT Africa (RIA) 1 . Other African's countries do not yet have national ethical AI principles, but some are part of (or in the process of becoming member of) OECD.

What are the most cited principles in the countries as well as the principles shared across different countries?

This research question aims to identify the principles on which countries place greater emphasis (i.e., the most cited principles) and the principles shared across different countries. Fig. 1 shows the number of principle occurrences per country from the set of principles extracted from the selected reports. A high (resp. low) circle diameter means a high (resp. low) number of principle occurrences in the country.

In North America, the Transparency principle has the largest frequency in United States and Canada. Transparency and Privacy principles are both cited in United States and Canada. In United States, we have identified 5 top frequent principles as follows: Transparent, Fairness, Accountability, Privacy, and Security. In Canada, the most 1. https://researchictafrica.net/ frequent principles are Transparency, Responsibility, Privacy, Sustainability, Autonomy, and Well-being.

In Europe, the Transparency principle is the most frequent principle across 13 countries: UK, Germany, France, Belgium, Switzerland, Finland, Sweden, Norway, Spain, Amsterdam, Russia, Ireland, and Italy. It is described in Fig. 1 by gray circle appearing in these countries. The second most frequently occurring principle is Fairness. This principle covers 9 countries: UK, Belgium, Sweden, Germany, France, Spain, Amsterdam, Finland, and Italy. Security, Responsibility and Accountability principles are found in 8 countries (see Fig. 1). The Security principle appears in Canada, US, UK, Spain, Switzerland, Sweden, Belgium, Germany, Finland, and Ireland. The Responsibility principle is found in Germany, UK, Switzerland, Sweden, France, Finland, Russia, and Netherlands. The Accountability principle appears in UK, France, Belgium, Sweden, Germany, Norway, Ireland, and Italy. Explainability and Safety principles appear in 7 countries including UK, Germany, Sweden, and France. The Privacy principle appears in 6 countries including UK, France, Norway, and Germany. In Asia, the Transparency principle is the most frequent principle across 8 countries: Japan, Singapore, South Korea, Israel, India, China, United Arab Emirates (UAE), and Hong Kong (see Fig. 1). The second most frequent principle is Accountability. This principle is found in 7 countries: Japan, Singapore, Israel, South Korea, India, UAE, and Hong Kong. Safety, Security, and Privacy principles appeared in 6 countries. The Safety principle appears in South Korea, Singapore, Japan, Israel, India, and UAE; Security and Privacy principles appear in Japan, China, Israel, South Korea, India, and UAE. Other principles like Fairness, Explainability, and Responsibility are ranked in fourth position and found in 4 countries. The Fairness principle appears in China, Singapore, Israel, and UAE; and the Explainability principle appears in South Korea, Israel, UAE, and Singapore. The Responsibility principle is found in China, Japan, South Korea, and Israel.

Summary 1

•

In the studied samples, United States, United Kingdom, and Canada released the highest number of principles between 2016 and 2021.

• Most countries from North America, Europe, Asia, Oceania, and South America placed a greater emphasis on the Transparency principle, identified by a high number of occurrences.

•

In North America, Transparency and confidentiality are the principles most frequently cited by countries.

• Transparency, Fairness, and Security are the principles most frequently stated by countries in Europe.

•

In Asia, Transparency and Accountability are the most frequently stated principles (by countries).

• Transparency, Fairness, Accountability, Security, Privacy, Reliability, and Well-being are the most frequently cited principles, by the countries in Oceania.

•

In South America, Colombia and Chile share principles such as Security (resp. Safety), Human Rights (resp. Human-centric), and Nondiscrimination (resp. Inclusiveness).

•

In Africa, South Africa is focused on Fairness, Inclusiveness, Safety, Privacy, Responsibility, and Human Rights principles.

Oceania, Australia and New Zealand have 7 principles in common: Transparency, Fairness, Accountability, Security, Privacy, Reliability, and Well-being. In addition, Australia has Explainability, Contestability, and Human-centric principles whereas New Zealand has Human Oversight and Justice principles. In South America, Columbia and Chile follows different principles. Columbia follows the Transparency principle and 7 others principles including Security, Privacy, Responsibility, Non-discrimination, Controllability, and Beneficence. Chile has the following principles: Humancentric, Sustainability, Safety, Inclusiveness, and Democracy. In Africa, South Africa follows Fairness, Inclusiveness, Safety, Privacy, Responsibility, and Human Rights principles.

Holistic evaluation

In this section, ethical AI principles are analyzed on a global scale to identify the principles on which continents place a greater emphasis (i.e. the most cited principles) and the principles shared across different continents. The identified shared principles are compared with common principles provided by international organizations such as OECD, GPAI, and G20 to which different countries are affiliated. To structure our analysis, we have defined two sub-research questions: What are the most cited principles in the continents as well as the principles shared across different continents? and How do continent-specific principles differ from those provided by international organizations (i.e., OCECD, GPAI or G20)?.

What are the most cited principles in the continents as well as the principles shared across different continents ?

This research question aims to identify the principles on which continents place greater emphasis (i.e., the most cited principles) and the principles shared across different continents. Fig. 2 shows that Europe and North America have the highest number of ethical AI principles. Human Rights and Privacy principles are shared by all continents. Transparency, Fairness, Security, and Safety principles have the highest number of occurrences and they appear in continents: Europe, North America, Asia, Oceania, South America, and Africa. Responsibility, Well-being, and Inclusiveness principles are also found in 5 continents: Europe, North America, Asia, South America, Oceania, and Africa. Accountability and Explainability principles appear in continents with a high number of occurrences: Europe, North America, Asia, and Oceania.

Global principles. We identified 11 global principles that cover most continents with a high number of occurrences: Transparency, Privacy, Fairness, Security, Safety, Responsibility, Accountability, Explainability, Well-being, Human Rights, and Inclusiveness.

How do continent-specific principles differ from those provided by international organizations such as OCECD, GPAI or G20 ?

In this research question, we aim to examine the alignment between the global principles developed on different continents and those proposed by international organizations (i.e., OECD, GPAI, G20). 4.2.2.1 GPAI: GPAI 2 is a multi-stakeholder association consisting of scientists, industries, civil society, governments, international organisations and academia aiming to bridge the gap between theory and practice on AI. They 2. https://gpai.ai/ q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q A c c o u n ta b il it y North America South America Africa Continent Principles value q q q 0 50 100 150

Fig. 4.2.2.3 G20: G20 4 is an intergovernmental forum with 19 country members and the European Union, that addresses global economy issues (e.g., climate change). G20 AI principles are drawn from the OECD AI principles. Thus, G20 AI principles do not also contain Privacy and Responsibility principles.

Summary 2

• According to our analysis, North America and Europe released the highest number of principles between 2016 and 2021.

•

The 11 global principles that cover most continents with a high number of occurrences are: Privacy, Transparency, Fairness, Security, Safety, Responsibility, Accountability, Explainability, Wellbeing, Human Rights, and Inclusiveness.

• GPAI ethical AI principles do not include the Responsibility principle.

• OECD and G20 ethical AI principles do not include Privacy and Responsibility principles.

EVALUATION OF ETHICAL PRINCIPLE IMPLEMEN-

TATIONS

In order to identify gaps between ethical AI principles and their implementations, it is important to review and understand the current implementations in the countries. In [START_REF] Shearer | Government ai readiness index[END_REF], [START_REF] Nettel | Government ai readiness index[END_REF], Oxford Insights and International Development Research Centre (IRDC) provide information about the AI readiness of countries in terms of governance, infrastructure, and technology for AI implementation support. Based on the information contained in [START_REF] Shearer | Government ai readiness index[END_REF], [START_REF] Nettel | Government ai readiness index[END_REF], we perform a country AI readiness analysis between 2020 and 2021, to identify the current progression state of AI implementations in the different countries. Next, we perform an in-depth analysis of recent ethical AI implementations documents, such as guidelines, code of conduct (i.e., regulation and laws, standards), governance rules, ethical AI assessment tools (i.e., checklists, algorithmic assessments), implementation tools, and awareness activities (i.e., online training, campaigns). In the following, we report about these analysis; presenting the level of preparedness of different countries (i.e., their AI readiness) and the existing ethical AI implementations (RQ2). We also report about gaps identified between published ethical AI principles and their implementation (RQ3).

What is the current level of AI readiness in countries?

In order to identify the country AI readiness (RQ2), scoring indexes [START_REF] Shearer | Government ai readiness index[END_REF], [START_REF] Nettel | Government ai readiness index[END_REF] evaluates the current level of AI preparedness following different levels: (1) government, (2) technol-4. https://g20.org/ ogy factor, and (3) data and infrastructure. At the government level, the score gives insights about the willingness of the government of the country to adopt AI, and its ability to adapt and innovate. In the technology sector, the score gauges the level of supply of AI tools in the country. At the data and infrastructure level, the score provides insights about the implementation of AI tools and assessing whether they are trained on high quality and representative data, and whether the government of the country has the appropriate infrastructure for delivering AI solutions and fostering its adoption by citizens [START_REF] Nettel | Government ai readiness index[END_REF].

Table 5 shows the AI readiness score for 20 countries in 2021: US, Singapore, UK, Finland, Netherlands, Sweden, Canada, Germany, Denmark, Republic of Korea, France, Japan, Norway, Australia, China, Luxembourg, Ireland, Taiwan, UAE, and Israel. At the government level, Singapore comes in first position (94.88) followed by US (88.46). Finland is ranked in the third position (88.45) followed by UK (85.69). The Republic of Korea and Canada are respectively in fifth (85.27) and sixth (84.36) position. In the technology sector, US leads with a score of 83.31, followed by Germany with a score of 67.68. Next is Sweden, ranked in third position (67.37), followed by UK (67.26). Singapore and Netherlands are respectively in fifth (66.69) and sixth position (66.17). At the data and infrastructure level, US is ranked first (92.71) followed by UK (90.81). Netherlands is ranked in third position (88.92) followed by Japan (87.32). Next is Luxembourg and France appearing respectively in fifth (86.8) and sixth position (86.53). From these results, US, UK, and Germany are more favorable for AI development, investment, business, and research since they are well prepared in terms of AI technology, data and infrastructure, and AI governance. Other countries such as Finland, Sweden, Netherlands, Japan, and Singapore are also propice for investment, business, and research in AI.

Table 6 shows the yearly AI readiness score of 18 countries: US, Singapore, UK, Finland, Netherlands, Sweden, Canada, Germany, Denmark, Republic of Korea, France, Japan, Norway, Australia, China, Luxembourg, UAE, and Israel. Overall, US and UK respectively maintained the first position (+2.681 increase) and second position (+0.126 increase) from 2020 to 2021. Singapore moved from the sixth position in 2020 to the second position in 2021 with a +3.756 increase. Finland moved from the third position in 2020 to the fourth position in 2021 with a -0.008 decrease. Similar to Finland, Germany's readiness score experienced a significant decrease of -1.714 from 2020 to 2021. Sweden's AI readiness score also decreased from 78.772 in 2020 to 78.16 in 2021 (-0.612 decrease). Netherlands have a +3.213 increase from 2020 to 2021. Canada have experienced a high growth from 2020 to 2021 with a +4.572 increase. China have the highest growth with a +5.34 increase from 2020 to 2021. Other countries like France, Japan, Norway, Australia, Israel, and Luxembourg also experienced a significant growth (as shown in Table 6). From these results, China, Canada, Singapore, and Netherlands have a high potential of increase in the next years in terms of AI governance, technology, data and infrastructure. This also makes them favorable for investment, business, and research in AI.

Summary 3

• At the government level, Singapore was the leader followed by US in 2021. In the technology sector, United States was the leader followed by Germany in 2021. At the data and infrastructure level, US was the leader followed by UK in 2021. Then, US, UK, and Germany are more favorable for investment, business, and research in AI.

•

In terms of AI readiness, United States maintained the first position followed by United Kingdom from 2020 to 2021. China and Canada experienced the highest growth (resp. +5.34 and +4.572 increase) from 2020 to 2021. The AI readiness of Germany and Republic of Korea decreased significantly (resp. -1.714 and -1.145) from 2020 to 2021. Then, China and Canada have a high potential of increase in the next years, also making them favorable for investment, business, and research in AI.

What are the current ethical AI principle implementations?

Several ethical AI guidelines, regulation, and tools have been proposed for designing, and implementing ethical AI systems. The ethical principle implementation materials found through our review (described in Sections 5.2.1-5.2.6) are divided in 6 categories: guidelines, code of conduct (e.g., standards), governance tools, assessment tools (e.g., checklists), AI ethics softwares, and awareness materials (e.g., training courses). Guidelines, code of conduct, and governance tools provide high-level implementations of ethical AI principles while awareness materials, assessment tools, and softwares are more technical. Here after, the existing ethical AI implementation tools are presented (RQ2).

AI Ethics Guidelines

Table 7 shows 26 ethical AI guidelines ranging from 2017 to 2022 and classified by name, author, organization, year, and location. T. Hagendorff [START_REF] Hagendorff | The ethics of ai ethics: An evaluation of guidelines[END_REF] analyzed 22 guidelines and found omissions in these guidelines. They were not included in the 26 guidelines since the paper [START_REF] Hagendorff | The ethics of ai ethics: An evaluation of guidelines[END_REF] itself provides a summary of those guidelines. T. Hagendorff found that artificial general intelligence was not discussed due to the fact that the guidelines were mostly written by research groups with computer science background. He also observed that diversity among people and technology is lacking in the AI community. He also found that very few guidelines was provided to avoid AI systems being used for political abuses (e.g., fake news, election fraud, automated propaganda).

J. Morley et al. [START_REF] Morley | From what to how: an initial review of publicly available ai ethics tools, methods and research to translate principles into practices[END_REF] built a typology to apply ethics at each stage of the Machine Learning (ML) development pipeline by identifying the tools and methods available, and developers or companies researching and producing them. B.C. Goehring et al. [START_REF] Goehring | Advancing ai ethics beyond compliance[END_REF] proposes to educate students and professionals by adding AI ethics in business and law schools, computer science programs, technical organizations; setting guidelines and standards at national, supranational, and global levels; and creating an unified approach to ensure a robust, complete, and holistic assessment of present and future AI ethics implications.

A. Kaminski et al. [START_REF] Kaminski | From principles to practice. an interdisciplinary framework to operationalise ai ethics[END_REF] proposes to apply the Values, Criteria, Indicators, and Observables (VCIO) model to AI ethics. The ethical values defined include Transparency. For transparency, the criteria as well as indicators include the disclosure of the origin of the datasets (e.g., an indicator can be whether data is documented) and the disclosure of properties of algorithm/model used (e.g., an indicator can be whether the model has been tested/used before). Then, observables can be a comprehensible logging of all training and operating data, version control of datasets, and wide usage and testing of the model both in theory and practice. D. Leslie [START_REF] Leslie | Understanding artificial intelligence ethics and safety: A guide for the responsible design and implementation of ai systems in the public sector[END_REF] describes constraints that an AI project must follow and explain how to practically build an ethical platform. An AI project must be ethically permissible, fair and non-discriminatory, worthy of public trust, and justifiable. Their proposed ethical platform has three building blocks: (1) Support, Underwrite, and Motivate (SUM) ethical values which are based on the notions of Respect, Connect, Care, and Protect; (2) Fairness, Accountability, Sustainability, and Transparency (FAST) actionable principles which provides the moral and practical tools to deliver safe and reliable AI solutions; and a (3) process-based governance (PBG) framework to operationalize the SUM Values and the FAST Track Principles across the entire AI project delivery workflow.

A.H. Nelson et al. [START_REF] Nelson | A toolkit for centering racial equity throughout data integration[END_REF] recommands to normalize, organize, and operationalize racial equity throughout data integration. They suggest an ongoing process at each stage of the data life cycle-planning, i.e., data collection (e.g., shared data, inclusive data), data access (open data, restricted data, available data), use of algorithms and statistical tools (i.e., following Responsibility, Explainability, Accuracy, and Auditability, Fairness principles), data analysis (e.g., using participatory research to bring multiple perspectives to the interpretation of the data), reporting and dissemination (e.g., providing clear documentation of the data analysis process with analytic files for reproducibility).

The AI High Level Expert Group (HLEG) [START_REF] Smuha | Ethics guidelines for trustworthy ai[END_REF] proposed technical and non-technical methods to implement AI ethics. Technical methods include architectures for Trustworthy AI, ethics and rules of law by design (e.g., transparency-by-design, fairness-by-design), explanation methods (e.g., IBM XAI), testing and validation (e.g., red teaming, bug bounty programs). Non technical-methods include regulation, codes of conduct, standardization, certification, accountability via governance frameworks, diversity and inclusive design teams, education and awareness to foster an ethical mind-set, stakeholder participation and social dialogue.

The Foundation for Best Practices in Machine Learning (FBPML) [START_REF]for Best Practices in Machine Learning[END_REF] provides technical and organisational best practices. These organisational best practices include a clear definition and diversity of team roles, determination and definition of an appropriate, feasibility, and solvability of the business problem, determination of the most desirable and feasible model to achieve the desired product outcomes, management and monitoring of product pipeline during the product management lifecycle (i.e., Design, Exploration, Experimentation, Development, Implementation, Operationalisation). Technical best practices include the use of fairness and non-discrimination in products and models, data quality control, representativeness of the product model and data, performance and robustness of model outcomes, monitoring and maintenance of products and models, and explainability of model functions and outputs.

The World Economic Forum (WEF) [START_REF] Forum | Guidelines for ai procurement[END_REF] proposes data governance guidelines including the use of procurement processes with focus on outlining problems and opportunities, the definition of the public benefit of using AI while assessing risks, inclusion of procurement's within a strategy for AI adoption across government, highlighting the technical and ethical limitations of using the data to avoid bias issues (e.g., data quality such as data completeness, representativenes and accuracy), working with a diverse and multidisciplinary team, focusing throughout the procurement process on mechanisms of accountability and transparency norms. Other ethical AI guidelines were also proposed by international organizations such as IEEE [START_REF]Ethically aligned design[END_REF], OECD [START_REF] Ocde | Tools and Ethics for Applied Behavioural Insights: The BASIC Toolkit[END_REF], International Association of Privacy Professionals (IAPP) [START_REF] Of | Building ethics into privacy frameworks for big data and ai[END_REF], Atomium-European Institute for Science, Media and Democracy (EISMD) [START_REF] Pagallo | On good ai governance: 14 priority actions, a smart model of governance, and a regulatory toolbox[END_REF], Hub for Next Generation Internet (HUB4NGI) Consortium [START_REF] Taylor | Responsible ai-key themes, concerns & recommendations for european research and innovation[END_REF]; national organizations such as the UK's Information Commissioner Office [START_REF] Office | Guidance on the ai auditing framework -draft guidance for consultation[END_REF], [START_REF] Office | Guidance on ai and data protection[END_REF]; companies such as Thomson Reuters [START_REF] Leidner | Ethical by design: Ethics best practices for natural language processing[END_REF] and Markkula Center [START_REF] Vallor | An ethical toolkit for engineering/design practice[END_REF]; and universities such as the University of Cambridge [START_REF] Zhou | Design ethically toolkit[END_REF], Trinity College [START_REF]Ethics canvas[END_REF], and ETH Zurich [START_REF] Jobin | The global landscape of ai ethics guidelines[END_REF].

AI Ethics Code of Conduct

The code of conduct describes regulatory norms, laws, and standards defined by governments and international organizations. 5.2.2.1 Regulations & Laws: Table 8 shows 20 ethical AI regulations and laws ranging from 1974 to 2018 and classified by name, country, and year of publication. In Canada, AI ethics is supported by the protection of personal information laws such as the Privacy Act (1985) [START_REF]Privacy act[END_REF], the Personal Information Protection and Electronic Documents Act (2000) [START_REF]Personal information protection and electronic documents act[END_REF], and the Financial Consumer Agency of Canada Act (2001) [START_REF]Financial consumer agency of canada act[END_REF]. In US, healthcare data and personal information is protected by the Health Insurance Portability and Accountability Act of 1996 (HIPAA) [START_REF]Health insurance portability and accountability act of 1996[END_REF]. The protection of personal information laws include Privacy Act (1974) [START_REF]Privacy act[END_REF], Privacy Protection Act (1980) [START_REF]Privacy protection act[END_REF], and California Consumer Privacy Act (2018) [START_REF]California consumer privacy act[END_REF]. The EU-U.S. and Swiss-U.S. Privacy Shield (2016) [START_REF]Eu-u.s. and swiss-u.s. privacy shield[END_REF] law protects personal data transfer between EU and US. US Finan- cial institutions have to comply Fair Credit Reporting Act (2018) [START_REF]Fair credit reporting act[END_REF] and Gramm-Leach-Bliley Act (1999) [START_REF]Gramm-leach-bliley act[END_REF] for the collection, dissemination, and use of consumer information (e.g., credit reports). Other laws were proposed by UK (i.e., Data Protection Act [START_REF]Data protection act[END_REF]), EU (i.e., General Data Protection Regulation [START_REF]General data protection regulation[END_REF]), Australia (i.e., Privacy Act [START_REF]Privacy act[END_REF]), China (i.e., Personal Information Protection Law [START_REF]Personal information protection law[END_REF], Cyber Security Law [START_REF]Cyber security law[END_REF]), Singapore (i.e., Personal Data Protection Act [START_REF]Personal data protection act[END_REF], Protection from Online Falsehoods and Manipulation Act [START_REF]Protection from online falsehoods and manipulation act[END_REF]), South Africa (i.e., Protection of Personal Information Act [START_REF]Protection of personal information act[END_REF]), Dubai (i.e., Data Protection Law [START_REF]Data protection law[END_REF]), and Germany (i.e., Bundesdatenschutzgesetz [START_REF]Bundesdatenschutzgesetz[END_REF]). [START_REF]Ieee guide for the use of artificial intelligence exchange and service tie to all test environments (ai-estate)[END_REF]), and ISO/IEC Joint Technical Committee (e.g., ISO/IEC TR 24028:2020 [START_REF]Information technology -artificial intelligence -overview of trustworthiness in artificial intelligence[END_REF]). ISO/IEC JTC 1/SC 42 [START_REF]Artificial intelligence[END_REF] committees are working on norms for AI including data (ISO/IEC JTC 1/SC 42/WG 2), fairness (ISO/IEC JTC 1/SC 42/WG 3), information security and data privacy (ISO/IEC JTC 1/SC 42/AHG 4).

Standards:

The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems also provides several standards including the model process for addressing ethical concerns during system design (IEEE 7000-2021 [START_REF]Ieee standard model process for addressing ethical concerns during system design[END_REF]), transparency of autonomous systems (IEEE 7001-2021 [START_REF] Winfield | Ieee p7001: a proposed standard on transparency[END_REF]), and ontological standard for ethically driven robotics and automation systems (IEEE 7007-2021 [START_REF]Ieee ontological standard for ethically driven robotics and automation systems[END_REF]). In 2021, NIST released a standard draft [START_REF] Stanton | Trust and artificial intelligence[END_REF] for AI and user trust to help build trustworthy systems and understand user trust in AI, to minimize the risks and achieve benefits. The Association of Com-puter Machinery (ACM) Code of Ethics and Professional Conduct [START_REF]Acm code of ethics and professional conduct[END_REF] provides general principles, professional responsibilities, and professional leadership principles. These principles provide a basis/fundamental for ethical decisionmaking applied to a computing professional's conduct. Other national standards were defined by the Australian Computer Society (i.e., Code of Ethics and Professional Conduct) and the UK Committee on Standards in Public Life (i.e., Artificial Intelligence and Public Standards).

AI Ethics Governance

Ethics in corporate governance ensures trust and transparent in the exercise of power by making it credible not only to the shareholders, but also to stakeholders globally (e.g., employees, clients, suppliers) [START_REF] Dessain | Corporate governance and ethics: Shareholder reality, social responsibility or institutional necessity?[END_REF]. Table 11 presents 7 the ethical AI governance documents ranging from 2018 to 2021 and classified by name, author, organization, and year of publication. In 2019, the Berkman Klein Center for Internet and Society [START_REF]Ethics and governance of ai[END_REF] highlighted the following five key areas in which action is needed for ethics and governance of AI: the public dialogue about information quality of AI ethics, support of the development of inclusive AI governance frameworks, engagement of companies in developing socially beneficial paths for AI technology, and development of new approaches to teaching students and the public at large about the social implications of AI technology.

The European Parliament [START_REF] Koene | A governance framework for algorithmic accountability and transparency[END_REF] also reviewed governance frameworks for algorithmic accountability and transparency such as principles vs. rules based governance, governance mechanisms (e.g., co-regulation by standards like GDPR, state intervention like funding research on transparent and accountable algorithms), and existing governance proposals such as algorithmic impact assessment-based solution for accountability of algorithmic systems used by public authorities. They observed that most organisations rely on principles vs. rules that are more risks-oriented (i.e., minimization of harms/impact arising from the use of the AI technology), using impact assessment tools and they recommended a strong coordination between agencies when regulating algorithms for synchronization. A report on AI governance was also released by the University of Oxford [START_REF] Dafoe | Ai governance: a research agenda[END_REF]. This report presents key points for an ideal AI governance: values and principles such as security and autonomy, and institutional mechanisms such as the ability to provide security, ensure safe AI, and ensure resilience to concept drift and hijacking.

In 2020, the school of governance of the Technical University of Munich (TUM) [START_REF] Ütge | White paper ai ethics and governance[END_REF] analyzed the relative weaknesses of major AI users and developers in terms of AI governance (e.g., heterogeneity in the multitude of principles and ethical AI solutions proposed in by organizations and governance) and identified three elements that are important for a proper embedding of ethics in AI governance:

(1) hard and soft frameworks for regulating different AI use cases need to be distinguished by policy makers, (2) a global collaboration on defining minimum standards for AI must be done by stakeholder groups and policy makers, and (3) potential conflicts between different principles of AI ethics and the perceptions/preferences of stakeholders on them must be taken into account by corporate decision and policy makers. For ethical AI and autonomous system governance, the University of the West of England [START_REF] Winfield | Machine ethics: The design and governance of ethical ai and autonomous systems [scanning the issue[END_REF] states that all systems must be subjected to an ethical risk assessment following standards (e.g., BS 8611:2016) and then must be redesigned to reduce the impact of any ethical risks (e.g., redesign sub-systems for securely logging system inputs, outputs and decisions).

In 2019, AI4People [START_REF] Pagallo | On good ai governance: 14 priority actions, a smart model of governance, and a regulatory toolbox[END_REF] also proposed a Scalable, Modular, Adapted, Reflexive, and Technologically-savvy (SMART) coordination model using priority groups such as no-regret actions (i.e., new standards and procedures for AI), forms of engagement (e.g., new forum for discussion and consultation), and coordination mechanisms such as de-regulated special zones (i.e., kinds of living environments for the empirical testing and development of AI and robotics), in order to make rational decisions for critical issues.

AI Ethics Assessment

AI ethics assessment tools can be checklists or algorithmic processes that allows one to evaluate whether AI systems are ethically aligned. 5.2.4.1 Checklists: Table 12 shows 6 ethical AI checklists ranging from 2007 to now and classified by name, organization, and year of publication. Deon [START_REF]Deon : An ethics checklist for data scientists[END_REF] helps to easily build ethics checklist for data science projects. In 2019, the Carnegie Mellon University (CMU)'s Software Engineering Institute [START_REF] Institute | Designing ethical ai experiences: Checklist and agreement[END_REF] proposed checklists with agreement topics (e.g., we will design our AI system with the following ethical principle in mind, we value respect and security) to guide the development of accountable and trustworthy AI systems.

In 2020, Microsoft [START_REF]Ai fairness checklist[END_REF] also proposed an AI fairness checklist to address fairness issues in different stages of the AI project lifecycle: planning meetings (e.g., envision system and scrutinize system vision for potential fairnessrelated harms), reviews (e.g., define and scrutinize system architecture considering potential fairness-related harms), prototyping (e.g., fairness evaluation in system/dataset prototype), building (e.g., fairness evaluation in production datasets/code/system), launching (e.g., revision of systems to mitigate any harms), and evolution (e.g., continuous updates of checklists, continuous monitoring of fairness in systems to identify and correct new biaises).

In 2021, AlgorithmWatch [START_REF] Algorithmwatch | Automated decision-making systems in the public sector: An impact assessment tool for public authorities[END_REF] also defined two checklists (i.e., transparency triage and transparency report) to help identify which ethical transparency issues must be addressed/documented before, during, and after the usage of an automated decision-making system w.r.t ethic principles such as justice, fairness, privacy, cybersecurity, autonomy, and transparency.

In 2007, the Center For Research Libraries [START_REF] For Research | Trustworthy repositories audit & certification: Criteria and checklist[END_REF] proposed checklists for the audit and certification of organizational infrastructures, digital object management, technologies, technical infrastructure, and security based on evidence, transparency, adequacy, and measurability criteria. This checklists help to address the trustworthiness of data management systems (including digital repositories), which is important to ensure the safety and security of data processed by AI systems. The Employer Assistance and Resource Network (EARN) [START_REF] Assistance | Checklist for employers: Facilitating the hiring of people with disabilities through the use of erecruiting screening systems, including ai[END_REF] on disability inclusion also defined a set of questions for leadership (e.g., compliance with diversity and inclusion policies), human resources personnel (e.g., inclusion of individuals with disabilities), equal employment opportunity managers (e.g., equity and diversity inclusion), and procurement officers (e.g., nondiscrimination/fairness for people with disabilities). These questions on diversity and inclusion are important for AI governance in order to foster ethical AI systems. Institute [START_REF] Institute | Foresight into ai ethics (faie): A toolkit for creating an ethics roadmap for your ai project[END_REF] proposed an algorithmic assessment tools consisting in 3 phases: identification of use cases and stakeholders (e.g., use of machine learning algorithm for better prediction of nanny profiles, professional nanny as stakeholder), ethic risk discovery (e.g., using societal values such as transparency, trust, fairness, diversity, human rights), and creation of an implementation road map w.r.t the societal values. The Canadian government also built an algorithmic impact assessment tool [START_REF]Algorithmic impact assessment[END_REF] to mitigate the impact associated to the use of an automated AI decision system. The ECP Platform for the Information Society [START_REF]Artificial intelligence impact assessment[END_REF] developed steps to conduct an AI impact assessment (e.g., determine the need to perform an AI assessment, describe the AI application and its benefits, checking whether application is reliable/safe/transparent). San Francisco City provides an algorithm toolkit 5 consisting of two parts: an algorithm risk assessment (e.g., impact, data use risk, accountability risk, third-party methodology risk, historic bias risk, technical bias risk) and an algorithm risk management that matches risk to mitigation, using various severity levels such as very low, low, moderate, high, and extreme.

Algorithmic assessment:

The AI Now Institute [START_REF] Reisman | Algorithmic impact assessments: A practical framework for public agency accountability[END_REF] also released an algorithmic impact assessment process consisting of 5 steps: pre-acquisition (e.g., procurement planning, contract negotiation and signing), disclosure requirements (e.g., automated decision system disclosure, potential harms/bias/inaccuracy evaluation), comment period (e.g., public engagement to the process), due process challenge period (e.g., the public challenges the process to identify weaknesses), and impact assessment renewal. Other algorithmic assessment tools were defined by the American Civil Liberties Union (ACLU) of Washington [START_REF]Algorithmic equity toolkit[END_REF], Smart Dubai [START_REF]Ai system ethics self-assessment tool[END_REF], the Data for Children Collaborative with UNICEF [START_REF] For | Children Collaborative with UNICEF[END_REF], the Data Society [START_REF] Moss | Assembling accountability: algorithmic impact assessment for the public interest[END_REF], the European Commission High-Level Expert Group (HLEG) AI [START_REF]Altai -the assessment list on trustworthy artificial intelligence[END_REF], the UK Statistics Authority [START_REF] Authority | National statisticians data ethics advisory committee -guidelines on using the ethics self-assessment process[END_REF], the University of Colorado Law 5. https://ethicstoolkit.ai/ School Legal Studies Research [START_REF] Kaminski | Algorithmic impact assessments under the gdpr: producing multi-layered explanations[END_REF], and the European Commission Responsible Research and Innovation [START_REF] Lepri | Fair, transparent, and accountable algorithmic decision-making processes[END_REF].

AI Ethics Software

Table 13 shows 28 ethical AI softwares found in the literature, classified by name and description. Aequitas [START_REF]Bias and fairness audit toolkit[END_REF] is a fairness audit software that allows one to audit machine learning (ML) models for discrimination and bias to make equitable decisions. Deon [START_REF]Deon : An ethics checklist for data scientists[END_REF] is another tool that allows one to build ethical checklists that an AI application must follow. Fairness, Accountability, and Transparency (FAT) Forensics [START_REF]Fat forensics: Algorithmic fairness, accountability and transparency toolbox[END_REF] is a toolbox for the evaluation of FAT principles in predictive systems. AI Fairness 360 6 is tool of metrics developed by IBM to check, report, and mitigate discrimination and bias in ML models. IBM also developed AI Explainability 360 7 to help explain/interpret ML models and AI FactSheets 360 8 to improve transparency by collecting relevant information (facts) such as the criticality of the model, metrics about dataset, model, service or actions taken during the deployment of the model/service.

Other tools such as Fairlearn [START_REF]Fairlearn[END_REF], What-If [START_REF]What-if tool[END_REF], Tensor-Flow (TF) Fairness [START_REF]Fairness indicators[END_REF], LinkedIn Fairness [START_REF]The linkedin fairness toolkit (lift)[END_REF], ML-fairnessgym [START_REF]Ml-fairness-gym[END_REF], and UnBias Fairness [START_REF]Unbias fairness toolkit[END_REF] allow one to check and improve the fairness of ML models. Audit AI [START_REF]Audit ai[END_REF], TF Model Analyzer/Remediation [START_REF]Tensorflow model remediation[END_REF], and REVISE [START_REF] Wang | Revise: Revealing visual biases[END_REF] tools also help to build unbiased ML solutions. Tools like SHAP [START_REF]Revise: Revealing visual biases[END_REF], LIME [START_REF] Lime | [END_REF], InterpretML [START_REF]Interpretml -alpha release[END_REF], and PAIR Facets [100] help to explain and interpret data and ML models. Tools such as Adversarial Robustness Toolbox (ART) [101], PySyft [102], and TF Privacy [103] help secure models and data during ML lifecycle. Data Version Control (DVC) [104] allows one to track and control the versioning of data and models (similar to Git for source code management).

What are the gaps between ethical AI principles and their implementations as well as their root causes?

After identifying current AI ethics implementations, we found several gaps between ethical AI principles and their implementations [START_REF] Ütge | White paper ai ethics and governance[END_REF]. In this section, we report about 5 gaps identified between ethical AI principles and their implementations, as well as their root causes (RQ3).

Principle-Implementation Gaps

From the analysis of the literature, we have identified the following gaps: 5.3.1.1 Lack of implementation tools for some ethical AI principles: Several open source tools for AI ethics have been developed including those in Table 13. However, most tools were focused on few principles such as Fairness, Non-discrimination, and Bias (see Table 13). Few tools targeted Explainability, Interpretability, Security, and Privacy principles (e.g., AI Explainability 360, LIME, ART, TF Privacy). In addition, the existing tools do not even cover all aspects of the principles targeted; for example, the fairness implementation is limited to input data, AI models, and outputs without taking into account elements such as the fairness of the AI design/engineering process, the fairness of the AI libraries/tools and infrastructure used in the process, and EthicsOps (i.e., the continuous integration and monitoring of ethics principles in the AI project lifecycle for harm prevention). On the 33 principles, there are missing tools for several principles including Accountability, Responsibility, Traceability, Controllability, Equity, Justice, Constestability, Reliability, Autonomy, Sustainability, Governance, Integrity, Trustworthy, Transparency (partially covered by explainability/interpretability but no traceability), and Inclusiveness.

Summary 4

• The existing ethical guidelines promote education and awareness to foster an ethical mindset, the use of fairness and non-discrimination in products and models as well as data quality control, diversity and inclusiveness of team roles, explainability of model functions and outputs, traceability of all training and operating data.

•

The current codes of conduct include laws such as Canada (e.g., Privacy Act), United States (e.g., Privacy Protection Act), Europe (e.g., GDPR), Australia (e.g., Privacy Act), China (e.g., Personal Information Protection Law), and South Africa (e.g., Protection of Personal Information Act); and standards such as Bristish Standards Institution (e.g., BS 8611:2016), CIO Strategy Council (e.g., CAN/CIOSC 101:2019), IEEE (e.g., IEEE 7001-2021), ISO/IEC (e.g., ISO/IEC TR 24028:2020), and NIST (e.g., NISTIR 8332).

•

The existing ethical governance frameworks promote the development of inclusive ethical AI governance, strong collaboration and coordination between stakeholder groups and policy makers, the use of algorithmic impact assessment-based solutions for accountability of algorithmic systems.

•

The current ethical assessments include checklists such as Deon, Microsoft AI Fairness, Center for Research Libraries [START_REF] For Research | Trustworthy repositories audit & certification: Criteria and checklist[END_REF], and EARN [START_REF] Assistance | Checklist for employers: Facilitating the hiring of people with disabilities through the use of erecruiting screening systems, including ai[END_REF]; and algorithmic assessments such as Open Roboethics Institute [START_REF] Institute | Foresight into ai ethics (faie): A toolkit for creating an ethics roadmap for your ai project[END_REF], Data Society [START_REF] Moss | Assembling accountability: algorithmic impact assessment for the public interest[END_REF], UK Statistics Authority [START_REF] Authority | National statisticians data ethics advisory committee -guidelines on using the ethics self-assessment process[END_REF], European Commission HLEG AI [START_REF]Altai -the assessment list on trustworthy artificial intelligence[END_REF], and UNICEF [START_REF] For | Children Collaborative with UNICEF[END_REF]. really applied during the design and development process of an AI system. There is a need for consistent and up-todate courses on the application of AI ethics in practice.

5.3.1.4 Weakness of the implementation of AI ethics principles in corporate governance: Corporate governance of AI have issues such as improper supervision and monitoring, deficient auditing of financial reports, decision making process issues (e.g., conflicts, divergence), and flaws in multilateral contracts. After analyzing the existing ethical AI governance, very few documents addressed the application of ethical principles to fix these issues, precisely in terms of transparency (e.g., in the auditing of documentation and financial reports, effective application of ethics guidelines and laws), responsibility (e.g., social duty, organisational), equity (e.g., in the decision-making processes with both shareholders and stakeholders, employees and managers), autonomy (e.g., free will of stakeholders), and sustainability (e.g., continuous adaptation of products/services to the market).

5.3.1.5 lack of coverage of ethics in artificial general intelligence by implementation materials: Like T. Hagendorff [START_REF] Hagendorff | The ethics of ai ethics: An evaluation of guidelines[END_REF], we found that ethics in Artificial General Intelligence (AGI) and its consequences on humanity are not discussed in implementations materials, while most ethics AI principles cover broad aspects of AGI. It is understood that we are currently at the stage of artificial narrow intelligence (weak AI) and have not yet reached artificial super intelligence [START_REF] Yampolskiy | Artificial superintelligence: a futuristic approach[END_REF], but AI is maturing fast and there is a need to anticipate potential ethical issues and consequences [START_REF] Bostrom | Existential risks[END_REF] that AI may pose in the near future (e.g., controllability, transhumanity).

Root causes of principle-implementation gaps

After analysing the implementations and gaps, the following causes of gaps have been identified.

Human bias:

The studied ethical AI implementations does not provide clear translation from the abstract ethical principles (shared by humans and the society) to ML algorithms and tools. The translation is distorted because (1) most people involved in the implementations are technical (i.e., with a computer science background) and their individual perception [START_REF] Arvan | Mental time-travel, semantic flexibility, and ai ethics[END_REF] limits the implementation, (2) there is lacking proethically designed human-computer interaction or networks of AI systems [START_REF] Morley | From what to how: an initial review of publicly available ai ethics tools, methods and research to translate principles into practices[END_REF], and (3) it is hard to translate complex human (ethical) values into design tools that are easy to use; which makes them hardly actionable [START_REF] Vakkuri | Ethically aligned design of autonomous systems: Industry viewpoint and an empirical study[END_REF].

Poor diversity within the AI community:

The diversity problem is still present in AI research and policy-making teams: (1) all races are not represented in those deciding AI groups (i.e., mainly consisting of white men), (2) there is no equity about genders in the deciding groups (i.e., mainly men and only few women), (3) no inclusion of people with disabilities, and (4) no skill balance (i.e., mainly technical people with science backgrounds). This lack of diversity makes ethical AI implementations (e.g., guidelines, laws, standards, tools) unaligned with ethical AI principles.

5.3.2.3 Complexity, interconnection of decisions, and processes that are learned from data: These implementation losses [START_REF] Koene | A governance framework for algorithmic accountability and transparency[END_REF] were addressed by the European Par-liament. Complexity expresses the behavior of common ML algorithms. For example, ML algorithm modules execute different tasks and the combination provides final results. However, each task has different meanings for data and the randomness of ML algorithms make a step-by-step explanation more challenging [START_REF] Koene | A governance framework for algorithmic accountability and transparency[END_REF]. In addition, decisions are often the result of multiple optimization (a batch of simultaneous solutions), that make explainability and interpretability difficult. Furthermore, the ML models that learnt form data are not understandable and transparent. In the context of online learning, the models are continuously updated with data; thus, the application of ethical AI principles must be continuous, or monitored to be effective. In addition, assigning responsibility to ML algorithm harms is difficult and this can lead to issues with moral responsibility [START_REF] Morley | From what to how: an initial review of publicly available ai ethics tools, methods and research to translate principles into practices[END_REF], [START_REF] Vakkuri | Ethically aligned design of autonomous systems: Industry viewpoint and an empirical study[END_REF].

Summary 5

• The 5 identified gaps are : the lack of implementation tools for some ethical principles (i.e., many ethics principles including Accountability/Traceability/Controllability are not supported, no EthicsOps), the lack of effective standards (i.e., limited work on the step-by-step application of ethics principles in the AI lifecycle), the lack of practical training (i.e., lack of practical courses on AI ethics), the weakness of Ethics in corporate governance (e.g., lack of transparency in the documentation and auditing of financial reports, lack of equity in decision making processes, lack of sustainability for services and products), and lack of coverage of ethics in artificial general intelligence by implementation materials (i.e., no discussion about this topic and its consequences [START_REF] Bostrom | Existential risks[END_REF]).

•

The 5 identified root causes are human (i.e., individual perception bias [START_REF] Arvan | Mental time-travel, semantic flexibility, and ai ethics[END_REF], team selection bias, no proethically designed human-computer interaction [START_REF] Morley | From what to how: an initial review of publicly available ai ethics tools, methods and research to translate principles into practices[END_REF], translation complexity of ethical values [START_REF] Vakkuri | Ethically aligned design of autonomous systems: Industry viewpoint and an empirical study[END_REF]), diversity (i.e., no balance about races/genders, no disability inclusion), complexity, interconnection of decisions, and processes learned from data (i.e., step-by-step explanation is hard due to the randomness of ML algorithms, multiple optimizations, and hiddenness of ML models), public-private partnerships and industry-funded research (i.e., AI democratization issue, AI research freedom [START_REF] Hagendorff | The ethics of ai ethics: An evaluation of guidelines[END_REF]), and lack of metrics for assessing the implementation of AI ethics principles [START_REF] Thomas | Reliance on metrics is a fundamental challenge for ai[END_REF], [START_REF] Eitel-Porter | Beyond the promise: implementing ethical ai[END_REF].

Issue in public-private partnerships and industry-funded research, in the field of AI:

In the studied ethical AI implementations, there are several actors such as inter-governmental partnerships (e.g., OECD, G20), international partnerships (e.g., GPAI, IEEE), national (e.g., Montreal Declaration for a Responsible AI), and universities (e.g., Universite de Montreal, Eindhoven University, ETH Zurich). Most universities are funded by public-private partnerships that limits their freedom in AI research [START_REF] Hagendorff | The ethics of ai ethics: An evaluation of guidelines[END_REF] and the democratization of AI; since some results of the research are often proprietary and the research is more focused on the needs of funding partners than on solving real problems in AI. Consequently, these issues affect the implementation of models and support tools for AI ethics.

Lack of metrics for ethical AI principles:

There is missing proper models and metrics to evaluate the application of those principles [START_REF] Thomas | Reliance on metrics is a fundamental challenge for ai[END_REF], [START_REF] Eitel-Porter | Beyond the promise: implementing ethical ai[END_REF]. Few metrics have been proposed including fairness [START_REF] Corbett-Davies | The measure and mismeasure of fairness: A critical review of fair machine learning[END_REF], privacy [START_REF] Mohassel | Secureml: A system for scalable privacy-preserving machine learning[END_REF], and robustness [START_REF] Nicolae | Adversarial robustness toolbox v1. 0.0[END_REF] but they are not enough and does not capture well the abstraction of ethical principles. In addition, some work such as [START_REF] Sablayrolles | Radioactive data: tracing through training[END_REF], [START_REF] Mora-Cantallops | Traceability for trustworthy ai: A review of models and tools[END_REF] tried to provide models for ML traceability but they are not yet effective.

GAP MITIGATION

In this section, we provide recommendations to mitigate the identified gaps (RQ4).

Recommandation 1. Several actors (from different genders, races, disabilities, skills) such as policy markers, customers, philosophists, compliance specialists, security experts, software architects and developers must be involved in the design and implementation of the ethics tools, not only technical people (i.e., security experts, software architects, developers), to ensure an effective implementation and application of ethical principles. Some researches have been done to define various metrics for fairness [START_REF] Corbett-Davies | The measure and mismeasure of fairness: A critical review of fair machine learning[END_REF], [START_REF] Mehrabi | A survey on bias and fairness in machine learning[END_REF], discrimination and bias [START_REF] Mehrabi | A survey on bias and fairness in machine learning[END_REF], privacy [START_REF] Mohassel | Secureml: A system for scalable privacy-preserving machine learning[END_REF], [START_REF] Abadi | Deep learning with differential privacy[END_REF]- [START_REF] Iyengar | Transforming data to satisfy privacy constraints[END_REF], security [START_REF] Mohassel | Secureml: A system for scalable privacy-preserving machine learning[END_REF], [START_REF] Bonawitz | Practical secure aggregation for privacy-preserving machine learning[END_REF], [START_REF] Barreno | The security of machine learning[END_REF], [START_REF] Papernot | Towards the science of security and privacy in machine learning[END_REF], robustness [START_REF] Nicolae | Adversarial robustness toolbox v1. 0.0[END_REF], [START_REF] Carlini | Towards evaluating the robustness of neural networks[END_REF], [START_REF] Brendel | Decision-based adversarial attacks: Reliable attacks against black-box machine learning models[END_REF], [START_REF] Brendel | Decision-based adversarial attacks: Reliable attacks against black-box machine learning models[END_REF]- [START_REF] Rauber | Foolbox: A python toolbox to benchmark the robustness of machine learning models[END_REF], explainability [START_REF] Ribeiro | why should i trust you?" explaining the predictions of any classifier[END_REF]- [START_REF] Carvalho | Machine learning interpretability: A survey on methods and metrics[END_REF], interpretability [START_REF] Ribeiro | why should i trust you?" explaining the predictions of any classifier[END_REF]- [START_REF] Carvalho | Machine learning interpretability: A survey on methods and metrics[END_REF], transparency and accountability [START_REF] Ribeiro | why should i trust you?" explaining the predictions of any classifier[END_REF]- [START_REF] Mitchell | Model cards for model reporting[END_REF], traceability and auditability [START_REF] Weng | Deepchain: Auditable and privacy-preserving deep learning with blockchain-based incentive[END_REF]- [START_REF] Mora-Cantallops | Traceability for trustworthy ai: A review of models and tools[END_REF]. However, more metrics need to be developed to evaluate whether AI systems are ethically-aligned.

Recommandation 2. Teams must aligned projects with EthicsOps to ensure that solutions built follow ethics principles. EthicsOps ensures the continuous application and control of ethics principles during the project lifecycle (planning, execution, delivery). During the planning phase, business requirement specifications must follow existing ethics laws, standards, and guidelines (see Section 5.2). The internal ethics policies of the organization must be updated continuously, since ethical standards and guidelines evolve, as more research results are produced (e.g., models, techniques) in the AI ethics field. In addition, team selection and change management must take into account diversity, inclusiveness, non-discrimination, and equity principles (i.e., equity irrespective of the gender, race, and disability).

Ethical controls using checklists, Algorithmic Impact Assessment (AIA), Key Ethics Indicators (KEI) based on ethics metrics [START_REF] Corbett-Davies | The measure and mismeasure of fairness: A critical review of fair machine learning[END_REF], [START_REF] Nicolae | Adversarial robustness toolbox v1. 0.0[END_REF], [START_REF] Mehrabi | A survey on bias and fairness in machine learning[END_REF]- [START_REF] Bonawitz | Practical secure aggregation for privacy-preserving machine learning[END_REF], [START_REF] Barreno | The security of machine learning[END_REF], [START_REF] Carlini | Towards evaluating the robustness of neural networks[END_REF], [START_REF] Ribeiro | why should i trust you?" explaining the predictions of any classifier[END_REF], [START_REF] Lundberg | A unified approach to interpreting model predictions[END_REF], [START_REF] Weng | Deepchain: Auditable and privacy-preserving deep learning with blockchain-based incentive[END_REF] must be also enforced during the planning, implementation, and delivery of AI projects. During the execution and delivery phases, ethics by design practices [START_REF] Commission | Ethics by design and ethics of use approaches for artificial intelligence[END_REF] help to proactively (by using the principles as requirements) prevent ethical issues from arising. For example, Transparency by design and Fairness by design ensure that outcomes of the project and the project itself are transparent, fair, and auditable by independent third parties (e.g., XAI from EthicalML). In an IT project, EthicsOps can be coupled with existing practices (e.g., DevSecOps [START_REF] Myrbakken | Devsecops: a multivocal literature review[END_REF], AIOps [START_REF] Dang | Aiops: real-world challenges and research innovations[END_REF]) to design, build, deploy, and monitor ethically-aligned AI systems.

Recommandation 3. More partnerships between publicprivate sectors and universities must be created to accelerate research, development, and knowledge transfer in AI ethics. For example, Facebook (now Meta) partners with the Technical University of Munich (TUM) to support the creation of an independent AI ethics research center with an investment of 7.5 millions USD [145]. In Canada, the Quebec and federal governments invested together an amount of 15 millions CAD in total for the creation of the International Centre of Expertise in Montreal on Artificial Intelligence (CEIMIA) 9 to foster research and innovation on responsible AI based on ethical principles, human rights, inclusion, diversity, innovation, and economic growth. In addition, research agreements between universities and companies must be clear about time investment (e.g., 50/50), confidentiality, ownership, commercialization rights, retained rights, compliance with laws, publications, warranties, indemnification and liability, and termination.

Summary 6

To mitigate the gap between principles and implementations, the solutions include:

• Inclusiveness and diversity of team roles (races, genders, disabilities, skills).

• Education and awareness on ethical values, culture, methods, and practices [146] to foster an ethical mindset (see Summary 4).

•

Increase of partnerships between private-public sectors and universities to accelerate research on AI ethics.

• Application of existing ethics laws and standards during the AI governance and engineering process while staying up-to-date with new changes (see Summary 4).

• Application of EthicsOps in combination with existing practices (e.g., DevSecOps [START_REF] Myrbakken | Devsecops: a multivocal literature review[END_REF], AIOps [START_REF] Dang | Aiops: real-world challenges and research innovations[END_REF]) to design, build, deploy, and monitor ethically-aligned AI systems from staging to the production environment.

Recommandation 4. More programs on human-centered research, and AI ethics must be included in universities and engineering schools, to instill and foster the adoption of ethical values, culture, methods, and practices [146] by future scientists and engineers. For example, universities such as the University of Montreal, Massachusetts Institute of Technology, Seattle University, and University of Michigan proposed some courses on AI ethics (see Table 14). Scientists and engineers need to know AI ethics in practice and work in an inclusive/diverse environment to build effective and sustainable tools to prevent AI systems from harming. Existing Standards (presented in Summary 4) must be followed during the AI engineering process and these standards must be continuously updated. National and international organizations need to stay abreast of new changes (i.e., new regulatory norms, standard updates). 9. https://ceimia.org/en/

THREATS TO VALIDITY

In this work, we have analyzed 350 reports and websites, and retained 100 reports containing AI principles and AI implementations. The 250 remaining that did not contained AI principles or implementations were ignored. This process was done manually and we may have missed information containing AI principles or implementations.

In addition, the extraction of principles and implementations was done manually with basic functions of Microsoft Excel. It took 2 months to search, read, identify, extract, and record relevant information from the 100 reports and websites. Nevertheless, we have verified document sources and the online publishers to ensure that they are trustworthy. During the extraction process, we have repeated statistics 2 times to avoid any miscalculation.

CONCLUSION

In this work, we have analyzed ethical AI principles and their implementations as well as the principleimplementation gaps. We have also proposed some recommendations to mitigate the gaps. Results show that Transparency is the most cited principle among the studied countries. The 11 global principles that cover most continents with a high number of occurrences are Privacy, Transparency, Fairness, Security, Safety, Responsibility, Accountability, Explainability, Well-being, Human Rights, and Inclusiveness. The ethical AI principles published by GPAI, OECD, and G20 are missing the Responsibility principle, compared to the global principles. United States, United Kingdom, and Canada released the highest number of principles between 2016 and 2021. After reviewing more than 121 ethical implementations, 5 principle-implementation gaps were identified including lack of effective implementation tools, lack of effective standards, and weakness of the implementation of AI ethics principles in corporate governance. In addition, 5 root causes of gaps were also found including human bias, lack of metrics for assessing the implementation of ethical principles, poor diversity and inclusion. Recommendations for gap mitigation include: inclusiveness and diversity of team roles, partnerships between public-private sectors and universities for fostering AI ethics research, and the use of EthicsOps. In the future, we plan to develop some ethical tools and propose practical assessment mechanisms, to bridge the gap between ethical AI principles and their implementations.

A c c o

  u n ta b il it y A c c u r a c y A u to n o m y B e n e fi c e n c e B ia s C o n te s ta b il it y C o n tr o ll a b il it y D e m o c r a c y E q u it y E x p la in a b il it y F a ir n e s s G o v e r n a n c e H u m a n -c e n tr ic H u m a n D ig n it y H u m a n O v e r s ig h t H u m a n R ig h ts In c lu s iv e n e s s In te g r it y In te r p r e ta b il it y J u s ti c e N o n -d is c r im in a ti o n N o n -m a le fi c e n c e P r iv a c y R e li a b il it y R e s p o n s ib il it y R o b u s tn e s s S a fe ty S e c u r it y S o li d a r it y S u s ta in a b il it y T r a n s p a r e n c y T r u s tw o r th y W e ll -b e in g

  A c c u ra c y A u to n o m y B e n e fi c e n c e B ia s C o n te s ta b il it y C o n tr o ll a b il it y D e m o c ra c y E q u it y E x p la in a b il it y F a ir n e s s G o v e rn a n c e H u m a n -c e n tr ic H u m a n D ig n it y H u m a n O v e rs ig h t H u m a n R ig h ts In c lu s iv e n e s s In te g ri ty In te rp re ta b il it y J u s ti c e N o n -d is c ri m in a ti o n N o n -m a le fi c e n c e P ri v a c y R e li a b il it y R e s p o n s ib il it y R o b u s tn e s s S a fe ty S e c u ri ty S o li d a ri ty S u s ta in a b il it y T ra n s p a re n c y T ru s tw o r th y W e ll -b e in g Europe Asia Oceania

TABLE 2 :

 2 Number of AI ethics implementations

	AI Ethics Implementation	Count
	Guidelines	26
	Regulations & Laws	20
	Standards	8
	Governance	7
	Checklists	6
	Algorithmic Assessment	16
	Software	28
	Training	10
	Total	121

Table 2 ,

 2 the extracted AI ethics implementations are grouped into 8 categories: guidelines, regulations & laws, standards, governance, checklists, assessment, tools, and training. Categories such as guidelines, regulations & laws, standards, and governance are more abstract and they provide orientations and rules to follow for AI ethics implementation. Other categories like checklists, assessment, and tools are more technical and they provide practical solutions for AI ethics implementation. The last category (i.e., training) contains courses about communicating and learning AI ethics.

TABLE 3 :

 3 Number of principles per country

	Country	Total Number of principles
	Canada	29
	United States (US)	167
	United Kingdom (UK)	53
	China	27
	Australia	12
	Singapore	14
	United Arab Emirates (UAE)	4
	Spain	5
	Switzerland	10
	Chile	4
	Colombia	10
	Sweden	9
	Belgium	12
	Israel	6
	Amsterdam	5
	New Zeland	11
	India	5
	Germany	28
	Hong Kong	3
	South Korea	19
	South Africa	6
	Finland	10
	Japan	25
	France	14
	Netherlands	3
	Norway	7
	Russia	10
	Ireland	9
	Italy	4
	Total	521
	4.1.1 What are the countries that released a high number
	of principles ?
	The aim of this question is to identify countries which are
	more involved in releasing national ethical AI principles.
	In Table 3, United States (US) have the highest number of
	principles among the 29 countries. The second is United
	Kingdom (UK). Canada and Germany are respectively in
	third and fourth position among the 29 countries. Next come
	China and Japan in fifth and sixth position, respectively,
	among the 29 countries. In North America, US has released
	the highest number of principles followed by Canada.
	Countries like Mexico are not mentioned because they

TABLE 4 :

 4 Number of principles per continent

	Continent	Total Number of principles
	Europe	210
	Asia	103
	North America	201
	South America	14
	Africa	6
	Oceania	23

  [START_REF] Jobin | The global landscape of ai ethics guidelines[END_REF] is also found in the global principles. OECD AI principles do not contain Privacy and Responsibility principles; yet they appeared in 5 continents with a high number of occurrences.

	2: Number of occurrences per principle per continent
	released 5 principles: (1) Inclusive growth, sustainable de-
	velopment and well-being; (2) Human-centred values and
	fairness; (3) Transparency and explainability; (4) Robustness,
	security, and safety; and (5) Accountability. The 11 global
	principles identified in the previous question are similar to
	these principles. Well-being, Human Rights, Inclusiveness,
	and Fairness principles cover principles (1) and (2), except
	the Sustainability principle. Security, Safety, Transparency,
	Explainability, and Accountability principles cover princi-
	ples (3), (4) and (5), except the Robustness principle. The Re-
	sponsibility principle encompasses Sustainability, Robust-
	ness principles and several other principles (e.g., trust-
	worthiness, reliability, impartiality). GPAI principles do not
	contain the Responsibility principle which is important (it
	appears in 5 continents with a high number of occurrences).
	4.2.2.2 OECD: OECD 3 is an international organi-
	zation for economic studies with country members from
	all continents. OECD promotes 5 AI Principles: (1) Inclu-
	sive growth, sustainable development and well-being; (2)
	Human-centered values and fairness; (3) Transparency and
	explainability; (4) Robustness, security and safety; and (5)
	Accountability. From the identified global principles, (1) is
	covered by Inclusiveness, Well-being, and Responsibility
	principles. Principle (2) is covered by Human Rights and
	Fairness. (3) and (4) are covered by Transparency, Security,
	Safety, Responsibility, and Explainability principles. Princi-

3. https://oecd.ai/en/ ple

TABLE 5 :

 5 country AI Readiness score(2021)

	Country	Government	Technology	Data& Infra.
	US	88.46	83.31	92.71
	Singapore	94.88	66.69	85.8
	UK	85.69	67.26	90.81
	Finland	88.45	63.85	85.4
	Netherlands	80.42	66.17	88.92
	Sweden	80.76	67.37	86.36
	Canada	84.36	63.75	85.08
	Germany	78.04	67.68	86.07
	Denmark	83.5	63.24	84.14
	Republic of Korea	85.27	58.49	85.89
	France	82.1	60.61	86.53
	Japan	81.9	59.31	87.32
	Norway	84.24	59.25	84.91
	Australia	83.79	57.07	85.37
	China	83.79	61.33	78.15
	Luxembourg	82.67	50.66	86.8
	Ireland	74.7	61.11	82.59
	Taiwan	77.59	59.42	78.92
	UAE	79.41	53.33	82.05
	Israel	64.64	65.87	79.52

TABLE 6 :

 6 country AI Readiness score per year(2020)(2021)

	Country	2020	2021	Growth
	US	85.479	88.16	2.681
	Singapore	78.704	82.46	3.756
	UK	81.124	81.25	0.126
	Finland	79.238	79.23	-0.008
	Netherlands	75.297	78.51	3.213
	Sweden	78.772	78.16	-0.612
	Canada	73.158	77.73	4.572
	Germany	78.974	77.26	-1.714
	Denmark	75.618	76.96	1.342
	Republic of Korea	77.695	76.55	-1.145
	France	73.767	76.41	2.643
	Japan	73.303	76.18	2.877
	Norway	74.43	76.14	1.71
	Australia	73.577	75.41	1.833
	China	69.08	74.42	5.34
	Luxembourg	72.616	73.37	0.754
	UAE	72.395	71.6	-0.795
	Israel	68.825	70.01	1.185

TABLE 7 :

 7 AI ethics guidelines

	Name	Author	Organization	Year	Location
	The Ethics of AI Ethics -An Evaluation of Guidelines	T. Hagendorff	University of T übingen		Germany
	Understanding artificial intelligence ethics and safety	D. Leslie	The Alan Turing Institute		US
	The global landscape of AI ethics guidelines	A. Jobin et al.	ETH Zurich		Switzerland
	Ethics Guidelines for Trustworthy AI	AI HLEG	European Commission		EU
	From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices	J. Morley et al.	Oxford Internet Institute		UK
	From Principles to Practice: An interdisciplinary framework to operationalise AI ethics	S. Hallensleben C. Hustedt	AIEI Group		Germany
	Building Ethics into AI: Lessons Learned from Pioneers in the Trenches	K. Baxter	Salesforce Research		US
	Technical and Organizational Best Practices		FBPML		Netherlands
	Advancing AI ethics beyond compliance From principles to practice	B. C. Goehring et al.	IBM		US
	Guidelines for AI Procurement		World Economic Forum		Switzerland
	Guidance on AI and data protection		Information Commissioner Office		UK
	Guidance on the AI auditing framework		Information Commissioner Office		UK
	Building Ethics into Privacy Frameworks for Big Data and AI		IAPP		US
	Ethical by Design: Ethics Best Practices for Natural Language Processing	J.L. Leidner	Thomson Reuters		UK
	Ethical OS Toolkit		IFTF & Omidyar Network		US
	Data preprocessing techniques for classification without discrimination	F. Kamiran et al.	Eindhoven University of Technology		Netherlands
	Responsible Design Assistant		Responsible AI institute		US
	AI RFX Procurement Framework		Institute for Ethical AI & ML		UK
	Ethically Aligned Design v1 and v2		IEEE		US
	A Toolkit for Centering Racial Equity Throughout Data Integration	A. H. Nelson et al.	AISP		US
	Design Ethically Toolkit	Kat Zhou	University of Cambridge		UK
	Responsible AI -Key Themes, Concerns &
	Recommendations for European Research and Innovation Summary of Consultation with	S. Taylor et al.	HUB4NGI		EU
	Multidisciplinary Experts
	Ethical Toolkit for Engineering/Design Practice	S. Vallor	Markkula Center		US
	Tools and Ethics for Applied Behavioural Insights: The BASIC Toolkit		OECD		France
	AI4People-An Ethical Framework for a Good
	AI Society: Opportunities, Risks, Principles,	L. Floridi	Atomium-EISMD		EU
	and Recommendations
	Ethics Canvas		ADAPT Centre & Trinity College		Ireland

TABLE 8 :

 8 AI ethics Regulation & Laws

	Name	Country	Year
	Privacy Act	Canada	1985
	The Personal Information Protection and Electronic Documents Act (PIPEDA)	Canada	2000
	Financial Consumer Agency of Canada Act	Canada	2001
	Health Insurance Portability and Accountability Act (HIPAA)	US	1996
	Privacy Act	US	1974
	Privacy Protection Act	US	1980
	California Consumer Privacy Act (CCPA)	US	2018
	EU-U.S. and Swiss-U.S. Privacy Shield	US	2016
	Gramm-Leach-Bliley Act	US	1999
	Fair Credit Reporting Act	US	2018
	Data Protection Act	UK	2018
	General Data Protection Regulation (GDPR)	EU	2016
	Privacy Act	Australia	1988
	Personal Information Protection Law (PIPL)	China	2021
	Cyber Security Law (CSL)	China	2016
	Personal Data Protection Act	Singapore	2012
	Protection from Online Falsehoods and Manipulation Act (POFMA)	Singapore	2019
	Protection of Personal Information Act	South Africa	2013
	Data Protection Law	Dubai	2021
	Bundesdatenschutzgesetz (BDSG)	Germany	2009

Table 9

 9

	presents 8 ethical AI
	standards ranging from 2017 to now and classified by
	name, organization, and year of publication. The Open
	Community for Ethics of Autonomous and Intelligent Sys-
	tems (OCEANIS) [51] is a global AI standards repository
	that captures AI and Autonomous and Intelligent Systems
	standards from different organizations such as the British
	Standards Institution (e.g., BS 8611:2016 [52]), CIO Strategy
	Council (e.g., CAN/CIOSC 101:2019 [53]), IEEE Standard
	Association (e.g., IEEE 1232.3-2014

TABLE 9 :

 9 AI ethics Standards

	Name

TABLE 10 :

 10 AI ethics algorithmic assessment tools

	Name	Organization
	AI Ethics Assessment Toolkit	Open Roboethics Institute
	Algorithmic Impact Assessment	Canada
	AI Impact Assessment	Platform for the Information Society
	Ethics & Algorithms Toolkit	San Francisco City
	Algorithmic Impact Assessments:
	A Practical Framework for Public	AI Now
	Agency Accountability
	Algorithmic Accountability Policy Toolkit	AI Now
	Algorithmic Equity Toolkit	ACLU of Washington
	Ethical Self-assessment Tool	Centre for Applied Data Ethics
	AI System Ethics Self-Assessment Tool	Smart Dubai
	Ethical Assessment	Data for Children Collaborative with UNICEF
	Algorithmic Impact Assessment for the Public Interest	Data Society
	The Assessment List on	European Commission
	Trustworthy AI	HLEG AI
	Ethics Self-Assessment Tool	UK Statistics Authority
	AI System Ethics Self-Assessment Tool	University of Colorado Law School Legal Studies Research
	Fair, Transparent, and Accountable
	Algorithmic Decision-Making	L. Bruno et al.
	Processes
	Self-Reflection Tool	Responsible Research and Innovation

TABLE 11 :

 11 Table 10 shows 16 ethical AI algorithm assessments found in the literature, classified by name and description. The Open Roboethics AI ethics governance

	Name

TABLE 12 :

 12 AI ethics checklists

	Name

TABLE 13 :

 13 AI ethics softwares The Seattle University course on AI ethics for business [106] instructs students on ethical issues around the AI technologies and then provides possible mitigations to them. The course Data Ethics, AI and Responsible Innovation [107] is also given by the University of Edinburgh. It helps to understand societal/political/legal/ethical issues

	Name	Description
	Aequitas	Bias & Fairness Audit Toolkit
	Deon	Checklist for data science projects
	eXplainability Toolbox	Algorithmic bias and explainability
	FAT Forensics	evaluating Fairness, Accountability and Transparency of AI systems
	AI Fairness 360	An Open Source Toolkit for detection and mitigation of bias in ML models
		An Open Source Toolkit for
	AI Explainability 360	interpretability and explainability of
		datasets and ML models
	AI FactSheets 360	Trust in AI by increasing trans-parency and enabling governance
	Fairlearn	Toolkit for assessing and improving fairness in AI
	Algofairness	BlackBox Auditing Tool
	FairSight	Fair Decision Making Tool
	PAIR Facets	Analyzing ML datasets by visualization
	What-if Tool	Playing with AI Fairness
	ML-fairness-gym	A Tool for Exploring Long-Term Impacts of ML Systems
	InterpretML	Explain blackbox ML
	LinkedIn Fairness Toolkit	Measurement of fairness in large scale ML workflows
	Audit AI	Detect demographic differences in the output of ML models
	SHAP	SHapley Additive exPlanations
	LIME	Local Interpretable Model-agnostic Explanations
	TF Fairness Indicators	Easy computation of fairness metrics for binary and multi-class classifiers
	TF Model Analyzer	Performing model evaluation
		Reduces or eliminates user
	TF Model Remediation	harm resulting from underlying
		performance biases
	Skater	Model Interpretation
	UnBias Fairness Toolkit	Critical and civic thinking tool
	REVISE	A Tool for Measuring and Mitigating Bias in Visual Datasets
	ART	Adversarial Robustness Toolbox
		ML Privacy using Federated
	PySyft	Learning,Differential Privacy, and
		Encrypted Computation
	TF Privacy	Training ML models with Privacy for training data
	DVC	Versioning and control of data and models
	5.2.6 AI Ethics Awareness
	Table 14 shows 10 ethical AI awareness training found in
	the literature, classified by name and organization. Several
	programs are led by universities and companies to train
	practitioners, researchers, and citizens on ethical AI prin-
	ciples, guidelines, and applications. The University of Mon-
	treal offers a Massive Open Online Course (MOOC) on Bias
	and Discrimination in AI [105]: types of bias, fairness met-
	rics, public laws and policies, technical/institutional bias
	and discrimination mitigation. The University of Helsinki
	also provides a course on Ethics of AI; it defines ethics
	and its principles such as non-maleficence, accountability,
	transparency, human rights, and fairness.

TABLE 14 :

 14 AI ethics -Online training

	Name	Organization
	Data Science Ethics	University of Michigan
	Secure & Private AI	Udacity
	Ethics in AI and Data Science	Linux Foundation
	Ethics of AI	University of Helsinki
	AI Ethics for Business	Seattle University
	Bias and Discrimination in AI	Université de Montréal
	Practical Data Ethics	Fast AI
	Data Ethics, AI and Responsible Innovation	University of Edinburgh
	Ethics for Engineers: Artificial Intelligence	MIT
	Explainable AI: Scene Classification and GradCam Visualization	Coursera
	around data lifecycle and ethical concepts (e.g., responsibil-
	ity, data governance) and it also helps to identify, assess,
	evaluate ethics issues in data science and industry. Mas-
	sachusetts Institute of Technology (MIT) also provides a
	course titled Ethics for Engineers: Artificial Intelligence [108].
	The course help engineers to understand ethics in different
	cases such as AI, Social media and data, Engineering and
	society broadly, Engineering and politics broadly, Engineer-
	ing and democracy. Other courses are provided by training
	companies such as Coursera, Udacity, and Fast AI to help
	implement ethical AI into AI projects (e.g., secure and
	private AI, explainable AI).

of practical training courses on AI ethics:

  Universities and training companies proposed generic training courses on what are AI ethics, guidelines, standards, and very few on how AI ethics is (and should be)

	•	The existing ethics tools include Aequitas, Deon,
		SHAP, LIME, AI FactSheets 360, AI Fairness
		360, AI Explainability 360, PAIR Facets, What-
		If Tool, InterpretML, PySyft, Adversarial Robust-
		ness Toolbox, Data Version Control, Tensorflow
		Privacy.
	•	The list of institutions currently providing ethical
		education and awareness materials include : Uni-
		versite de Montreal (course: Bias and Discrimina-
		tion in AI), Seattle University (course: AI ethics for
		business), University of Edinburgh (course: Data
		Ethics, AI and Responsible Innovation), and MIT
		(course: Ethics for Engineers: Artificial Intelligence)
		5.3.1.2 Lack of effective standards: There are few
	ethical AI standards [51] that cover ethical principles and
	some are still at the stage of draft [109], [110]. In the
	studied ethical AI implementations, most standards address
	AI ethics in terms of governance policies with limited work
	on technical policies, i.e., step-by-step instructions to apply
	ethical principles (e.g., transparency, traceability, account-
	ability) during the AI project's lifecycle (i.e., planning, im-
	plementation, delivery, and monitoring).
		5.3.1.3 Lack

ACKNOWLEDGMENT

This work is partly funded by the Fonds de Recherche du Québec (FRQ), Natural Sciences and Engineering Research Council of Canada (NSERC), Canadian Institute for Advanced Research (CIFAR), and Mathematics of Information Technology and Complex Systems (MITACS).