José Marcelo A P Cestari
email: jose.cestari@pucpr.br

Eduardo De Freitas

Rocha Loures

Eduardo Alves
email: eduardo.loures@pucpr.br

Portela Santos

A Capability Model for Public Administration Interoperability

Keywords: Interoperability models, Capability model, Diagnosis, Assessment, Decision making, Public administration

Interoperability is a critical factor for public administration-related entities that operate in collaborative/cooperative environments. Thus, performing an interoperability diagnosis based on a reference model provides an organization the opportunity to know its strengths and to prioritize actions to improve its performance and maturity. In public administrations, this issue is even more emphasized due to the need of considering standards policies imposed by national or international authorities. This paper proposes a Public Administration Interoperability Capability Model (PAICM) that describes intervals of expected results regarding the capability degree of certain measurable features related to its potential interoperability. The PAICM was applied to a government information technology agency, enabling the identification of the different barriers affecting organizational performance and a stratified analysis of the potential interoperability.

Introduction

Because a company rarely conducts meaningful transactions alone, working in an integrated manner with other competitors and stakeholders is becoming a critical success factor for enterprises operating in collaborative-cooperative environments in a globalized marketplace.

As explained by [START_REF] Charalabidis | Interoperability Approaches For Enterprises And Administrations Worldwide[END_REF], "Interoperability is identified as one of the main challenges that needs to be overcome to create a unified and sustainable European Information Space".

Interoperability can be a useful tool to increase efficiency and performance because it considers the ability of systems and the business processes they support to exchange, use and share data and information (European Commission 2004). [START_REF] Daclin | Developing enterprise collaboration: a methodology to implement and improve interoperability[END_REF] presented ways to increase collaboration and efficiency with a methodology to implement and improve interoperability. Interoperability has two main dimensions (concerns and barriers) that, when analyzed together, can help to increase the level and quality of collaboration, interaction and transactions between organizations and between areas inside the same organization (Panetto et al. 2016).

The need to interoperate is not an exclusive concern of private organizations. This is because there is a need for increasing information exchange among government agencies, providing online services to citizens, and reducing the costs of public operations, all of which demand that government organizations be ready to provide an adequate interface for their users [START_REF] Pardo | E-Government Interoperability: Interaction of Policy, Management, and Technology Dimensions[END_REF]. In this context, the public administration interoperability domain arises as the ability of public agencies to work together to meet interoperability requirements with the possibility of measuring the relation between human, technological and organizational entities (e.g., resources, systems, and departments) considering interoperability aspects.

Considering that the relation between government agencies and their collaborative networks (e.g., G2G, G2C, G2B) 1 is relevant for their performance and public opinion regarding their services [START_REF] Pardo | E-Government Interoperability: Interaction of Policy, Management, and Technology Dimensions[END_REF], important questions are identified: How can interoperability be measured and assessed? Which models can be used? How can maturity 1 Government-to-Government (G2G), Government-to-Citizens (G2C), Government-to-Business (G2B) levels be defined? Which dimensions must be considered? Which are the impacts (financial, operational, strategic, legal and political) of the noncompliant interoperability?

Considering the presented issues, an interoperability diagnosis can provide an evaluation of how mature an organization is. In public administrations, this is even more emphasized because of existing standard policies edited and imposed governments and other authorities. A better understanding of the measurable aspects and their influences in the public domain, as well as an adequate methodological basis to assess the interoperation capability, represents some of the main gaps in the pertinent literature. In this sense, an adequate reference model and diagnosis approach provide a public organization the opportunity to know its strengths and weaknesses, helping to prioritize actions for improving its performance and maturity related to its collaborative network.

Using the research methodology typology proposed by [START_REF] Filippini | Operations management research: some reflections on evolution, models and empirical studies in OM[END_REF], the purpose of this paper is to present a theoretical/conceptual capability model called the Public Administration Interoperability Capability Model (PAICM). The PAICM is composed of attributes, guidelines and capability levels, describing the intervals of the capability degree of certain measurable attributes related to the interoperability domain. An attribute is a quality or feature regarded as a characteristic or inherent part of someone or something (Oxford Dictionaries 2017), the aim of the guidelines is to provide an orientation and/or a statement that can determine an action regarding the achievement or the measurement of the attribute which orients the context of that guideline and capability is the ability to perform or achieve certain actions or outcomes (ISO/IEC 2015). Such concepts will be revisited and better explored throughout the text.

After Section 2, where the motivations and adopted methodology is presented, Section 3 presents the theoretical reference and a background overview, and the PAICM is detailed in Section 4. Section 5 presents the results of applying the PAICM in an application case, followed by final considerations and conclusion.

Motivation and relevance, goal and originality

In the modern context, organizations need to rethink and adjust their systems, processes and methods in order to better operate, cooperate and integrate with the environment and their stakeholders. One way to achieve this and increase the degree of the relationship between the organizations is through modeling, implementation, execution and measurement of an interoperable business. In comparison with private companies, which aspects such as profit and competition are strongly applied, government organizations, by nature, do not have this kind of concerns. The issues may be similar, but government focus is on the welfare of citizens, reduction of costs, integration with other agencies and political aspects.

One important common characteristic for all modern organizations, whether they're public or not, is the use of Information and Communication Technologies (ICT) to improve their process performance, increase profit, productivity and reduce their costs. To [START_REF] Guédria | Contribution to Enterprise Interoperability Maturity Assessment[END_REF], progress occurs when entities communicate and share information in order to achieve, together, something that they could not achieve alone. Therefore, despite public or private organizations, machines, systems, people, business and political interoperability is becoming a key success factor in all areas. Characterization and measurement of different degrees of interoperability allow an organization to know its "as is" stage and plan the ways to achieve higher degrees ("to be") of organizational performance regarding interoperability. This scenario highlights the scientific relevance and motivation of the subject in which it is possible to mention:  Public administration interoperability leads to a better decision-making, allows better coordination of government agency services, have a foundation on a citizen-centric delivery of services and leads to cost savings and/or avoidance.

In other words, interoperability contributes to good governance (UNDP 2007).

 The official Brazilian interoperability framework (e-Ping) deals mainly with technical aspects (Ministério do Planejamento 2012).

 Interoperability will allow data compiled by different agencies to be used together to make faster and better decisions and allows governments to manage better their internal operations (UNDP 2008).

 Interoperability increases transparency and accountability. Governments are better able to justify their programs and citizens are better informed, both prerequisites for a vibrant democracy (UNDP 2008).

 The government services in Brazil were questioned (e.g., riots in June/July 2013) [START_REF] Watts | Brazil erupts in protest: more than a million on the streets[END_REF], generating an opportunity to achieve better quality, time and cost [START_REF] Cestari | Interoperability Assessment Approaches for Enterprise and Public Administration[END_REF] performance in public administration.

Considering the originality aspects, the complexity presented in the public administration context requires additional effort regarding its influencing such as legal, political, sociocultural and other issues. This scenario is particularly prominent in some emergent countries as Brazil, providing a broad field for research in the public administration interoperability domain, since government interoperability frameworks focus almost entirely (90%) only in the technical domain (CSTRANSFORM 2010). The literature analysis revealed that there are many papers regarding interoperability models and government initiatives. Nevertheless, it was not possible to identify frameworks, assessment models or models organizing and dealing with influence factors other than the technical ones. Another issue detected is that the Brazilian initiative in the area is almost entirely focused on the e-Ping framework (Ministério do Planejamento 2012). According to the literature review, in the past regarding the evaluation of public administration entities not related to (almost entirely) technical issues.

The major contributions of this research can be highlighted as the following aspects:

 Rationale, rules and procedures regarding the knowledge discovering steps of interoperability aspects in public administration interoperability.

 Adoption of a mathematical and quantitative approach (AHP) for the structure of the proposed capability model and the diagnosis process.

 A combination of quantitative and qualitative methods and tools (e.g., semantical similarity, Natural Language Processing, context analysis).

The research question can be exposed as "how a capability model and a diagnosis method of interoperability, in the public administration domain, allows measuring an entity's level of potential interoperability?" And the main goal is to Propose a framework methodology to diagnose the interoperability in a public administration scenario, including the capability model and the diagnosis method.

Methodology

Although the main purpose of this paper is to propose and test a capability model and not necessarily building a theory from a single case (or a theory at all), it is important to mention that the empirical context plays an important role within model and theory generation, testing and elaboration, as exposed in [START_REF] Ketokivi | Renaissance of case research as a scientific method[END_REF]. According to [START_REF] Carlile | The Cycles of Theory Building in Management Research[END_REF], this research can fit within the structure (Figure 1) of the theory building process, where every complete lap around the theory building structure consists of an inductive side and a deductive side.

Figure 1. Theory building based on [START_REF] Carlile | The Cycles of Theory Building in Management Research[END_REF]. Retrieved from Cestari et al. (2018).

The research is based on the elaboration of a theoretical and conceptual capability model, following the structure of a deductive process and using a case study to apply and test the generated model. As an iterative, incremental and interactive approach, the research also can evolve and works in an inductive way, using the gathered information from the empirical observations and measurements to improve and evaluate the proposed model (as illustrated in Figure 1).

Other methodology aspect is related to case study. Although this research uses an application case to evaluate and apply the proposed capability model, some characteristics of a case study are still pertinent. [START_REF] Yin | Case study research and applications: Design and methods[END_REF] exposes that the case study approach deals with qualitative and quantitative evidence, several sources of information, using empirical inquiry to investigate a phenomenon within its real-world context, considering that the frontiers between the phenomenon and the context are not clearly defined. In this context, this research adopts a descriptive and exploratory approach for its preliminary case, since it is a type of pilot study which can be used to test the main aspects of the research. At this point, the research is not looking for cause-and-effect relationships. Instead, the goal is to evaluate the proposed model and the rationale considering their feasibility, usability, and usefulness and organize and analyze aspects of the reality regarding one entity (organization).

Related works and theoretical references

Interoperability

The main dimensions of interoperability are 'barriers' and 'concerns' (Chen et al. 2006).

'Concerns' define the content of interoperation that may occur at various levels of the organization (e.g., data, service, process, business). That is, on the level/area the interoperation occurs. The 'barriers' dimension is related to interoperability obstacles considering three categories: conceptual, technological, and organizational. A barrier is a kind of 'incompatibility' or 'mismatch' that obstructs the sharing and exchanging of information.

Figure 2 shows a cross-reference example between the 'concerns' and 'barriers' dimensions. As can be observed in Figure 2, each 'concern' dimension has three types of 'barriers' (conceptual, technological and organizational). Or, from another point of view, each 'barrier' dimension has four types of 'concerns' (data, service, process and business).

Maturity and capability levels

The maturity is a measurement of the ability of an organization for continuous improvement in a particular discipline (area, practice area). The higher the maturity, the higher is the quality (or the capability) of the implementation/execution of a discipline implemented by the organization CMMI Product Team (2011). In complement, a capability is a measure of the ability of an entity (department, organization, person, system) to achieve its objectives (ISO/IEC 2015).

Maturity and/or capability models are usually organized in levels that describe the evolutionary path recommended for an organization that wants to improve its practice areas. According to CMMI Product Team (2011), a practice area (PA) is 'a cluster of related practices in an area that, when implemented collectively, satisfies a set of goals considered important for making improvement in that area.' It can be interpreted as a 'discipline' or 'knowledge group'.

Examples of PAs are 'Configuration Management', 'Measurement and Analysis', 'Project Planning' and 'Risk Management'. Maturity levels represent a staged path for an organization's performance and process improvement efforts based on predefined sets of practice areas (PAs). Within each maturity level, the predefined set of PAs also provide a path to performance improvement. Each maturity level builds on the previous ones by adding new issues or rigor. Capability levels apply to an organization's performance and process improvement achievements in individual practice areas. Within practice areas, the practices are organized into practice groups (levels) which provide an evolutionary path to performance improvement. Each level builds on the previous ones by adding new issues or rigor.

The result of a process/practice diagnosis (applied to the entire organization, small areas or projects) can position existing processes in certain maturity and/or capability levels, depending on which model path/representation is adopted. Commonly, there are two process improvement paths/representations (CMMI Product Team 2010):

(1) The maturity levels path/structure: enables organizations to incrementally improve processes corresponding to an individual practice area (or group of practice areas) selected by the organization.

(2) The capability levels path/structure: helps organizations to enhance a set of related processes by incrementally addressing successive practice areas.

In a maturity level structure, the focus is on the overall maturity of an entity, as measured by maturity levels. Maturity levels apply to an entity's improvement achievement across multiple areas. Taking CMMI® (CMMI Product Team 2010) as an example, there are five maturity levels. Each maturity level is composed of a set of practice areas, and all of them must be implemented to achieve a certain maturity level. The progression of maturity levels occurs only when all the practice areas of that level are accomplished (in a cumulative way).

In contrast, a capability level (CL) structure focuses on individual practice areas capabilities as measured by capability levels. These levels are ways for incrementally improving the activities corresponding to a given process or practice. This approach is concerned with selecting a set of practice areas to improve their performance level and the PAs may have different degrees of implementation within a company or entity, i.e., each PA may have different CLs. Considering enterprise architecture maturity models, a similar approach was made in [START_REF] Vallerand | Analysing enterprise architecture maturity models: a learning perspective[END_REF].

Interoperability Maturity Models and Frameworks

The Enterprise Interoperability Assessment (EIA) provides a company the opportunity to know its vulnerabilities, forces and prioritize actions to improve its performance and maturity [START_REF] Guédria | Maturity Model For Enterprise Interoperability -review[END_REF]. Therefore, assessing interoperability implies the establishment of measures to evaluate the degree of interoperability between organizations or areas.

Interoperability assessment is a relatively new subject, despite having been the topic of several studies by [START_REF] Chen | Framework For Manufacturing Process Interoperability CEN/ISO 11354[END_REF], [START_REF] Elmir | A Novel Approach For Periodic Assessment Of Business Process Interoperability[END_REF], [START_REF] Cornu | Customizable Interoperability Assessment Methodology To Support Technical Processes Deployment In Large Companies[END_REF], [START_REF] Yahia | Formal measures for semantic interoperability assessment in cooperative enterprise information systems[END_REF][START_REF] Yahia | Formal measures for semantic interoperability assessment in cooperative enterprise information systems[END_REF][START_REF] Rezaei | Interoperability evaluation models: A systematic review[END_REF]. The interoperability degree can be measured according to the maturity/capability model reference intended to be adopted. There are many interoperability maturity models (IMMs) that describe and graduate the degree of interoperation [START_REF] Guédria | Maturity Model For Enterprise Interoperability -review[END_REF] [START_REF] Leal | Towards a comparative analysis of interoperability assessment approaches for collaborative enterprise systems[END_REF]. Most of the IMMs present a 'CMMI-like' structure, including ratings and maturity levels. An analysis of several IMMs can be found in [START_REF] Panetto | Towards A Classification Framework For Interoperability Of Enterprise Applications[END_REF]. Some of these models can be used for various types of organizations, including public administration-related entities. For example, an organization with a low interoperability level may suggest that its performance is usually ad hoc or even inconsistent and, perhaps, isolated from other companies. An organization with a high interoperability level can be considered able to work with other enterprises in a manner that maximizes the benefits of this collaboration.

These models have a strong connection to eGovernment issues, considering it as a key strategy for improving the effectiveness of public policies and programs [START_REF] Pardo | E-Government Interoperability: Interaction of Policy, Management, and Technology Dimensions[END_REF], although approximately 90% of the frameworks focus only on the technical domain.

Some IMMs include the following: LISI (Levels of Information System Interoperability) [START_REF] Chiu | Can Level Of Information Systems Interoperability (LISI) Improve DoD C4I Systems' Interoperability?[END_REF], OIMM (Organizational Interoperability Maturity Model) [START_REF] Clark | Organisational Interoperability Maturity Model For C2[END_REF], LCIM (Levels of Conceptual Interoperability Model) [START_REF] Wang | The levels of conceptual interoperability model: applying systems engineering principles to M&S[END_REF], EIMM (Enterprise Interoperability Maturity Model) [START_REF] Berre | The ATHENA Interoperability Framework[END_REF]), Government Interoperability Maturity Matrix (GIMM) [START_REF] Sarantis | Towards standardising interoperability levels for information systems of public administrations[END_REF], MMEI (Maturity Model for Enterprise Interoperability) [START_REF] Guédria | Maturity Model For Enterprise Interoperability -review[END_REF], Information Systems Interoperability Maturity Model (ISIMM) (Van Staden and Mbale 2012), and Interoperability Maturity Model (IMM) [START_REF] Mannot | Interoperability Maturity Model -Guideline[END_REF]. Table 1 presents the existing maturity/capability levels and general characteristics of IMMs and other models, including the PAICM proposed in this study. Notably, the maturity level structure is very similar among the models, although each model has its own characteristics and proper application.

Not applicable.

In addition to the studies of IMMs, mapping the interoperability frameworks (IFs) allows for the identification of structural characteristics in the definition of 'concerns' and 'barriers', some of which are related to the public administration domain and, in some cases, already consider legal and political perspectives [START_REF] Ray | A critical survey of selected government interoperability frameworks[END_REF] Few studies address these aspects, one being the MMEI [START_REF] Guédria | Maturity Model For Enterprise Interoperability -review[END_REF], further studied by [START_REF] Leal | Towards a comparative analysis of interoperability assessment approaches for collaborative enterprise systems[END_REF]. The MMEI provides an assessment methodology based on the theory of fuzzy sets, which addresses relative scoring and uncertainty but can be difficult and complex to apply in daily operations.

In an extension of the study by [START_REF] Guédria | Interoperability Maturity Models -Survey and Comparison[END_REF], a qualitative analysis was performed with IMMs, IFs and other capability and maturity models, including CMMI®, ISO 33001 (ISO/IEC 2015), MPS.BR® (SOFTEX 2012), ISO 9001:2015 (ISO 2015) and the proposed PAICM. Attributes were grouped into major characteristics (e.g., structure, design, process approach), and a qualitative relation (or 'adherence degree') of the models is presented in Table 2. The '+++' symbol indicates that there is a strong concern and that the model meets the criteria well; the '+' denotes a weak relation and the '++' is in between. The '-' symbol indicates that the model has a very low adherence to the criteria (lower than the '+' symbol). assessment aspects, design and process approach) with subitems that help to qualify the adherence of the models to their set of characteristics. For example, the models were evaluated considering their feasibility, usability and usefulness, which are characteristics within the process approach [START_REF] Platts | A Process Approach to Researching Manufacturing Strategy[END_REF]) criteria major group.

The qualitative intervals presented in Table 2 gives a preliminary positioning of the PAICM with respect to the models belonging to the same type of application found in the literature. However, this qualitative evaluation lacks a useful ranking evaluation to enable a more accurate and discriminating positioning of the PAICM with respect to its peers. Therefore, the PROMETHEE method based on the qualitative evaluation depicted in Table 2 [START_REF] Behzadian | PROMETHEE: A Comprehensive Literature Review On Methodologies And Applications[END_REF]) was applied to rank the models. The adoption of the PROMETHEE method is appropriate for the nature of the proposed analysis because this multi-criteria evaluation method, which is more robust to parameter variations, compares alternatives based on binary relations representing the performance of the alternatives for a given criterion.

After the application of the PROMETHEE method, the resulting evaluation matrix provides an equal weighting distribution among the grouped criteria. A possible nonhomogeneous weighting in the criteria can be obtained with the application of the Mudge diagram or using AHP [START_REF] Saaty | The analytic hierarchy process-what it is and how it is used[END_REF]) [START_REF] Emrouznejad | The State Of The Art Development Of AHP (1979-2017): A Literature Review With A Social Network Analysis[END_REF]. This strategy is adequate for evaluating scenarios in specific areas of public administration; however, it does not fit appropriately with the qualitative analysis applied in this study, which is broader in its purpose (model comparison).

Figure 3 presents the results of the rankings obtained after applying the PROMETHEE II method. The proposed PAICM appears in the first position (Phi = 0.52) followed by MMEI (Phi = 0.34), which is a model with similar characteristics with respect to interoperability maturity assessments. An analysis of related studies reveals that, although there are many papers regarding interoperability models associated with public administration initiatives, it was not possible to identify specific models, frameworks or assessment procedures that did not focus almost entirely on technical aspects. In addition, the existing studies do not describe the methodology used to create and maintain the models and frameworks. The papers usually portray the ideas without a rationale ('how to') or the construction process behind them.

Capability model proposition

The The definition of capability adopted in this research is based on Princeton University (2018), CMMI Product Team (2010) and ISO/IEC (2015), and can be described as:

(i)

The measure of the ability of an entity to achieve its objectives;

(ii) The ability to perform or achieve certain actions or outcomes through a set of controllable and measurable faculties (e.g., features, functions, processes, or services) (iii) The degree of how good the implementation or achievement of some faculty is.

In complement, a capability level describes an interval of expected results within an entity (i.e., what is the capability degree of certain measurable faculties regarding an organizational entity?). In this research context, the measurable items are the attributes, which were extracted from the domain literature review using quantitative and qualitative techniques presented latter on. The PAICM is composed of four capability levels indicating the degrees of evolution and implementation of certain guidelines related to attributes, as similarly exposed in the study of [START_REF] Gottschalk | Maturity Levels For Interoperability In Digital Government[END_REF]. A basic description of the levels is presented in Table 3.

Model structure, components and capability levels

Table 3. PAICM capability levels and basic characteristics.

Level 1: Ad hoc  One or more aspects of the attributes guidelines are not executed or present in a satisfactory way.  There is very poor (or none) documentation and no management or institutionalization.

No (or only a few) processes or activities are documented.  Although interoperability within capability level 'ad hoc' can perform with a kind of success in some ways, it can be lost over time if they are not defined, managed and institutionalized.  Somewhat chaotic tasks and actions.  Success depends on the 'heroism' and competence of the people and not of consolidated processes.  Interoperability is very limited and occurs almost by 'luck'.  The guideline (or attribute) is not performed or is partially performed in the manner that its implementation is done in such a way (according to the aspects of the level) that characterizes an ad hoc implementation. Level 2: Defined  The activities and processes (related to the guidelines) are executed and documented.  Interoperability is limited and occurs mostly because of the existence of documents, processes, and formal proceedings.  There are few (or none) controls and no (or very few) management regarding the guidelines aspects.  People are trained to execute the guidelines and meet the attributes goal in an eventbased way (i.e., according to some specific need). There is no (or very few) managed training plan with a strategic approach (e.g., yearly planning, strategic needs for training).

 The guideline (or attribute) is performed or partially performed in a manner that its implementation is in such a way (according to the aspects of the level) that characterizes a defined implementation. Level 3: Managed  There are controls and formal management regarding the guidelines execution and attributes aspects.  Measurements occurs in a project and/or area/department level, but it is not yet institutionalized for all the entity or for all the attributes, guidelines or processes.  People are formally trained regarding the execution and implementation of the guidelines to increase the performance of the interoperability attributes.  The involvement of stakeholders is monitored, controlled, and reviewed.  Interoperability occurs as a management strategy (and not on an ad hoc or people dependent way). It is not yet institutionalized, but it is managed.  Decision-makers can share information between systems.  The guideline (or attribute) is performed in a manner that its implementation is in such a way (according to the aspects of the level) that characterizes a managed structure.

Level 4: Institutionalized  The processes, attributes and guidelines practices are institutionalized in the entity.  Interoperability is a strategic focus and is embedded within the entity's strategic plan.  Interoperability and the entity, in general, can adapt to changes in the business environment in an agile, flexible and responsive manner.  Support dynamic interoperability and adaptation to changes and evolution of external entities.  The entity seeks for qualitative and continuous improvements.  The guideline (or attribute) is performed in a manner that its implementation is in such a way (according to the aspects of the level) that characterizes an institutionalized structure.

To organize and facilitate the understanding and composition of the PAICM, and as a basis for the diagnosis process, the attributes are organized into cards as exemplified in Table 4 and Table 5 with respectively the 'Accessibility' and the 'Commitment' attribute. These cards are part of the model and will support the appraisers in conducting the diagnosis process.

Table 4: 'Accessibility' card.

Attribute

Accessibility

Id ACC

Description

The access to all the information needed improves the interoperability. There can be degrees of access information, according to roles and job functions. Nevertheless, once the interoperability definitions relate to the exchange and use of information, the access to information is crucial. Information must be able to be used, obtained, reached or approached.

Interoperability areas

Service x Technological (ST), Data x Technological (DT).

Work products examples

Access plan and rules, data access policy, open data available, data protection plan, data configuration management. Guidelines ST (Service x Technological) ACC-ST-Gui.01: Provide services (e.g., web services) to the users and other potential stakeholders so they can have transparent access to information. ACC-ST-Gui.02: There are policy and procedures (or equivalent documents) for accessing information within the entity (not only considering web services exchange of information but also related to the exchange of e-mail and other formal communication). ACC-ST-Gui.03: Provide accessible services to users using Internet technology (e.g., citizens and other users can perform activities using the internet). DT (Data x Technological) ACC-DT-Gui.01: There are defined protocols or formats to exchange information between databases, services or systems. ACC-DT-Gui.02: Identify all the data that can be shared, including criteria definitions criteria for access and change those data. ACC-DT-Gui.03: Implement open data principles (e.g., whenever it is possible, the data is public available and can use to build new services or tools to users).

Attribute

Commitment Id COM

Description

The state or quality of being dedicated to a cause or activity or the attitude of someone who works very hard to do or support something. The commitment aspect also deals with the term "support", including, for example, a senior management support. In a complementary view, commitment is measured by what an entity (person, organization) are willing to give up for a certain cause, belief, project or activity.

Interoperability areas

Business x Organizational (BO).

Work products examples

The strategic plan, perception, and daily actions, explicit agreed commitment (e.g., commitment to a plan or a project). Guidelines BO (Business x Organizational) COM-BO-Gui.01: There is senior management support (top-down support and involvement in the tasks). COM-BO-Gui.02: There is a clear leadership (ownership, sponsorship, management are defined, and it is known for every involved). COM-BO-Gui.03: There is an implemented sense of trust, loyalty and honesty. COM-BO-Gui.04: There is an implemented sense of motivation, responsibility, and respect among the team. COM-BO-Gui.05: Commitments are formally established when needed (e.g., commitment to a project plan, commitment to the entity, commitment with the suppliers).

The next section describes how the attributes were defined and organized.

Define and organizing the attributes

According to [START_REF] Nonaka | A dynamic theory of organizational knowledge creation[END_REF], knowledge is a justified belief that increases an entity's capacity for taking effective action. In [START_REF] Swan | Knowledge management and innovation: networks and networking[END_REF] the authors define knowledge as experience, facts, rules, assertions and concepts about their subject areas that are crucial to the business. There are two basic dimensions of knowledge: tacit and explicit [START_REF] Nonaka | A dynamic theory of organizational knowledge creation[END_REF], which can be combined with an ontological dimension. In complement, knowledge management is a process of identifying, capturing and leveraging collective knowledge to help the organization compete (Von [START_REF] Krogh | Care in knowledge Creation[END_REF]. The creation of knowledge can be achieved through (i) socialization; (ii) externalization; (iii) combination and (iv) internalization [START_REF] Nonaka | A dynamic theory of organizational knowledge creation[END_REF]. Knowledge can be represented using different formal forms (e.g., semantic nets, rules, ontologies, mind maps and conceptual maps). In summary, the steps to extract the attributes are: (i) concept identification; (ii) keyword analysis and word count as support mechanisms;

(iii) tagging words to identify groups of knowledge and (iv) matching process to search for relations among the words found and root concepts of the research.

In the interoperability domain, attributes can also be mapped to best practices, requirements or desired interoperability characteristics within public administration entities.

The connections exposed in Figure 5 represents that an 'attribute' can be an adjective that expresses some sort of quality or defines characteristics of an entity. Guidelines 'instantiates' attributes, since the guidelines provide orientation and/or statement that can determine action regarding the achievement or measurement of that attribute 'instantiated' by the guideline. For example, the first guideline (Guideline 01) of the 'Culture' attribute, mapped within the 'Business x Organizational' (BO) interoperability dimension, states the following: 'CUL-BO-Gui.01: There is the appropriate degree of liberty to the employees, respecting the divergences of opinion, religion, race, gender and belief.' Details of the 'hierarchical' organization among interoperability dimensions, attributes and guidelines are presented in the next sections.

In fact, for measuring and diagnosing capability levels aspects, the attributes contain measurable items exposed in form of guidelines. Similar to the strategy adopted in the initiatives by [START_REF] Chen | Framework For Manufacturing Process Interoperability CEN/ISO 11354[END_REF] and [START_REF] Lee | Knowledge Maps For E-Learning[END_REF], a keyword analysis was performed as a first attempt to extract an initial set of attributes based on the literature review. However, the analysis detected only general aspects of the documents and not measurable faculties that could be derived into attributes.

Once the keyword analysis did not work (there were only generical aspects regarding the papers and no useful information regarding possible attributes were found), an approach using Natural Language Processing (NLP) was applied, using a group of words extracted from the documents of the literature review as the first set of data. For further details please see Annex I.

Finally, a qualitative analysis of all the related terms that could complement the initial attributes was executed. Therefore, all the pertinent influential factors, requirements, principles, perspective or categories that could be considered an attributional aspect of the capability measurement were collected from the IMMs, IFs, and related literature.

All the retrieved information was organized into 22 groups of similar concepts. In Figure 6, the rim of the circle (bordered in bold) represents the 22 groups of concepts retrieved from the literature (related to interoperability within the public administration domain). The arrows connecting the groups on the rim to the small circles (i.e., A1 to A22) indicate the derivation of the grouped concepts into attributes (22 attributes). The dashed lines indicate that the information derived from the concept's groups is used to compose the guidelines within each attribute. The inner lines indicate a connection between the attributes and its guidelines to the proposed capability levels. Concepts that were not explicitly used to compose the attributes were considered to help with the definition of the guidelines. Figure 6. Grouping attributes (A1 to A22).

A similar approach (but adopting other techniques such as semantic modeling and data mining) were used by [START_REF] Diamantopoulos | Enhancing requirements reusability through semantic modeling and data mining techniques[END_REF] to enhance requirements reusability and by Cai et al ( 2016) using an ontology-based semantic configuration.

Attributes review

Despite the quantitative and qualitative methods adopted to extract the attributes from the literature, another evaluation was performed using practitioners and researchers as respondents of a survey. Therefore, a confirmatory survey [START_REF] Forza | Survey Research In Operations Management: A Process-Based Perspective[END_REF] was applied to test the consistency of the developed concepts and validate their boundaries.

The main goal of the survey was the evaluation of the 22 extracted attributes using a five-level rating scale: '(1) Strongly disagree', '(2) Disagree', '(3) Neither agree nor disagree', '(4) Agree' or '( 5) Strongly agree'. The sample selection of the respondents was nonprobabilistic, considering characteristics such as convenience, similarity, typical cases, the number of publications and domain experience (for both practitioners and researchers). The application of these filters generated a set of 95 respondents. A Cronbach's alpha was applied to the attributes evaluation questions to quantify the reliability [START_REF] Christmann | Robust Estimation Of Cronbach's Alpha[END_REF] of the questionnaire within an interval from 0 to 1 (1 having the highest reliability). The survey reliability was considered satisfactory because the calculated alpha value was 0.87 [START_REF] Forza | Survey Research In Operations Management: A Process-Based Perspective[END_REF].

The respondents' profiles were almost equally distributed between the engagement areas (Table 6), and the following considerations were extracted from the survey results:

(1) All the attributes have at least 84.21% of the answers positioned in a greater than/equal to the 'Neither Agree nor Disagree' option.

(2) Almost all the attributes scored at least 73% for greater than/equal to the 'Agree' option. There are only four attributes with a lower percentage.

(3) The averages are all greater than 3.5, demonstrating that the sensibility is greater than the 'Neither Agree nor Disagree' option.

(4) There is only one attribute (Accessibility) with a median value of three; 14 attributes have a median value of four; and the remaining seven attributes have a median value of five. ) was adopted using the answer scale (from one to five). The null hypothesis (H0) was defined as 3 (i.e., the respondent 'neither agrees nor disagrees'), the alternative hypothesis was defined as 'greater than' 3, and the reliability degree was defined as 0.95. All the calculated pvalues were less than 0.05, which is the maximum reference value to discard the null hypothesis.

With the results and further analysis and considerations related to the open questions (e.g., comments and suggestions), the attributes exposed in Figure 6 were validated by the respondents.

Categorize the attributes within interoperability aspects

Because all the attributes are defined and detailed, it is necessary to classify them into interoperability categories (i.e., within the 'concerns x barriers' spectrum). That is, the idea is to identify the pertinent location/position for an attribute, within the matrix generated by the 'product' of the 'Concerns x Barriers'

A two-phase approach was adopted to perform this categorization. The first phase is based on semantical similarity aspects, and the second is a context and qualitative analysis (considering the literature and pertinence to public administration interoperability). This dual approach is similar to that adopted for the attribute extraction (described in Section 3.2), applying quantitative and qualitative-oriented techniques.

As a starting point to categorize the attributes into interoperability aspects (i.e., concerns and barriers), a Semantic Similarity Service (UMBC Ebiquity Research Group 2013) was adopted. The process consists of comparing all the attributes with all the concerns to obtain a value from 0 to 1 ('1' being the highest) for semantical similarity. This rating (and further analysis) provides an initial reference for the decision of 'where' the most appropriate concern is to 'insert' each attribute.

The next step is to define an interval of distribution (based on the values of the semantical similarity) and organize the information on the relational structure and symbology of its relations and correlations inspired by the Quality Function Deployment (QFD) [START_REF] Akao | Quality Function Deployment[END_REF]. The proposed structure is called the Interoperability Attributes Correlation Matrix (IACM) and aims to extend the relational basis representation and analysis toward interoperability perspectives. Table 7 presents the interval, the type of relationship, the symbols and the weight of the relationships. After the semantical analysis, a context analysis and qualitative approach were performed to review and, if it was necessary, update the type of relationship generated by the semantical approach.

As a final step for the attributes categorization and inspired by a risk management approach (Project Management Institute 2017), a matrix was created to provide the final distribution. The product between the concerns' values and barriers' values (Table 8) assists in selecting the quadrant in which each attribute will occur and, consequently, define the characteristics that each attribute has.

Table 8. The product of concerns and barriers (according to symbology from Table 7).

Barriers Strong Moderate Weak ◊ • □ Concer ns Strong Θ Θ Θ Ο Moderate Ο Ο Ο ▲
Weak ▲ ▲ ▲ ▲

The entire operation produces a 264 cells matrix (4 concerns x 3 barriers x 22 attributes) considering all types of relationships ('weak: ▲', 'moderate: Ο', and 'strong: Θ'). In this scenario (after a qualitative review), there are 116 'weak', 85 'moderate' and 63 'strong' relationships, resulting in the final representation illustrated in Figure 7. For this research, only the cells with strong relationship ('Θ') will have their attributes evaluated during the diagnosis case.

Figure 7. Distribution of attributes within the IACM structure.

▲: weak relationship, Ο: moderate relationship, Θ: strong relationship.

Figure 7 shows the 22 attributes across the vertical columns and all 12 combinations of 'concerns x barriers' in the horizontal rows. The same attribute can be seen in more than one 'concern x barrier' spectrum, according to its 'strength' or 'level of pertinence'.

The results of the attribute distribution are used in the PAICM structure proposed in this study, characterizing the measurable components of each interoperability dimension.

Drill-down the model

A more detailed organization of the model can be found in Figure 8, which illustrates the PAICM structure with layers organized by visions or aspects depicted hierarchically, representing the organizational dimensions (the layers) in the perspective of interoperability aspects.

Figure 8. PAICM detailed structure. and the four capability levels (from 'CL-1' to 'CL-4'). In the PAICM structure, the capability levels are all at the same horizontal level; however, for space reasons (not hierarchical), Figure 8 represents the capability levels stacked. The horizontal bullets indicate that the structure of the guidelines and capability levels replicates horizontally in the same way. Because of this type of structure, the capability level assessment can be verified in each layer separately, providing granularity of information and an adherence value regarding specific dimensions or the whole entity.

The public administrations entities can benefit from the PAICM once they can be assessed based on the PAICM model (which was built considering the public administration point of view regarding its attributes). After the diagnosis, the entity can visualize its capability levels towards the attributes, and them know it strengths and weakness and build a process improvement plan to increase the capability levels of certain practices. A detailed method of how to execute the diagnosis using the PAICM is exposed in [START_REF] Cestari | A method to diagnose public administration interoperability capability levels based on Multi-Criteria Decision-Making[END_REF] but the macro steps are described in the Table 9. The next section (Section 4) describes an application case with the diagnosis results and the use of the PAICM to calculate the public administration interoperability capability levels using the Analytic Hierarchy Process (AHP) method [START_REF] Saaty | The analytic hierarchy process-what it is and how it is used[END_REF] as basis. The AHP uses a pairwise comparison to rate which of each item is 'preferred' or whether the two items have an identical rating. Relative weighting is assigned to each paired comparison according to Saaty's nine-point scale as presented in Table 10. The AHP multi-criteria decision-making analysis is appropriate for addressing the nature and complexity of the modeling and execution of the diagnosis process because the PAICM presents a structural similarity to the AHP with regards to the hierarchical modeling of the space problem related to interoperability assessments (see Figure 8).

Application case

The application case applies the defined PAICM using a diagnosis method to a public administration-related organization. A consistency ratio derived from the AHP method was adopted as a measuring reference for the quality of the interviews and to address the uncertain nature of the diagnosis.

Entity characterization

The organization used in this case study is an Information and Communication Technology (ICT) service provider with coverage in southern Brazil and other cities across the country. The company has more than 500 employees and provides different types of ICT services (e.g., software development, incident management, logical and physical installation, and support).

The ICT organization is part of the municipality as a 'secretary' and provide IT solutions of for the city. Four people with more than ten years of experience in the domain were selected for individual interviews, and the data collection and consolidation were performed using the geometric mean of the answers. The tabulated results were integrated into the Super Decisions® software [START_REF] Adams | Super Decisions (version 2.8.0)[END_REF] for the capability level calculation and ranking. The interviews followed a semi-structured questionnaire in Excel spreadsheets that were built to create the structure of a pair-wise comparison regarding a pertinent goal (AHP style). The respondents had to mark (e.g., an 'X' within the cell) the list of defined attributes, guidelines and capability levels, as exemplified in Table 11 and Table 12.

Table 11. Spreadsheet for attribute evaluation within 'Business x Conceptual'.

Attributes' evaluation (within BC)

Regarding the attributes within each "Concerns x Barriers" cell (i.e., the attributes are distributed within the interoperability structure), which is the attribute that the entity is paying more attention to? For the evaluation, consider the "as is" viewpoint, not a "to be". Compare the attributes according to the "Instructions" sheet and insert an "X", or mark with another form (e.g., coloring the cell), in the adequate cell containing the pertinent value. Within BC (Business x Conceptual): Attributes Degrees Attributes Communication 9 8 7 X 5 4 3 2 1 2 3 4 5 6 7 8 9 Conflicts Communication 9 8 7 6 5 4 3 2 X 2 3 4 5 6 7 8 9 Policy andregulations Communication 9 8 7 6 5 4 3 2 1 X 3 4 5 6 7 8 Guideline number: 03 COM-BO-Gui.03: There is an implemented sense of trust, loyalty and honesty. Regarding the above guideline, insert an "X", or mark with another form (e.g., coloring the cell), in the adequate cell containing the pertinent value considering the comparisons between the Capability Levels.

Capability Levels Degrees Capability Levels

Capability Level 1 9 8 7 6 X 4 3 2 1 2 3 4 5 6 7 8 9 Capability Level 2 Capability Level 1 9 8 7 6 5 X 3 2 X 2 3 4 5 6 7 8 9 Capability Level 3 Capability Level 1 9 8 7 6 5 4 X 2 1 X 3 4 5 6 7 8 9 Capability Level 4 Capability Level 2 9 8 7 6 5 4 3 X 1 2 X 4 5 6 7 8 9 Capability Level 3 Capability Level 2 9 8 7 6 5 4 3 X 1 2 3 4 5 6 7 8 9 Capability Level 4 Capability Level 3 9 8 7 6 5 4 3 2 X 2 X 4 5 6 7 8 9 Capability Level 4

Results and analysis

The results are presented in the form of tables and organized according to the goals and views represented by the PAICM structure and layers (as presented in Figure 8): (i) strategic capability level; (ii) management capability level; (iii) tactical capability level; and (iv) operational capability level. Due to space limitations, only a few results are presented in Table 13. The strategic view (Table 13, element '(A)') shows the results indicating that the capability levels for this application case are mostly at levels three (34.72%) and four (34.64%), with lower adherence in levels two (21.07%) and one (9.55%). The results indicate that the case has a high interoperability capability, with regulations and formal management. Interoperability occurs as a management strategy (as opposed to ad hoc or people-dependent) and some processes, attributes, and guidelines are institutionalized in the entity.

Table 13 (element '(B)') shows the capability levels regarding the management view for the 'Process' concern, with the higher adherence mostly at levels three (34.36%) and four (37.67%), and the lower adherence in levels two (19.40%) and one (8.55%).

A tactical capability level overview for the 'Data x Organizational -DO' aspect is presented in the element '(C)' of the Table 13, with the higher values at level 3 (with 36.89% adherence)

and level 4 (with 37.27% adherence). Finally, Table 13 (element '(D)') shows the results of an operational view considering the 'Policy and Regulations' attribute, within the 'Process x Conceptual' area. In this case, the adherence to Level 3 is 37.56% and to Level 4 is 39.59%, remaining the Level 2 and Level 1 respectively with 17.47% and 5.36%.

Considering the description of the levels derived from the obtained answers, it can be stated that the entity has regulations, formal management, and the institutionalization of processes, guidelines, and attributes, as defined in the PAICM. Other detected aspects are as follows:

 Decision-makers can share information.

 People are formally trained to perform their processes.

 The processes help to ensure that existing practices are retained during times of stress.

 Interoperability is embedded within the entity's strategic plan.

 Several processes are institutionalized.

 There are quantitative objectives for quality and process performance.

 There are shared values and goals with a readiness to interoperate.

 Data and applications are fully shared and can be distributed with a common interpretation regardless of the form.

A sensitivity analysis was also conducted to create independent variables to compare with the values of the alternatives. This way, it is possible to detect how changing the priorities of certain criteria can affect the capability levels.

Figure 9 shows a sensitivity analysis (what-if) for the 'Process' concern using the attribute 'Legality' (which is within 'PO') as the analyzed variable (represented by the vertical dashed line on the 0.3 value of the x-axis). In this scenario, the distribution of Capability Levels

shows Level 4 (0.329) to be higher than Level 2 (0.219). If the 'Legality' attribute has a higher degree of importance/priority in the diagnosed entity (e.g., moving the vertical dashed line from 0.3 to 0.7 at the x-axis), then Capability Level 2 has a slightly higher value than Level 4. That is, if the importance of the 'Legality' attribute increases, in comparison with the other attributes, Capability Levels 3 and 2 become more predominant. This behavior occurs because other attributes have higher capability levels than 'Legality', once this simulation scenario increases the weight/importance (from 0.3 to 0.7 at the x-axis) of an attribute that may not be so mature (in comparison with other ones). Figure 9. An example of a sensitivity analysis.

In this case, independent of the variable position, there are no changes in the major capability level. That is, Capability Level 3 (CL3 1, black line) is always the most predominant. Therefore, the variable ('Legality') has no influence on the change in the highest capability level diagnosed. Instead, more weight is given to the variable (e.g., up to 0.9). For example, when an increase in Level 2 (CL 2 2, blue line) occurs, this nearly equalizes the values with Capability Level 4 (CL 4 3, green line), which has a decreasing behavior when 'Legality' is at approximately 0.3.

Conclusions

This paper presents a Public Administration Interoperability Capability Model (PAICM) and describes its structure and the rationale adopted to compose its elements. An explanation that covers the selection and extraction of the attributes to their categorization within the interoperability and public administration domain is presented.

The PAICM structure and its components (cards, attributes, description, and capability levels) are implemented for practical use to obtain two research outcomes: the first is the collection of spreadsheets that organize and materialize the model into usable artifacts, which can then be used to collect and store information during the diagnosis process according to the AHP method. The second outcome is the AHP model, which supports the structural representation requirements of the PAICM addressing the difficult process of diagnosing potential interoperability in the public administration domain.

The diagnosis based on the PAICM reveals practical and functional aspects of the capability model, and its application provides public administration entities the opportunity to know their strengths, fragilities and prioritizes actions to improve their performance and maturity. The complexity of the subject is associated with the quantity and complication of the influential factors and government-specific potential barriers (e.g., political, language, culture, and structure). Some of the domain difficulties are related to the identification and evaluation of the attributes regarding interoperability within public administration. Other issues include the identification and evaluation of the barriers, comprehension of the influence and causal relationship of the attributes, and the high tacit knowledge and subjectivity in diagnosing the qualifying attributes.

In addition to the PAICM, this paper also presents the rationale, rules, and procedures related to knowledge discovering steps (including the attributes extraction). Considering the process approach evaluation criteria proposed by [START_REF] Platts | A Process Approach to Researching Manufacturing Strategy[END_REF], the application case helps to evaluate the proposed capability model and has shown that the PAICM is feasible (the model is replicable), usable (software, spreadsheets, and training make the PAICM applicable), and useful (a new reference model to diagnosis interoperability degrees).

Annex I

An approach using Natural Language Processing (NLP) was applied, once the keyword analysis did not work. The NLP approach began with a group of words extracted from the documents of the literature review as the first set of data. The words were extracted according to their number of characters, considering that an average English word has between 4.5 to 7 characters in length [START_REF] Pierce | An introduction to information theory. 1st arg[END_REF]. A software tool (MaxQDA®) to perform quantitative and qualitative content analysis was adopted to organize the data gathered (MaxQDA 2015). In the first round, 44,111 words were found, and after eliminating personal names, symbols, out of context words and words that appeared less than ten times, a total of 21,644 words was selected.

The quantitative analysis alone did not solve the problem of finding attributes, so it was necessary to investigate the semantic meaning of this large group of words. Therefore, a lexical linguistic categorization (e.g., verbs, adjectives, adverbs) of the words was applied focusing on gathering adjectives (see Figure 5).

Because an attribute may define (or derives into/from) adjectives and because of the high number of initial words (21,644), a Natural Language Processing Software was applied to detect the adjectives in the word database. The software is based on the research of log-linear part-of-speech taggers described in [START_REF] Toutanova | Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network[END_REF], assigning parts of speech to a noun, verb, adjective and other grammatical elements with a similar approach adopted in [START_REF] Keeling | Development Of Systematic Knowledge Management For Public Health: A Public Health Law Ontology[END_REF]. The list of 21,644 words was processed with the Stanford POS Tagger software tool (Stanford 2016) and 3,739 words were tagged as adjectives. At this point, because the set of words identified as adjectives could contain a subset of attributes, there was a need to search for a semantical connection between each identified word and the concept term that represents the domain subject (i.e., interoperability within public administration).

Figure 2 .

 2 Figure 2. Concerns x Barriers illustration.

  . It is important to mention IFs such as IDEAS (Interoperability Developments for Enterprise Application and Software) (Chen and Doumeingts 2003), AIF (Athena Interoperability Framework) (Berre et al. 2007 Ruggaber 2006), FEI (Framework for Enterprise Interoperability) (Chen and Daclin 2006), and EIF 2.0 (European Interoperability Framework for European public services) (European Commission 2010). Most of the mentioned IMMs and IFs do not provide assessments or diagnosis methods highlighting procedural aspects to address the inherent complexity of a lifecycle management assessment. They only provide general guidance (using questionnaires) to evaluate capability/maturity levels based on absolute scores, without adopting any further techniques to analyze and quantify/qualify the information gathered from uncertain and tacit environments.

Figure 3 .

 3 Figure 3. PROMETHEE II Complete Ranking.

Figure 4

 4 Figure 4 presents the structure and components of the PAICM, where the rectangular boxes

Figure 4 .

 4 Figure 4. General structure model.

Figure 5 .

 5 Figure 5. Attribute concept model derived and evolved from Cestari et al. (2014).

  Figure acronyms: B: Business, P: Process, S: Service, D: Data. BC: Business x Conceptual, BT: Business x Technological, BO: Business x Organizational. PC: Process x Conceptual, PT: Process x Technological, PO: Process x Organizational. SC: Service x Conceptual, ST: Service x Technological, SO: Service x Organizational. DC: Data x Conceptual, DT: Data x Technological, DO: Data x Organizational. AT: Attribute (AT-1...AT-n). G: Guideline (GL-1...GL-n). CL: Capability Level (CL-1...CL-4).

Table 1 .

 1 Interoperability Maturity Models

	Models	Level 1		Level 2	Level 3		Level 4	Level 5
	LISI	Isolated		Connected	Functional	Domain based	Enterprise
								based
		Manual		Homogeneou	Minimal	Shared databases.	Distributed
		gateway (CD,	s	product	functions.	Sophisticated	information.
		DVD, flash	exchange (e-	Heterogeneous	collaboration.	Simultaneous
		drives).		mails, files).	product		interactions.
						exchange.
	OIMM	Independent Ad hoc	Collaborative Combined	Unified
		Communicati	General	General		Shared	Organization
		on	via	guidelines.	frameworks.	communications.	interoperates
		personal		Basic		Some	shared	Shared	culture	daily.
		contacts.		electronic	knowledge.	influenced by the
		Limited		shared				home
		shared		information.			organization.
		purpose.
	LCIM	System		Documented	Aligned Static	Aligned Dynamic	Harmonized
		Specific Data	Data		Data		Data	Data
		Isolated		Black	box	Black box with a	White	box.	Common
		systems.		with	a	standard	Common	conceptual
		Black box.		documented	interface.	Ontology.	model. Beyond
				interface.			white box.
	EIMM	Performed		Modeled	Integrated	Interoperable	Optimizing
		Some		Defined	The process is	Dynamic.	Integrating
		collaboration.	collaboration.	formal	and	Adaptation	to	systems,
		Ad hoc.		Repetition.	frequently used.	changes	and	models	and
								evolution.	technologies.
	MMEI	Unprepared Defined	Aligned	Organized	Adapted
		No capability	Very limited.	Able to adhere	Interoperate with	Dynamic
		for		Simple	to	common	multiple	adjustments.
		interoperation	electronic	formats	or	heterogeneous	Shared domain
		.		data		standards.	partners.	ontologies.
				exchange.

Table 2 .

 2 Evaluation of the models regarding characteristics

	LISI	OIMM	LCIM	EIMM	MMEI	CMMI	MPS.BR	ISO	9001:2015	ISO	33001:2015	PAICM

  objective of this section is to propose and describe a capability model called Public

	Administration Interoperability Capability Model (PAICM), regarding its structure, levels and
	organization. Generally, PAICM defines capability levels and measurable aspects (attributes),
	including, instantiating those attributes, guidelines related to 'best practices' regarding the
	public administration domain. A capability model describes the complete set of capabilities an
	organization (or entity) requires accomplishing its goals or mission within a certain domain (in
	our research, the domain is interoperability within public administration). A capability model

is not a process, although it can suggest some practices and/or examples. The proposed model is focused more on 'what to do', than necessarily on 'how to do it'.

Table 5 :

 5 'Commitment' card.

Table 6 :

 6 Type of experience regarding interoperability In addition to the Cronbach's test, a hypothesis test regarding the entire survey and each individual answer was also applied. The one-sample Wilcoxon test (Anaene Oyeka and Ebuh 2012

	Profile	%	Average years of experience	Average years of experience
			(interoperability)	(public administration)
	Professional 31.58%	10.5	13.16
	Academic	36.84%	14	13.42
	Both	31.58%	17	15.33

Table 7 .

 7 Type of relationship, interval, weight, and symbol (regarding concerns).

	Type	of	Interval	Concerns	Barriers	Weight
	relationship			symbols	symbols
	Weak		<= 0.3333	▲	□	1
	Moderate		> 0.3333 and < 0.6666 Ο	•	3
	Strong		>= 0.6666	Θ	◊	9

Table 9 .

 9 Macro processes for the diagnosis the interoperability level.

	Stages (macro-	Activities
	processes)
	1. Plan	1.1 Request diagnosis
		1.2 Define entities
		1.3 Define objectives
		1.4 Define scope
		1.5 Define team of participants and roles
		1.6 Define interviews strategy
		1.7 Define schedule and efforts
		1.8 Define logistics
		1.9 Consolidate the diagnosis plan
		1.10 Obtain commitment to the plan
	2. Prepare	2.1 Schedule activities
		2.2 Request necessary infrastructure
		2.3 Provide confidentiality agreements
		2.4 Cancel and communicate involved
		2.5 Train the involved in PAICM
		2.6 Train the involved in the diagnosing
		method
		2.7 Update diagnosis plan
		2.8 Obtain commitment to the plan
		2.9 Discuss next steps

Table 10 .

 10 Comparison scale.

	Scale		Inverse scale
	(comparing i	General interpretation	(comparing j
	to j)		to i)
	1	Equal importance	1
	2	Between equal and moderate importance	1/2
	3	Moderate importance	1/3
	4	Between moderate and strong importance	1/4
	5	Strong importance	1/5
	6	Between strong and very strong importance 1/6
	7	Very strong importance	1/7
	8	Between very strong and extreme importance 1/8
	9	Extreme importance	1/9

Table 12 .

 12 Spreadsheet for guideline evaluation considering the capability levels. Related to the 'commitment' attribute.

	Interoperability area: Business x Organizational (BO)
	Attribute: Commitment (COM)

Table 13 .

 13 Assessed elements and its related adherence results.

	Adherence degree (related to each
	Capability Level)

The structure proposed by the PAICM transcends current assessment-related models such as CMMI® (CMMI Product Team 2010), ISO 33001 (ISO/IEC 2015), MPS.BR (SOFTEX 2012) and other models illustrated in Table 2. The PAICM gives more granularity and perspectives based on the definition of goals and the capability model structure. In addition, the PAICM provides a composite quantitative and qualitative approach (using the AHP) not found in other 'commercially oriented' methods.

The limitations open new directions and extensions of the research toward: (i) the adoption of the PAICM on more application cases, and (ii) the diagnosis evaluation or integration with other multi-criteria decision analysis methods (e.g., ANP, Electre family, and PROMETHEE).

Disclosure statement

The authors declare no potential conflicts of interest regarding the research, authorship, and/or publication of this article.