Johan Ivar Nielsen

Saebø

Flora Nah Asah
email: florana@ulrik.uio.no

Petter Nielsen

Johan Ivar Saebø

Challenges for Health Indicators in Developing Countries: Misconceptions and Lack of Population Data

Keywords: health indicators, population-based data, health management information system, Cameroon

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

A key goal of a health system is to provide the necessary healthcare services to all those who are in need; thereby improving the health status of the population. International organisations such as WHO and United Nations, in partnership with governments of countries, have developed global initiatives to monitor performances of the health system. One concrete example is the United Nations' Sustainable Development Goals (SDGs) which target major health problems alongside other related issues, constitutes a coordinating framework for these efforts (WHO 2010). Universal Health Coverage (UHC) is one of the goals of this initiative. UHC has been defined as the ability of all people who need health services to receive them without incurring financial hardship (WHO 2010[START_REF] Kieny | Universal Health coverage[END_REF]. To monitor countries' health progress and performances towards supporting UHC, a range of health determinants' indicators have been developed. These indicators are measurable and time-bound.

Examples of these health indicators include; children under 5 sleeping under insecticide Treated Nets (ITNs) and births delivered in a birth facility.

Health indicators are powerful tools used at all levels of the health system, for monitoring and evaluating and communicating information about the population's health [START_REF] Mant | Process versus outcome indicators in the assessment of quality of healthcare[END_REF][START_REF] Klazinga | Indicators without a cause. Reflections on the development and use of indicators in health care from a public health perspective[END_REF], Maniz 2003). They could be used to track how well (quality) and how far (quantity) countries' health system are performing [START_REF] Klazinga | Indicators without a cause. Reflections on the development and use of indicators in health care from a public health perspective[END_REF], Maniz 2003).

An indicator has two parts; numerator and denominator that go into the formula for calculating them [START_REF] Klazinga | Indicators without a cause. Reflections on the development and use of indicators in health care from a public health perspective[END_REF], Maniz 2003). Numerators are the things we count, i.e. infants immunized or new cases of TB. Denominators are the group with which the things we count are compared, i.e. total population or all births in a year. In the health system, indicators could also be used to measure a variety of dimensions concerning the health situation of the population; mortality, morbidity, health status.

There exist different types of indicators which could be used to monitor health progress, namely; health status, health systems, risk factors and health service coverage. Of important to this paper are health service coverage indicators.

These indicators reflect extent to which people in need actually receive the health service they need [START_REF] Tanahashi | Health service coverage and its evaluation[END_REF], WHO 2009). This group of indicators measure the effectiveness of health programs. They help service providers to understand how effective an intervention is, and whether one target group is reached more effectively than another. They also help to identify underserved areas or regions which need more attention (Boerma, et al 2014). Most UHC indicators of UHC fall in this group.

Indicators in this group use population data as denominator, as it facilitates comparison of health status or health service provision over time and space. Coverage indicators are equally specific and valid for small and large geographic entities, and for any given time interval (WHO 2009). Examples of such indicators are; immunization coverage, HIV testing coverage, and delivery in facility rate. A common challenge is that there is lack of reliable population data to calculate them.

Population data is usually not accessible for smaller areas such as communities or districts (WHO 2009[START_REF] Linard | Population Distribution, Settlement Patterns and Accessibility across Africa in 2010[END_REF]. Often, available population data is either outdated, not available for current administrative entities, not available for certain target populations (such as women of child-bearing age), or they may be duplicated sources of population data that do not necessarily provide the same figure [START_REF] Linard | Population Distribution, Settlement Patterns and Accessibility across Africa in 2010[END_REF][START_REF] Leegwater | A concise, health service coverage index for monitoring progress towards universal health coverage[END_REF][START_REF] Bharti | Measuring populations to improve vaccination coverage[END_REF]. The purpose of this paper is twofold. First, we are empirically exploring the understanding and use of health indicators by health program managers for decision-making. Second, we draw implications from this situation related to local practices and global initiatives such as UHC.

This study is derived from an ongoing implementation of an electronic system as a measure to strengthen Health Management Information system (HMIS) in Cameroon. This system is the main tool used for the management of health information nationwide. In this system, indicators are core element of data analysis, used to measure healthcare services.

Related Literature

In this section, we present relevant literature on health indicators to establish an understanding of what they are, what makes a good or poor indicator, and how they are used in health management.

Indicator(s) is developed by international organizations, reference groups and interagency groups, countries, academics, advocacy groups and others. From a performance management perspective, [START_REF] Flowers | Mini-symposium -Public health indicators[END_REF] describe indicator as a measure used to express the behavior of a system. In the public health context, indicators are specific tools for programme management, including the analysis and diagnosis of problems, and for taking correctives actions [START_REF] Sahay | The data indicator (Mis)Match: Experiences from trying to strengthen this link in the Health Information System in Tajikistan[END_REF].

Indicators are key statistical measure used to describe a situation, track progress and performance over time and to compare entities doing similar work. They could be used as a guide to decision making and set priorities [START_REF] Donabedian | Evaluating the quality of medical care[END_REF]. They help to inform policy and policy-makers and can be used to improve quality of care and promote accountability [START_REF] Donabedian | Evaluating the quality of medical care[END_REF][START_REF] Mainz | Developing Clinical indicators[END_REF].

The literature explains that the process of selecting indicators should be systematic and based on facts rather than on feelings [START_REF] Mant | Process versus outcome indicators in the assessment of quality of healthcare[END_REF]. The process should involve all stakeholders and be based on "who wants the indicator", "how it is used" and "by whom" [START_REF] Mant | Process versus outcome indicators in the assessment of quality of healthcare[END_REF][START_REF] Klazinga | Indicators without a cause. Reflections on the development and use of indicators in health care from a public health perspective[END_REF]. [START_REF] Heywood | Using information for action: A manual for health workers at facility level[END_REF] explains that the process should employ understanding, discipline, teamwork and negotiation. Therefore, an ideal indicator should: (i) be based on agreed definitions; (ii) measure what it is intended to measure; (iii) give the same results if used by different people in different places; (iv) be simple to calculate using readily available data; (v) fit local needs, capacity and culture and the decisions to be made; (vi) be highly sensitive to changes in the situation concerned; (vii) permit useful comparison; (viii) be evidence-based (Heywood, et la 2001[START_REF] Mainz | Developing Clinical indicators[END_REF][START_REF] Larson | Global health indicators: an overview[END_REF].

Indicators can be classified as: input, process, output, outcome and impact. Input indicators are the resources needed to implement work. Input indicators measure resources, both human and financial, allocated to a particular program. Process indicators measure whether planned activities did take place. For example, in antenatal care, process indicators can be antenatal 1 st [START_REF] Mainz | Developing Clinical indicators[END_REF][START_REF] Heywood | Using information for action: A manual for health workers at facility level[END_REF], WHO 2016).

Indicators can be calculated in different ways; ratios, proportions, or rate [START_REF] Worning | Policy on quality development for the medical profession in Danish[END_REF][START_REF] Mainz | Developing Clinical indicators[END_REF]. Ratios are numbers expressed in relation to another by dividing one number by the other. Here, the numerator is not part of the denominator. Proportion measures a part or amount that is part of a whole. The numerator is part of the denominator. Rate represents the frequency of an event in a specified period. In calculating a rate indicator, the numerator is the number of occurrences of an event during a period of time. The denominator is the number of person exposed to that event in the time period. These indicators are illustrated in table 1 below.

Indicator type

Research Context

The empirical setting within which the study was conducted is Cameroon. It is a lowincome country, situated in the sub-Saharan Africa (SSA) region. It has an estimated population of 20.6 million [START_REF] Chen | How have the World's poorest fared since the early 1980s?[END_REF]. In Cameroon, basic public and social amenities for the vulnerable are either absent or inadequate. Nationally, 29.7 percent of the population does not have access to safe drinking water and 66.9 percent lack adequate sanitation, resulting in regular outbreaks of cholera and other water-related diseases (UNICEF 2015). The burden of healthcare financing is born largely by households through out-of-pocket payments (OOP). The government of Cameroon spends an average of USD 61 as per capita per person on health. Out of this amount, only USD 17 paid by the state, USD 8 comes from international donors, and USD 36 is OOP (Cameroon Economic Update 2013). Over the past two decades, health indicators have remained poor, and in some cases even worsened. Cameroon is struggling with high mortality and morbidity especially in rural communities. Mortality rate for children under 5 is 148 per 1000 live births, ranking Cameroon as 18th amongst 20 countries in the world with the highest mortality rate. Only 13 percent of children under the age of five sleep under insecticide-treated nets, in a country where malaria accounts for more than 40 percent of all deaths in this age group. Maternal mortality rate is alarmingly high, 670 per 100,000 births as compared to 546 per 100,000 live births in SSA. In addition, many women and girls have limited access to, and utilization of, prevention-of-mother-to-child transmission (PMTCT) services, resulting in HIV infection transfer to children (UNICEF 2015).

The healthcare system adheres to the district health approach, organised in three levels: the operational level, corresponding to district health care; hub of all health interventions; the intermediate level which is responsible for technical support, while the central level deals with the development of health policies. Different programmes operate at all three levels, engaged in the provision of specialised services such as maternal and child health, malaria, HIV/AIDS, TB, and are supported by different donor agencies. The health system suffers from qualitative and quantitative shortage of human resources, and lack of technical and managerial expertise (Cameroon Economic Update 2013).

The health management information system (HMIS) in Cameroon is fragmented and characterised by vertical and fragmented information system and nonstandardized data collection methods. DHIS2, the electronic tool used is housed at "Cellular National d'Informations Sanitaires" (CNIS), the department responsible for the management of health information in the country.

Research Design and Methods

The study is drawn from an interpretative strand [START_REF] Walsham | Interpretive Case Studies in IS Research: Nature and Method[END_REF]. Interpretive research in Information Systems (IS) is useful as it helps researchers understand the problem in the contextual nature [START_REF] Klein | A Set of principles for conducting and evaluating interpretive field studies in Information systems[END_REF]. Data was collected by the first author using qualitative methods. It included interviews, group discussion, and a document review. A total of 2 focus discussion and 7 interviews were conducted from January and July of 2016, with Maternal and child health Programme Managers at district and national levels; data managers at district healthcare facilities, Matron in-charge of data management in wards, and Sister-incharge of health facilities. 22 health and program managers participated in the study, of which 13 were females. Ten had no access to a computer and only 6 had Internet access. The document review included annual reports, strategy documents, and program reports.

Purposive sampling technique was used to select interviewees [START_REF] Creswell | Research Design: Qualitative, Quantitative, and mixed methods approaches[END_REF]). This technique is used to achieve a homogeneous sample; that is sample of cases who share the same characteristics e.g. background or occupation. In this study, the interviewees shared a similar occupation; i.e. involved in data management at their respective facilities. An interview guide with broad themes around data management was used, focusing on understanding and use of health indicators. Permission to conduct the study was obtained from the Office of Regional Health delegate of each region and signed informed consent was obtained from each interviewee. The principle of data saturation was applied; i.e. interviews were ended when further probing were not adding new information. Permission to audio record interviews was obtained from interviewees at the start of every discussion and interviews were transcribed.

Data analysis was driven by the interpretive process and a descriptive approach with content analysis was used to analyse the data [START_REF] Elo | The Qualitative content analysis process[END_REF]. The interviews were transcribed verbatim. The interviews were read through several times to obtain a sense of the whole. The text was then divided into condensed meaning units. Open codes were used to group headings into categories to formulate a general description of the research questions. The interviews revealed themes which are interrelated. Dur-ing this process, the data, themes and topics was discussed among the authors and based on these discussions the data was revisited.

Analysis

This study is based on how nurse managers at district and regional health facilities, program managers at district and regional levels, and district managers' understand and use health indicators.

A general observation in Cameroon is that population data is either outdated or not available. In the absence of reliable census data, the National Bureau of Statistics issues population figures per province and district to the Ministry of Public Health annually. However, for areas below the districts, it provides percentage per population group and annual projected growth rate per age group that has to be calculated by those who want to use population data. Usually, healthcare managers lack the necessary numeracy skills to perform such calculations. Also, these figures are not publicly available.

Data is collected at health facilities, and reported upward through health districts to the national level. At health facilities, daily registers are used to record activities from various units of the facility; outpatient, antenatal unit, labour and delivery, immunization and in-patient. At the end of each month, data from these registers are collated manually in to the Monthly Reporting Activity1 (MRA) and also on Program Template forms. Thereafter, these forms are forwarded to the district office where data is captured electronically and their respective databases. MRA is captured into DHIS2 while program-specific data is captured on pre-designed Excel templates, hence forwarded to the regional level. At the regional level, data is also aggregated and synchronized to get the profile of the entire region before it is forwarded to the national level where analyses is done. We observed that the MRA is not comprehensive, as it does not contain all the data elements of health programs, as explained by one district manager:

"Previously, programs manage have their own reporting system but since the new CNIS director was appointed, he is trying to put some order in the system; starting by standardizing data collection tools. The data elements in MRA were selected based on the 100 indicator datasets as stipulated by WHO."

Data Quality -Data Review and giving feedback

It is essential to perform data quality checks. Exploring issues of data quality, we observed that data reviews and feedbacks are not standardized processes of data management. They are often done haphazardly and not frequently conducted at most facil-ities. District managers explain that due to lack of feedback, data clerks have cultivated the habit of falsifying data: "When data clerks submit data and receive no feedback, they believe nobody checks the data submitted. Thus, they have cultivated the habit of cooking data and submitting."

Despite being aware that neither data quality reviews are done nor feedbacks provided, district managers said they trust the quality of data: "I only type and submit I do not use data. I cannot say anything about the quality of data. But I trust the data submitted by the matron in-charge"

Understanding indicators: Nurse Managers at district and regional health facilities

We observed that health managers at operational levels seldom discuss about indicators and do not share the same level of understanding of indicators. In particular, facility level managers interviewed were not aware that there is a difference between an indicator and data element. They were neither able to understand what makes up an indicator nor how an indicator is calculated. They were also not able to differentiate a data element from an indicator. During the discussions, they would refer to "indicators" but when probing it became apparent that what they meant were "data elements". When asked to present a list of monthly priority indicators, a matron incharge showed a list of monthly summary data element forms, noting: "these are all our indicators that we report monthly. I write everything here on these forms and submit to them." A similar situation was observed among district program managers. When asked to give example of some of the SDGs indicators in the program she manages she noted: "Last week I went to the hospital for a visit. As I was walking out of the hospital, I caught sight of this beautiful picture with the following caption "breastfeeding is a sustainable development goal as will reduce infant mortality". That was the very first time I heard about sustainable development goals. I was so happy."

Despite giving some examples of SDGs indicators, she was still unable to name any indicators they are reporting against.

While attending a monthly district meeting, Program managers were presenting monthly reports on facility visits conducted in their sub-districts. In the course of presenting these reports, they use phrases such as "low ANC coverage, low immunization coverage" as outcome of their visits. It was not clear from the presentations how they arrived at the decisions.

Used of indicators: Nurse Managers at healthcare facilities

We also observed that health managers' use of indicators was minimal. A nurse manager explained that she uses indicators when preparing her performance-based business plan to help allocate targets (excerpt of plan presented on table below). A general complaint was the difficulties to do the calculations as illustrated in table 2 below. For example, column 2 depicts how population data are presented to managers at health facilities. "At the beginning of the year we have priority activities and also set which are reviewed annually. For example I know in this hospital, our consultation at the end of last year was 40,000. For this year, we should move up to about 45,000. If that is not achieved, we will know that we have not performed well and need to do something so that we can achieve more consultations. Two years ago, our consultations moved from 20,000 to 30,000. As a result of this increase, the director saw that the consultation area has become small. He expanded the waiting area, bought more chairs so that patients should sit down while waiting to be seen by a doctor."

Indicators

In the explanation above, the manager talked of "indicators" but what she is referring to are "data elements". She added that to calculate indicators, they would use the performance of the previous year or quarter to evaluate their facility's performance against set targets. To set indicator targets, it was observed that there are no clearly defined criteria to use. With reference to the table 2 above, Columns 3 and 4, the nurse manager from another facility shared her experience:

"As we prepare the business plan for PBF, we have working sessions with the PBF team. They will say for example, on the indicator: "Outpatient consultations (new cases): nurse", since we achieved 4025 in the last quarter, what do you think if we shift your objective for next quarter to 660 that is 4025 +660= 4685. What do we do, we just have to accept? They do not take into consideration any other factor. It is our responsibility to ensure that the number cases are achieved at the end of the quarter. If we do not achieve it, we shall not benefit a single franc on this particular indicator."

Another facility manager added:

"To set a target, we merely take the performance for the previous quarter, and add at least 5 percent. Five percentages is just an amount we decide to use. The idea is to encourage them to perform better than the previous quarter. However, there is a challenge because most often those targets sets are not achieved."

5.4

Use

Discussion

The study provides tangible examples of health managers' understanding and use of indicators in healthcare settings. So how transferable are the findings? We found that these behavior were similar to other countries reported by [START_REF] Silvia | Usage of quality indicators in hospital nursing in services in Brazil[END_REF][START_REF] Thaizy | The importance of using quality indicators in nursing care[END_REF] At the operational level, data to generate these indicators come from various health interventions done at district and communities levels. Therefore, it is important that healthcare professionals understand indicators and also know how they are calculated, as it might help them to monitor and evaluate their performances, and in the event of an outbreak, they intervene immediately.

While managers are should understand health indicators, it is important the government provides reliable population-based data. In Cameroon, population-data is available for the regions, and for administrative units below the district, projected population estimates are provided instead, and it is the responsibility of the district or facility managers to calculate these estimates, who in most cases lack the necessary numeracy skills.

A reliable source of population data is through census, but census is not frequently conducted. Data available is either outdated or does not exist in administrative unit below the district. However, there are other reliable sources of population-data. Citing an example from the developed countries, in Norway, the civil registration and vital statistics (CRVS) systems are reliable sources of population data. The government has introduced a method whereby these systems are maintained through incentives given to bother public and private entities and citizens to engage in proper and timely registration of vital events [START_REF] Nielsen | Civil Registration and Vital Statistics: A Neglected Area in Information Systems Research[END_REF]. Thus, the saying goes "population-data is the true denominator for development" [START_REF] Purcell | The Importance of Population Denominators for High-Impact Public Health for Marginalized Populations JMIR Public[END_REF].

Population-based provide an important piece of mosaic of evidence for decisionmakers as well as healthcare providers. It can be used to assess the magnitude of health problems of which population are most vulnerable, such as to track and evaluate the effectiveness of health intervention for UHC.

Furthermore, health care providers dealing with indicators need to understand their meanings and what goes into calculating it [START_REF] Sahay | The data indicator (Mis)Match: Experiences from trying to strengthen this link in the Health Information System in Tajikistan[END_REF]. In addition, indicators should be standardized as it improves data quality (Maniz 2003, Flower, et al 2005).

Conclusion

The study found that healthcare managers have difficulties understanding data elements and indicators. In the absence of population data, they use alternative method to calculate indicators. Although managers have priority indicators they report on monthly, coverage indicators are not calculated using population as denominator. This study extends research on how health information is used by healthcare providers based on a case study in Cameroon. Though this study was limited to data and program managers at district and regional levels of the health systems, the findings raise issues that have wider applicability in the implementation of electronic health information system as well as how health indicators such as UHC goals are calculated.

The primary concerns of case study research centers around validity and reliability [START_REF] Merriam | The case study in educational research: a review of selected literature[END_REF]. To address these concerns; for validity, we employed the process of triangulation, i.e. the use of a variety of data sources (interviews, focus group discussions and documents review) as opposed to relying solely upon one source. We have also included verbatim quotations [START_REF] Johnson | Examining the validity structure of qualitative research Education[END_REF]) in the analysis section, consulted with senior directors at the ministry of health. To ensure reliability, the following measures were employed; interviews were recorded and transcribed, and during data analysis themes identified were discussed and agreed among the authors, before it was included in the paper [START_REF] Roberts | Reliability and validity in research[END_REF].

visit before 20 weeks rate, antenatal client re- test rate, and post-natal visit with 2 weeks rate. Output indicators measure first level results associated to an intervention. They are defined as what we produce. Some examples include; Infant 1 st PCR test around 6 week uptake or delivery by caesarean

  section rate. Outcome indicators measure what we have achieved and should be linked to concrete goals. Examples include; delivery at facility for women under 18 years and live birth under 250 grams in facility rate. Finally, impact indicators are the cumulative effect of the overall program

Table 1 :

 1 Different types of indicatorsPopulation data is required as denominator to calculate these types of indicators. Since these are integral for health management, the lack of population data poses a challenge.

		Description	Example
	Ratio	Numerator is not included in	Ratio of male TB deaths to female
		the denominator	TB deaths
	Proportion	Numerator is contained in the	Proportion of children one year old
		denominator	immunized against measles
	Rate	Frequency of the event during	Deaths of children less than one
		a specific time in a given	year of age per 1000 live births
		population

Table 2 :

 2 Level of achievement of the objectives of the last quarter business planA matron in-charge explains how she uses indicators:

		Monthly Target Calculation	Objective for	Objective
			the previous	planned for the
			quarter (A)	quarter (B)
	Out Patient	Total pop. of catchment area /12	4025	4026 + 660
	Consultations	x 80% x 90%
	(new cases):
	Nurse
	Referral re-	Pop x 1/12 x 1%	55	55+ 18
	ceived in the
	hospital
	Cases of STIs	Total pop. of catchment area x	134	134 + 30
	treated	3% /12 x 80%
	Children com-	Total pop. of catchment area /12	223	223+ 70
	pletely vac-	x 4%
	cinated
	Normal As-	Total pop. of catchment area x	112	112 + 50
	sisted Delivery	4% /12 x 80% x 70% x 90%

of Indicators at District level

  District managers use indicators to compile quarterly reports as explained by one of them:"Programs have specific indicator targets. Take for example, the indicator "immunization coverage." I am a statistician so when data arrives my office, after capturing, I do my calculation base on the targets and population projection I have. Hence, I compare the results (figures) with what was submitted for the previous months to know when I have achieved the target or not."Another Program manager shared her experiences: "For example, as a HIV/AIDS program manager, this program is a priority program. I have monthly targets that were set and sent to me from Yaoundé [the capital in Cameroon] at the beginning of the year. Consider, for example, the indicator, "Treatment coverage among HIV/AIDS positive pregnant women". Based on the data submitted, at the end of each month, I calculate the indicator and send the report to the national office in Yaoundé."Furthermore, the district program manager explains: "If for examples the targets are not met, I will have to wait for instructions from Yaoundé before going out or planning for an intervention because Yaoundé makes all the decisions."A director at the national explained that all decisions are made at the central level, the reason why data analyses are done there; at the national level. He added that indicators are used to evaluate program, but lamented on the quality of data. For example, he mentioned that most of the indicators routinely collected for his program are irrelevant. He also added that district managers are not involved in decision-making as this is the responsibility of those at the central level:"Districts are not involved in decision-making. Except for the HIV/AIDS program, where program managers at district level have targets. Reasons being it is a priority program, with lots of international funding from PEPFAR. However, about which SDGs my program is reporting against, SDGs have not yet been made published. They are still waiting for the indicators and targets."Further, he added, indicators are mostly discussed only when preparing annual reports. This is the responsibility of the national program manager for monitoring and evaluation.

	5.5	At the national level

  al 2015. The world's political agenda for the next 15 years (2016 -2030) on health systems' strengthening is to reduce poverty through SDGs in general and UHC goals in particular. These are global initiative to monitor countries health performance. UHC are time-bound and measurable indicators to be monitored and reported against, annually by each country. Though internationally developed, most of the indicators are similar to those in a country's operational plans. Thus, decision and interventions made by policy makers and healthcare providers should be based on these indicators.

The MRA does not have certain data elements which programs have to report on. Consequently, individual program have created their own data collection tools to collect program specific data


Thanh Ngoc Nguyen
email: thanhng@ifi.uio.no

Petter Nielsen
email: pnielsen@ifi.uio.no

From Routine to Revolt: Improving Routine Health Data Quality and Relevance by Making them Public

Keywords: Information use, data quality, rapid scaling

Health Information Systems in developing countries struggle with vicious cycles of lack of information use. Substantial investment has been spent to improve the situation but results are still very limited. Adding to the body of research on strategies and solutions to break out of such cycles, this paper focuses on the effects of making routine data public through mass media and using data to fuel debates on critical health issues. Based on an action research project building a reporting system for accidents and emergencies during the Tet holiday in Vietnam, this paper discuss how making data public can have direct impact on the use and quality of health data in the health system. We discuss and draw implications related to tactics to improve the demand and use of routine health data.

Introduction

Health Information Systems (HISs) have long been recognized as an important component of public health systems because they provide vital data for effective planning and sound decision making [START_REF] Abouzahr | Health information systems: the foundations of public health[END_REF]. Although HISs play an important role in improving healthcare service delivery, they often fail, especially in developing countries [START_REF] Heeks | Information systems and developing countries: Failure, success, and local improvisations[END_REF]. Many HISs end their lifecycle in the pilot stage because they do not scale and provide useful data for managers [START_REF] Braa | Developing health information systems in developing countries: the flexible standards strategy[END_REF]. Together with the scaling challenge, developing countries are also challenged by health managers' limited use of data in decision making. Nutley, Gnassou [START_REF] Nutley | Moving data off the shelf and into action: an intervention to improve data-informed decision making in Cote d'Ivoire[END_REF] argue that turning data into action is critical for improving the health services delivery and outcomes. There are many reasons attributed to limited data use, including culture and capacity [START_REF] Braa | Integrated Health Information Architecture: Power to the Users: Design[END_REF] as well as the information needs of decision makers [START_REF] Davies | Conceptualising the information needs of senior decision makers in health[END_REF]. Another complication is the poor quality of data. Braa, Hanseth [START_REF] Braa | Developing health information systems in developing countries: the flexible standards strategy[END_REF] use the notion of a vicious cycle to refer to situations in which "national health data are used little because they are of poor quality, and their relative lack of use, in turn, makes their quality remain poor" (p. 379). This is a hard to break cycle. Several attempts have been made to break out of the vicious data use cycle (see for example Nutley and Reynolds [START_REF] Abouzahr | Health information systems: the foundations of public health[END_REF], Braa, Heywood [START_REF] Heeks | Information systems and developing countries: Failure, success, and local improvisations[END_REF], Rhoads and Ferrara [START_REF] Braa | Developing health information systems in developing countries: the flexible standards strategy[END_REF]). These attempts can broadly be classified into two streams. The first deals with problems related to the data supply side such as data quality, availability, and access. The second stream focuses on the demand side by encouraging data use based on changing social institutions such as information culture and human capacity.

With a focus on understanding data use and approaches to generate data demand from decision makers, we conducted an action research which involved the design, customization, and implementation of an information system that supports the collection and analysis of various health indicators during the Lunar New Year in Vietnam between 2014 and 2016. Data reported from more than one thousand hospitals and health facilities (accounting for 95% hospitals in Vietnam) revealed a harsh reality of violence and traffic accidents during Lunar New Year holidays. And the general public was perplexed when figures of traffic accidents announced by MoH and Ministry of Public Security (MPS) contradicted each other significantly. It also sparked a big public debate on data quality, data collection and related processes.

In developing countries, routine health data and indicators are not usually high in demand. Those collecting it commonly collect it only for the purpose of upward reporting and receive little if any feedback. And those receiving it commonly do not trust the data nor use it for decision making. This paper presents a contrasting case to this where the mundane and neglected routine health data suddenly was in the spotlight.

The rest of this paper is organized as follows. In the next section, we review related research and debates on the problem of data use and offer our perspective. Research method and the case description are provided in section 3 and 4 respectively. We discuss the findings and implications for theory and practice in section 5 before conclude the paper in section 6.

Related Research

The ultimate goal of any HISs intervention is to ensure quality data that informs decision making. Informed decisions will lead to better use of limited and often scarce resources where they are needed. If data are not used, HISs and efforts to collect, aggregate and distribute health data are meaningless. Unfortunately, despite investments in and improvements of HISs, lack of data use is common (Braa et al., 2012, Wyber et al., 2015). This section presents recent studies that discuss efforts to tackle the problem of little data use.

Lomas [START_REF] Lomas | The in-between world of knowledge brokering[END_REF] in a study from the healthcare sector in Canada emphasizes the role of intermediaries in disseminating research results, thus facilitating the use of health data. He argues that there is a poor understanding of context where the research result is generated and realities facing policy makers. A solution to this problem is to im-prove the communication between the two sides. He discusses the approach that has been applied in Canada for many years, which involves the institutionalization of the knowledge brokering roles in disseminating the data. In particular, research results should be communicated effectively and succinctly through Mythbusters and Evidence Boost, which are a form of research summary, to other groups of users such as legislative, administrative, and industrial decision-makers.

Bowen, Erickson [START_REF] Bowen | More than "using research": the real challenges in promoting evidenceinformed decision-making[END_REF] identify 8 different barriers that constrain the use of data for decision making in a study that involves health staff from 11 Regional Health Authorities in Manitoba province of Canada in three years. They argue that efforts to improve data accessibility and data use competency are important but might not increase the level of data use if the key barriers to the issue are not properly lifted. Their findings include an interesting discovery of "politics trumps evidence" which refers to a phenomenon where data are sought to back a political decision rather than inform decision making process. Sometimes, decision makers could not make an evidenceinformed decision because they lack of means and supporting structure to implement a decision. Nutley and Reynolds [START_REF] Nutley | Improving the use of health data for health system strengthening[END_REF] synthesize previous works related to the issue of health data use including the World Health Organization's Health Metric Network tool [START_REF] Network | Assessing the national health information system: an assessment tool[END_REF], the Performance of Routine Information System Management (PRISM) framework [START_REF] Aqil | PRISM case studies: strengthening and evaluating RHIS[END_REF][START_REF] Aqil | PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems[END_REF], Lomas [START_REF] Lomas | The in-between world of knowledge brokering[END_REF], and Patton [START_REF] Patton | Utilization-focused evaluation[END_REF] to propose a comprehensively logical model that encompasses a set of eight processes and activities to strengthen data use.

In a developing country context, Braa, Heywood [START_REF] Braa | Improving quality and use of data through data-use workshops: Zanzibar, United Republic of Tanzania[END_REF] conduct an action research with the aim to break the "vicious cycle" of data quality and data use. The underlying assumption of their intervention is that piecemeal increased data use can gradually improve the data quality and build up data use capacity. Quarterly workshops on data use are organized at both national and district levels. During the workshops, district staff will show data from their districts and present their own interpretation while other district staff will discuss and criticize. In three years between 2005 and 2008, there are noticeable changes in data use as a result of the intervention. Similarly, based on action research in Kenya with the aim to establish mutually agreed activities among key stakeholders to improve data use for action, Manya et al. (2015) argue that data quality audits must be organized at the same time with monthly data review meetings and training in data management in order to address the problem of little use of data. Moyo, Kaasbøll [START_REF] Moyo | The Information Transparency Effects of Introducing League Tables in the Health System in Malawi[END_REF] also discusses the process of development and introduction of a League Table to compare the performance of different health districts in Malawi. They argue that the module is a useful tool to remedy the problem of information transparency. Also according to them, transparency of information should be treated as a totality of all aspects including disclosure, clarity, and accuracy rather than separately.

In the extant literature on the use of health data, the attention is central on how to provide health managers high quality data to support the decision making process. There is little discussion on how health data mean to the public and how the public can participate in the routine health system design and use. Presumably, most data from the routine health management information systems (HMIS) are supposed to serve managerial purposes. The public who is subject of all health policies is often ignored from such systems and they do not have access to their data. They neither know about how such systems operate, what data are collected nor raise their voice on how such systems should be designed to benefit them. The primary aim of this research is to contribute to that discussion and more specifically focus on finding appropriate approaches that could make boring routine health data become attractive to the public.

Method

This research follows an action research approach [START_REF] Avison | Action research[END_REF]. Action research is a method that aims to solve practical problems and at the same time generate theoretical knowledge [START_REF] Winter | Learning from experience: Principles and practice in action-research[END_REF]. Action researchers have dual responsibilities: to improve the situation and to report findings of their study. Action research is different from other methods in its ability to develop knowledge for both theory and practice. In studying technology in social context, action research is a strong candidate [START_REF] Baskerville | A critical perspective on action research as a method for information systems research[END_REF]. While there are many forms of action research, canonical action research is commonly used as it ensures the rigor of the research [START_REF] Davison | Principles of canonical action research[END_REF].

Data for this study mainly come from the daily interactions, participation and observations of the first author in the daily activities of the project. A research diary was used to capture important events, incidents, or other important discussions related to the system development and implementation. To improve the reliability, data from this source were regularly shared with the second author to independently verify. Other modes of data collection include interviews. A total of 15 interviews with different kinds of informants: MoH officials (2), the technical and support teams [START_REF] Davies | Conceptualising the information needs of senior decision makers in health[END_REF], provincial health administrators [START_REF] Heeks | Information systems and developing countries: Failure, success, and local improvisations[END_REF], and data clerks from hospitals (5) was conducted in early 2014 through face to face and phone. Each interview lasted between 30 and 45 minutes. Notes were taken during the interviews for subsequent use in the analysis step. Informants who were hospital users were selected from those who called the supporting team to get support. The purpose of these interviews was thus to understand better users' difficulties in using the system. Examples of interview questions for this type of informants were "How do you find the design of that functionality? How should it be improved?". Archival records were another means utilized for collecting data. To understand the view of the public on various social issues such as traffic accidents and violence during Tet, mass media articles were also considered, mostly in the electronic form (e-newspapers) including statements of officials from the Government and Assembly.

After being collected, data was grouped into broad themes such as technical infrastructure and social infrastructure, the rapid process of development and implementation of the system, the inconsistency of reported data in comparison with other available sources, and the public's attitudes and the perspectives of different governmental agencies. Several concepts from the literature information transparency, development, and data use were used to guide the analytical process. Part of the analysis process is the narrative of the case, which is now presented.

4

The Case

In early 2014, the Department of Medical Services Administration (VAMS) of the Ministry of Health decided to build an online system to support collecting data related to accident and emergency during the Lunar New Year (Tet) in Vietnam. A medical doctor from VAMS, who is responsible for Information Technology (IT) and health data, consulted a technical team (here after called the DHIS2 Team) that had worked in previous health information systems projects at MoH. His aim was to explore the possibilities of using an open source platform especially developed for the health sector called DHIS2 (dhis2.org) to build an online system for Tet reporting. As the decision by VAMS was made very late, they only had one week to complete the system. After some considerations, the DHIS2 Team confirmed that they were able to build the system and immediately embarked on the mission. DHIS2 is a software platform developed by University of Oslo, Norway. It provides a flexible mechanism in handling data elements and forms, making it easy to define new datasets. However, the default data entry form in DHIS2 only allowed data entry for one period at a time. This was considered to be confusing for the users. The team thus decided to design a custom data-entry form that shows multiple periods in a single screen.

After building the data entry forms, the DHIS2 Team deployed the system on one of MoH servers. A few months before Tet, MoH had implemented a hospital quality and inventory system which was also based on the same DHIS2 software. User accounts for this system were generated using a script and forwarded in Excel files to each health province which subsequently forwarded to their subordinate hospitals. The DHIS2 Team decided to leverage this existing user base. Since they already had access and knew the system, this would shorten the implementation process and eliminate the need of training. An official letter requesting hospitals to use the Tet reporting system was distributed by the MoH.

As it was the first year using the Tet reporting system, many hospitals still viewed the system as a "pilot". As a consequence, lack of rigor in data entry was observed. Some users entered garbage numbers into important fields like injured cases by firecrackers, as if such figures would not be subsequently aggregated and reported to the government and visible to the public via mass media. It should be also noted that according to the law in Vietnam, the head of province will receive disciplinary action if there is illegal use of firecrackers in his/her province. To avoid such data entry incidents, the IT specialist from MoH who was assigned to work on this system had to frequently verify if there were any suspicious numbers entered. He also called the person who was responsible for reporting on that day in the hospital to confirm the reported numbers.

During the Tet 2014, data from more than one thousand hospitals were daily aggregated by MoH and subsequently reported to the Government Office. Many journalists approached MoH to acquire data to write articles because they believe accident and emergency during the Tet are matters of public concern. After articles were published, the public was shocked to know that nearly 7,000 people were hospitalized, out of which 15 people died, due to violence during the 9 days of the Tet holiday. The Deputy Chair of Social Committee, Vietnam Assembly said: "This figure was an alarm of the increasing violence in our society. It was very unusual because the Tet holidays were the time dedicated for joy and relax. The Committee welcomes MoH for its first time publication of the data which we did not have previously." [START_REF] Moh_Portal | 6.200 cases hospitalized due to violence: usual or unsual?[END_REF]. The Prime Minister insisted that such figures were very serious and directed all concerned agencies to propose and implement effective measures to mitigate violence [START_REF] Life&law | Prime Minister: Violence during Tet is not minor. Life and Law[END_REF].

In addition, many scholars, educators, and psychologists proactively joined the debate, trying to locate the root causes of the issue. An economist from the Center of Economic Research in the South stated: "People solve their conflicts by using violence because they have lost their trust on justice and government" [START_REF] Vu | Why aggressive behavior is on the throne[END_REF]. And a researcher from Research Center of Sino-Nom said: "It is a really crisis of the crowd, once its psychology is compressed throughout the year and it bursts on the Tet. It accumulates all conflicts with the root cause of unstable and insecure society. Culture, morality, and education in Vietnam have never been degraded like they are now" [START_REF] Moh_Portal | 6.200 cases hospitalized due to violence: usual or unsual?[END_REF][START_REF] Vu | Why aggressive behavior is on the throne[END_REF]. There was a broad agreement that in combination with stricter punishment for violent crimes, schools should focus more on ethical and behavioral education for youngsters.

Regarding the reported figures of traffic accidents, the public was at the same time very puzzled by the gap [START_REF] Tuong | Every insitution reports a different number: the public can not trust[END_REF] between the figures announced by Ministry of Public Security (MPS) and Ministry of Health (MoH). While the figures published by MPS showed a significant decrease in the number of injuries (-25%) and death (-5%), MoH announced an increase of traffic accidents during the Tet holiday. Quantitatively, MPS reported only 408 traffic accidents whereas MoH reported more than 40,000 hospitalized cases related traffic accidents. This triggered debates related to the trustworthiness of these two different information sources. For instance, the Head of Road Transportation Department (MPS) argued that the police could only collect data from traffic accidents that were reported to them while, in reality, most victims were transferred directly to hospitals. This argument was supported by a statement made by a representative of National Committee of Transportation Safety (NCTS): "NCTS will make a proposal to the Prime Minister to get permission to use data from MoH as an official source for traffic accident reporting, instead of exclusively using data from the police" [START_REF] Ha | Death by traffic accident: contradictory figures[END_REF].

Analysis and Discussion

The design and implementation of the Tet reporting system can be considered as a successful story because it rapidly achieved national coverage and generated data that triggered public debates on several social issues. We now provide analysis on how the system attracted the public's attention to what is commonly treated as boring and mundane routine data.

Collecting data that highly concern the public and commodifying health data

Hospital users often see reporting duty as a burden rather than something useful for them or the healthcare sector (Krickeberg, 2007, Kuhn and Giuse, 2001, Littlejohns et al., 2003). In the case of the Tet reporting system, the number of data elements was minimal and only important data elements were included. The small dataset approach also helped reduce complexities of the system both at technical and social levels. The flat structure of dataset made the design and use of the system become easy and helped shrink "the time to market" (Smith, 2004). This small dataset approach has another implication. Data entry forms of HISs in developing countries tend to be overly complex (Sahay et al., 2010) and merely serve the need of health managers. The public, however, should have the right to know about data that directly concern their life. In other words, HISs should also be designed in a way that it collects kinds of data that are highly related to the lives of majority of citizens. This is not the case in Vietnam where figures on traffic accidents and violence are completely absent on the routine report.

Routine health data are often too complex to understand and use even for health professionals [START_REF] Moyo | The Information Transparency Effects of Introducing League Tables in the Health System in Malawi[END_REF]. In order to attract attention of the public, the data need to be simplified and commodified. For example, the number of deaths related to traffic accidents and number of injuries related to violence are simple and easy enough for anyone to understand. The use of raw numbers to some extent gives better impression about the situation compared to the use of indicators. Regardless the size of the population, thousands of people injured by violence is something hard to believe and unacceptable.

Commodifying data also relates to the two aspects of information transparency which are disclosure and clarity [START_REF] Moyo | The Information Transparency Effects of Introducing League Tables in the Health System in Malawi[END_REF][START_REF] Schnackenberg | Organizational transparency a new perspective on managing trust in organization-stakeholder relationships[END_REF]. While disclosure only focuses on the availability and accessibility of data to interested parties which are very often governmental agencies and not-for-profits organizations, commodifying also involves making sense of data to the public. In the attempt to engage the public in the debate of critical health issues, clarity must be more focused to the level that it is understandable for the public who do not know anything about the healthcare domain.

Engaging mass media and fueling public debates

Routine health data are often moved upward, i.e. from health facilities to higher level. Little if any feedback from higher level health managers are provided to health facilities after data are reported [START_REF] Charles | The great legitimizer: ICT as myth and ceremony in the Indian healthcare sector[END_REF]. To attract the public's attention on critical health issues, novel channels of rapid and large-scaled data propagation must be sought. One such channel is mass media, i.e. newspapers and magazines. With large numbers of readers and hourly news posting, newspapers (both paper-based and online) can draw attentions from the public in a quick and effective way. Also, thanks to the rhetoric of journalists, titles of articles are often written in an impressive and attractive way which no doubt makes the boring and mundane data become lively and attract the attention of the public.

As a general practice, mass media often engage important and renowned figures to comment on emergent social issues. They can be professors from universities, experts in various domains, and incumbent officials or politicians. This approach is to bring the readers multiple perspectives on a particular issue. In the case of violence, the debate of data accuracy was sparked by MPS officials who expressed suspicion on the big number of hospitalized cases because of violence. There was a firm response from the Head of Medical Service Administration (VAMS, MoH) about that. The response included the unpacking of procedures and systems through which the number was aggregated. This was an important step to keep the dialog continue and remain constructive. Later, the IT specialist who was the key contact point of the project agreed to give an exclusive interview to an online newspaper on the history and backstage story of how the system was built and implemented.

Creating feedbacks and triangulating data

Also through newspapers the public knew more about the work of hospitals during the holiday. This gave users the feeling of participation. Through the circulation process, data are extensively validated and flexibly interpreted by a large collection of human actors. We argue that data circulation processes are crucial for sustainable HISs as it blurs the boundaries between data collection and data use, creating the congeniality [START_REF] Sanner | Grafting: Balancing Control and Cultivation in Information Infrastructure Innovation[END_REF] between the ICT initiative and the environment. The findings of our case are slightly distinct from literature discussing the role of new social media such as Facebook, Twitter etc. on changing awareness of the public and creating new channel of two-way communication between public health experts and the public. Recent works on social media emphasize the significance and effectiveness of using social media to engage the public in public health issues [START_REF] Abroms | New media cases in Cases in Public Health Communication & Marketing: the promise and potential[END_REF][START_REF] Heldman | Social media engagement and public health communication: implications for public health organizations being truly" social[END_REF][START_REF] Newbold | Media and social media in public health messages: a systematic review[END_REF][START_REF] Willoughby | Communication Strategies and New Media Platforms Exploring the Synergistic Potential of Health and Environmental Communication[END_REF]. However, we argue that depending on the situation, traditional media are still very powerful means in amplifying and visualizing health issues to the public which in turn would help to increase accountability and transparency.

We believe one factor that played an important role in the success of the Tet system was the feedback mechanism. The extant literature often reports cases of HISs in which data reported by data entry clerks do not receive adequate feedbacks to improve the quality of data [START_REF] Braa | The struggle for district-based health information systems in South Africa[END_REF]. In our case, there exist two types of feedback mechanisms: feedback from MoH to hospitals and from the public to hospitals. As the reporting process took place on a daily basis, hospitals received feedback from MoH within almost the same day. This instant feedback played a crucial role in not only improving data quality but also increasing the completeness of data. In order to have timely data to report to government and share with the public, the technical team had to play the role of designer and user at the same time. For example, they had to check data on an hourly basis to early discover any mistake in reported data. They had to call hospitals to verify the figures and sometimes directly modified them. In another flow of feedback, when a data entry clerk reported a figure through the system, the figure was aggregated at the national and subsequently shared with the public. As a member of the public, the clerk also received that aggregated figure and had her own interpretation thus she became a user of the system.

Many studies report the problem of fragmentation in HISs in developing countries (see for example Chilundo and Aanestad [START_REF] Chilundo | Negotiating multiple rationalities in the process of integrating the information systems of disease specific health programmes[END_REF]). Fragmentation refers to the situation where multiple, duplicate and overlapping systems coexist but cannot provide data necessary for decision making. While we agree that overlap in data collection should be minimized to the best extent, in some special occasions, it is a chance for restructuring and improvement. For example, in the case of traffic accident reporting, MPS and MoH are the two agencies that have routine data collection systems. The contradiction in data reported by the two systems made the public suspicious about the veracity of the data. The number of traffic accidents reported by MPS system was unreasonably too small compared to what reported by MoH system. As a result of this triangulation, the Head of National Traffic Safety Committee agreed that data from MoH system would be incorporated into the report next year.

Getting full-coverage data ready and publishing data

One of the issue that face HISs practitioners and researchers is the dilemma of "all or nothing" (see Braa, Hanseth [START_REF] Braa | Developing health information systems in developing countries: the flexible standards strategy[END_REF]). Incomplete data for a geographical area have little value in use. Complete and Full-coverage data is only attainable when a full scale implementation of HISs is achieved. Scaling, however, is a big challenge as it is not about mechanically replicating the same technical artefact to other contexts but also multiple sociotechnical rearrangements [START_REF] Sahay | Scaling of health information systems in India: Challenges and approaches[END_REF].

Research emphasizes the role of installed base in terms of existing technology and established use practices in shaping (and being shaped) the outcome of ICT interventions [START_REF] Grisot | Innovation of, in, on infrastructures: articulating the role of architecture in information infrastructure evolution[END_REF]. Taking advantage of what already exists to kick-start, or bootstrap [START_REF] Hanseth | Design as bootstrapping. On the evolution of ICT networks in health care[END_REF], is an advisable strategy. The Tet reporting system was built on the existing DHIS2 software system already implemented in the hospitals. By leveraging the existing software, the team cut down the expensive and lengthy processes of full life cycle of software development. Since users at the hospital were knowledgeable of the DHIS2 system, expensive and time consuming training was not necessary. DHIS2 has a data entry module that is highly configurable to adapt to a wide range of requirements. However, this module is sometimes too complicated for beginners. The technical team was inspired by the design of social media tools such as Facebook where anyone can create an account and start to use Facebook without attending any training classes. Therefore, they designed the system in a way that minimal or no training was needed. For instance, the data entry form was customized to enable data entry for multiple days in a single screen. Also, a report printing button was integrated into the data entry form, making it easier for users to view and export data to other formats such as Microsoft Excel for further processing. This approach was also more responsive to changing requirements. For instance, the Tet reporting system required an additional dataset. This could be easily be made through the dataset management module of DHIS2. No line of code was required. As a result, the system became ready in a short time.

Many researchers emphasize the role of good data in improving the quality of health service delivery [START_REF] Donaldson | Health Data in the Information Age:: Use, Disclosure, and Privacy[END_REF][START_REF] Rhoads | Transforming healthcare through better use of data[END_REF]. Substantial efforts have been made in building reliable and sustainable HISs. However, having data does not automatically lead to data use. Noir and Walsham [START_REF] Noir | The great legitimizer: ICT as myth and ceremony in the Indian healthcare sector[END_REF] discuss the ceremonial and mythical use of HISs in the public health sector in India where health centers entered data into local computers, exported them to flash drives and carried the flash drives to higher level to submit data. The authors challenge the simplistic view that ICTs adoption will directly generate efficiency and gains. In our case study, the collected data were processed by the MoH team and subsequently shared with other stakeholders and news agencies to trigger data use process. The decision of MoH to publish the data to mass media created a feedback loop that reinforced the data reporting activities at hospitals, motivating hospitals to collect and report accurate data.

Publishing data goes in hand with accountability and transparency because MoH must be responsible for data it published. Prior to the implementation of this system, MoH only used the data for internal management. It was not obliged to make the data public. This decision was thus an important step toward public transparency. And public transparency led to public accountability.

Conclusion

Making life better is a goal of any ICTs project. This research presents an empirical case where data collected by a software system were used to trigger public debates that could contribute to shape policy and the public's attitudes. The approach discussed in the paper emphasizes the significance of selecting matters of public concern (few critical data elements) and rapidly scaling the implementation to get full data coverage. Collected data are disseminated and amplified by mass media to attract public's attention and trigger debates. Through the mass media, public express their concerns on critical health issues. As a collective effect, voices of people are better heard by the government. Lessons learnt from this research are hoped to provide pub-lic health administrators and IT practitioners design ideas and principles that help design sustainable HISs that empower people especially those who are marginalized. Time constraint did not allow us to venture deeply into the interpretations of collected data, i.e. geographical analysis of violent cases or traffic accidents. We believe the quantitative analysis in this area will be useful for the public and policy makers in preventing and reducing the cases related to traffic and violence. Our findings also contribute to the debates of how to improve the efficiency of HISs through the use of data for decision making.