M ; Wyatt

Health Brixton

Daniel Chandramohan

Bayard Roberts

Oliver W Morgan

Mohammed Ghaus Sultani

Peter Nyasulu

Sunday Rwebangila

Mark Myatt

Egbert Sondorp

Francesco Checchi
email: francesco.checchi@lshtm.ac.uk

A New Method to Estimate Mortality in Crisis-Affected and Resource-Poor Settings: Validation Study

Keywords: Mortality, capture-recapture, validation, humanitarian crisis, survey, surveillance mortalit y, epidemiology, war, Afghanistan, Tanzania, Malawi, Thailand Mortality, death rate, validation, humanitarian, crisis, survey, surveillance, capturerecapture

Key message

• Current methods to estimate mortality in crisis-affected and resource-poor settings feature important methodological limitations and have not been sufficiently validated. • We developed and validated a new method for estimating mortality which potentially offers a more rapid and timely means of estimating mortality compared to existing methods. • The new method showed only moderate sensitivity for assessing mortality but it demonstrates sufficient promise to warrant further development given the importance of real-time mortality measurement in such settings.

Background

Data on the levels and causes of population mortality are critical for assessing a population's health status and informing public health interventions in crisis-affected and resource poor settings.(1-2) However, in such settings, routine vital registration systems are often unreliable, cease to function, or do not exist. The two main alternative approaches currently used in such situations to measure population mortality are prospective demographic surveillance (regular exhaustive enumeration by home visitors of individual births and deaths) and retrospective mortality surveys (estimation of mortality over a specified recall period based on interviews with a representative sample of households). Both approaches have serious limitations. Surveillance requires a large number of closely supervised staff to collect data; it can be difficult to implement in geographically dispersed populations [START_REF] Checchi | HPN Network Paper 52: Interpreting and using mortality data in humanitarian emergencies: a primer for non-epidemiologists[END_REF]; and the sensitivity of the surveillance system may be low, even in long-standing refugee camps. [START_REF] Spiegel | The accuracy of mortality reporting in displaced persons camps during the post-emergency phase[END_REF] Retrospective surveys are poorly suited to generating essential operational data on current levels and causes of mortality, because over very short recall periods (e.g. <1 month) they require unfeasibly large sample sizes to avoid meaninglessly imprecise estimates (with imprecision further compounded by the design effect inherent in cluster sampling). [START_REF]Working Group for Mortality Estimation in Emergencies. Wanted: studies on mortality estimation methods for humanitarian emergencies, suggestions for future research[END_REF][START_REF] Prudhon | A review of methodology and analysis of nutrition and mortality surveys conducted in humanitarian emergencies from October 1993 to April 2004[END_REF][START_REF] Spiegel | Who should be undertaking population-based surveys in humanitarian emergencies?[END_REF] Retrospective mortality surveys are also subject to various selection and reporting biases stemming mainly from inadequate sampling frames, unequal probability of selection of households (especially in cluster designs), and deliberate or involuntary inaccuracies in questionnaire responses. [START_REF] Alkhuzai | Violence-related mortality in Iraq from 2002 to 2006[END_REF] Statistical analysis of retrospective survey data is also more difficult than for surveillance data. Partly as a result of these limitations, the quality of mortality surveys done in the humanitarian sector is generally low. [START_REF] Prudhon | A review of methodology and analysis of nutrition and mortality surveys conducted in humanitarian emergencies from October 1993 to April 2004[END_REF] In this paper, we report on a multi-site evaluation of an alternative method (which we refer to as the informant method) designed to furnish precise and unbiased estimates of mortality over a very short recall period (i.e. almost on a real-time basis) and be simple enough so that practitioners without statistical training can implement it to rapidly measure mortality in their communities of intervention. We evaluated

Methods

Ethical approvals

The study was approved by the Ethics Committee of the London School of Hygiene and Tropical Medicine; the Institutional Review Board of the Ministry of Public Health of Afghanistan; the Malawi National Health Sciences Research Committee; and the National Institute for Medical Research and also the Commission for Science and Technology in Tanzania. In Thailand, no local institutional review board has jurisdiction over the refugee camps. Household respondents and focus group discussion members were asked to provide oral consent to participate in this study. In Thailand, written consent was sought from all study participants.

Study sites and sampling design

Recognising that validity may be affected by local culture and types of human settlement, we aimed to document the informant method's implementation, performance and limitations in a variety of settings. Specifically, while we mainly conceived of the method as a useful option in refugee or internally displaced persons' camps, we also wanted to document its validity and feasibility in at least one chaotic, dense urban setting and one rural, scattered community. Furthermore, we wanted to ensure that the cultural settings of the study sites would be as unrelated as possible, though we specified that half should be in Sub-Saharan Africa. Accordingly, we selected the following four sites: (i) District (nahia) 1 of Kabul city, Afghanistan, a poor urban community with chaotic layout; (ii) Mae La camp for Karen refugees, on the Thai-Burma border; (iii) Chiradzulu District in Malawi, a rural, scattered community with a high HIV/AIDS burden; and (iv) Lugufu and Mtabila camps (hereafter considered as one site) in

F o r R e v i e w O n l y
western Tanzania, which host refugees from the Great Lakes region. The Kabul site was chosen as it featured an existing London School of Hygiene and Tropical Medicine research programme. The broad regions of the other three sites (Thai-Burma border camps; Malawi rural; East Africa camps) were determined a priori, and the actual sites were identified through investigators' field contacts.

In each site we enquired about a recall period of two months (60 days), but restricted the main analysis to the most recent one month (30 days). Results for the full 60 days period are also presented, as they provide useful information about recall of increasingly remote dates.

We surveyed the populations of District 1, Kabul; Mae La camp; and the Tanzania camps exhaustively.

Assuming a minimum crude mortality rate (CMR) of 0.2 deaths per 10 000 person-days within each of the above sites, and given that each site had an estimated population of >40 000, we calculated that the method's sensitivity (i.e. proportion of all deaths that are detected by the method) for a 30 day recall period could be estimated among ≥26 deaths in each site. This number of deaths would have provided a precision within ±20% for any sensitivity point estimate >80%, which we considered an acceptable performance of the method.

In Chiradzulu district, it was considered inefficient to visit all 757 villages. Instead, we surveyed a fraction, selected using a modified centric systematic area sampling design. We divided the district into 32 five Km x five Km quadrats with area mostly falling within the district boundaries. Using highresolution maps, we then selected the three villages closest to the centre of each quadrat, thus yielding a non-self-weighting sample of 96 villages. Within each sampled village the search for recent deaths was exhaustive. Assuming an average population per village of 500, nearly 50 000 people would be sampled, yielding a minimum precision comparable to that of the other sites.

Implementation of the informant method

The informant method consists of an exhaustive, active search for all deaths occurring in the surveyed community over a defined recall period. Key community informants, selected after rapid qualitative work, recall households that have experienced a recent death and refer data collectors to these households; decedents' next-of-kin also refer data collectors to other recently bereaved households. The number of deaths recorded from the method is then combined with population estimates to calculate a mortality rate for the recall period (see below).

We held focus group discussions (FGDs) in each study site before data collection. Each FGD obtained information for the whole study site and sought to: (i) identify which key community informants would be most knowledgeable about recent deaths; (ii) explore information sharing about recent deaths in the community; and (iii) identify alternative sources of mortality information that would provide additional lists for capture-recapture analysis, later used to validate the method (see below). Local study team members and collaborators helped to identify participants, who were selected based on their knowledge of the community (Table 1). While elements of this method have been used previously, such as obtaining information on deaths from key-informants such as persons in charge of cemeteries [START_REF] Woodruff | Interpreting mortality data in humanitarian emergencies[END_REF] and Traditional Birth Attendants (for maternal deaths) (9), or using snowballing technique for referrals to households that have experienced a maternal death [START_REF] Aggarwal | Risk factors for maternal mortality in Delhi slums: A communitybased case-control study[END_REF][START_REF] Singh | House-to-house survey vs. snowball technique for capturing maternal deaths in India: a search for a cost-effective method[END_REF], to the best of our knowledge this is the first method to combine the use of formative FGDs, information from key-informants, household referrals and population estimation to compute mortality rates.

To carry out the exhaustive search for deaths, we divided each community into sectors, corresponding to pre-existing administrative areas: guzar in Kabul (n=24); sections in Mae La camp (n=22); villages in Chiradzulu District (n=96); and zones in the Tanzania camps (n=23). The primary and secondary informants selected based on the FGD (See list 1 of Table 2) were located for each sector. The data collection then proceeded sector by sector. Sector leaders were informed in advance about the study team's upcoming visit. We asked key informants to separately list all deaths occurring among community residents during the previous 60 days and then visited the households in which those deaths had occurred.

We administered a questionnaire to next of kin aged 18 years or older to record the date of death, which we established with the aid of a detailed calendar of local events; age and sex of the decedent; and the cause and place of death. We included deaths among neonates if they were breathing at birth. In Chiradzulu only, we also administered verbal autopsy questionnaires, using the latest World Health Organization standards [START_REF]Verbal Autopsy Standards: Ascertaining and attributing causes of death[END_REF]. To uniquely identify decedents for the purpose of validation only (see below), we also collected the decedent's name; in Kabul and Mae La we collected the name of the father; and in Chiradzulu and Tanzania camps, the name of the household.

After administering the questionnaire, we asked respondents to list other deaths in their household or anywhere in the community within the previous 60 days. Once we had visited all households with deaths identified by the key informants and households of decedents, we considered the sector exhausted and moved on to the next sector. We did not visit deaths among households outside the community.

Validation

To measure the method's sensitivity we obtained a best estimate of the true number of deaths in the recall period through capture-recapture analysis, used extensively in epidemiology to evaluate the completeness of reporting and/or the true burden of various diseases [START_REF]Capture-recapture and multiple-record systems estimation II: Applications in human diseases. International Working Group for Disease Monitoring and Forecasting[END_REF][START_REF]Capture-recapture and multiple-record systems estimation I: History and theoretical development. International Working Group for Disease Monitoring and Forecasting[END_REF], and recently applied to war-related killings. [START_REF] Silva | The Profile of Human Rights Violations in Timor-Leste, 1974-1999: A Report by the Benetech Human Rights Data Analysis Group to the Commission on Reception, Truth and Reconciliation of Timor-Leste[END_REF](16)[START_REF] Ball | Killings and refugee flow in Kosovo[END_REF] This technique analyses the overlap among different lists of the events in question (in

F o r R e v i e w O n l y
our case, recent deaths) to estimate the number of deaths that do not appear on any lists, which is then added to the "known" or ascertained deaths (i.e. those that are found on one or more lists) to provide the best estimate total.

In each site we generated three different lists of deaths, the first being that obtained from implementation of the informant method itself. We selected sources for the two additional lists (Table 2) based on criteria including sensitivity as judged by FGD participants, feasibility (i.e. how easy it would be to contact informants), and independence from the key informants already consulted (i.e. we avoided sources that would themselves obtain their knowledge from the informant method key informants, or vice versa). We collected these two additional lists only after the implementation of the informant method, to minimise investigator bias. We approached sources for the two additional lists in the same way as the informants used for our method, and asked about the same recall period. We considered deaths recorded in different sources as the same if they matched the sex, age, place of residence, and name of the deceased in Kabul and Mae La, or father or household name in Chiradzulu and Tanzania camps.

Because the probability of a death appearing on a list is usually not independent from the probability of appearing on a different list (e.g. prominent individuals may be remembered by most sources; a death mentioned by one religious group might be unlikely to appear in the list of another religious group), possible interactions among lists need to be explicitly factored into the capture-recapture estimation. To do this, we fitted log-linear Poisson models [START_REF] Fienberg | Classical multilevel and Bayesian approaches to population size estimation using multiple lists[END_REF] to the three-way contingency table formed by each x abc category, where a, b and c denote the presence (1) or absence (0) of a death within lists 1, 2 and 3 respectively (e.g. x 110 is the number of deaths on lists 1, 2 but not 3). Model terms represented the probabilities of inclusion in lists 1, 2 and 3; additional interaction terms represented possible dependencies among lists (i.e. inclusion in one list being a function of inclusion in another). Eight and N, the estimated total number of deaths, is the number of deaths appearing on one or more lists plus m 000 . [START_REF]Capture-recapture and multiple-record systems estimation II: Applications in human diseases. International Working Group for Disease Monitoring and Forecasting[END_REF] Instead of selecting one of the models as the most likely, we used Bayesian Model Averaging [START_REF] Hoeting | Bayesian Model Averaging: A tutorial[END_REF] with uninformative priors to derive average estimates from all the models combined (in this method, the prediction is a weighted average of predictions from each model, with the weight provided by each model's posterior probability, a Bayesian statistics that indicates how likely it is that the model is the correct representation of reality). However, we first excluded any models that did not converge to a solution or had a Chi-square goodness of fit P-value >0.60 (we considered this evidence of overfitting; in practice all these models also predicted implausibly high m 000 values). Using the remaining models, we computed

∑ = = = K i i i i m m 1 , 000 000
Pr where K is the total number of models averaged over, i is one of these models and Pr is its posterior probability. 95% confidence intervals were derived from the models' likelihood profiles.

When there were a sufficient number of deaths recorded, we estimated the total number of deaths among children under 5 years, as above: however, this was not always feasible due to small numbers. For each site and recall period (30 and 60 days), we expressed the sensitivity of the informant method or any other source as the number of deaths captured by the source, divided by the capture-recapture estimate of the total number of deaths (N).

Calculation of mortality rates

Existing population estimates for all ages and children under 5 years in Mae La and the Tanzanian camps were judged reliable and adopted. In Kabul and Chiradzulu, we performed our own population estimation (see Supplementary File 1). We expressed mortality rates as deaths per 10 000 person-days. We estimated three different mortality rates using the number of deaths (i) captured by the informant method, (ii) captured by all sources combined and (iii) estimated by the capture-recapture analysis. We computed point estimates and 95%CIs through bootstrapping programs specific to each field site, taking into account observed design effects and sampling weights where needed (see Supplementary File 1).

We double-entered and cleaned all data using EpiData version 3.0 software (EpiData Association, Odense, Denmark). All analysis was done using R software.

Results

Data collection was conducted between July and October 2008, taking 11, 5, 16 and 6 working days in Kabul, Mae La, Chiradzulu and the Tanzania camps, respectively (Table 3); this includes time for collection of additional lists for capture-recapture analysis. The size of the four study populations ranged from 43 794 to 80 136 people. All community informants who were found agreed to provide In all four sites, primary and secondary informants provided the majority of reports (Table 4). Few reports came from respondent households; of these 3/11 in Kabul, 3/9 in Mae La, 3/6 in Chiradzulu, and 12/14 were also reported by community informants, considering a 60 day recall period. Several deaths were excluded from the analysis, for the following reasons: (i) stillbirth (n=5); (ii) death outside the 60 day recall period (n=16); (iii) not residing in the community (n=8).

Of the three different lists collected in each site (Table 5), the informant method identified the largest number of deaths, except in Mae La, where the existing surveillance system identified more deaths (the overlap of the deaths recorded among the three lists is shown as Venn diagrams in Supplementary File 2).

In each site, the age and sex profile of deaths was similar for all lists, except in Kabul where hospitalregistered deaths were all male and mostly children, and surveillance records in Mae La which had a higher proportion of children. None of the lists showed notable differences in the age and sex profile comparing the 60 and 30 day time periods.

Estimated sensitivity of the informant method and other sources

Sensitivity of the informant method ranged from 45% to 65% over a 60 day period, according to the site, and from 55% to 73% over a 30 day period (list 1, Table 6). The sensitivity of the informant method was higher than that of the other lists, except in Mae La and in Kabul over a 30 day period. The sensitivity of any source was higher over a 30 day recall period than a 60 day recall period, with the exception of the informant method in Kabul. The combination of all three sources appeared to achieve good sensitivity in all sites. Among children under 5, the method's sensitivity was 52.6% (95%CI 17.1-71.4) in Kabul (60 days period), 66.7% (95%CI 27.1-81.2) in Chiradzulu (60 days), and in Tanzania it was 53.7% (95%CI 36.1-61.1) over 60 days and 47.1% (95%CI 15.7-61.5) over 30 days. Other sensitivity estimates could not be computed due to the low number of child deaths, precluding capture-recapture analysis. However, considering ascertained deaths only, the informant method detected no more than 1/5 in Kabul (30 days), 2/16 and 0/4 in Mae La (60 and 30 days respectively), and 12/15 in Chiradzulu (30 days).

Estimated mortality rates

The mortality rates estimated by the informant method, all the lists combined, and capture-recapture analysis are given in Table 7. The mortality rates estimated by the informant method and any other single method are substantially lower than those estimated through capture-recapture analysis over the 30 day and 60 day periods. Mortality rates among children under 5 were approximately double the all-age CMR, as typically observed in developing country settings. Comparisons of our estimated mortality rates over a 60 day recall period with recent survey-based estimates of mortality from broadly comparable regions and also national-level estimates are provided in Table 8 and these show our capture-recapture estimates are broadly consistent with these other estimates.

Discussion

To our knowledge this is the only recent study with the aim of developing and validating a new mortality estimation method based on primary data collection. The informant method potentially offers a more rapid means of estimating mortality than retrospective surveys and surveillance systems. The results show that the informant method performed better than the active surveillance system in the Tanzania refugee camps, but achieved moderate sensitivity compared to a best estimate of mortality, and was not good at There are different reasons why the informant method did not identify some deaths. The choice of key informants may have been sub-optimal and inappropriate for childhood deaths. This may have arisen because of dominance of community leaders among FGD participants, which may have masked the diversity of opinions in the community and discouraged other participants from expressing their views, resulting in over-direction by community leaders about which key informants we used. Furthermore, in both Kabul and Chiradzulu, key informants derived their information about deaths from similar sources.

A greater number and diversity of key informants may have been warranted.

It may have been inherently difficult for key informants to know about all deaths in their sectors, due to (i) large sector populations (about 3000 people in Kabul, 2000 in Mae La, and 4500 in Tanzania); (ii) lack of trust and thus information sharing between key informants and community members (e.g. in Kabul many residents were short-term renters without kinship ties to the wakils); and (iii) inexperience of key informants (e.g. in Mae La many section leaders were newly appointed as part of a repatriation scheme).

Household respondents provided a minority of all referrals; given the relatively low mortality in the study sites, this may have been because households did not know about other deaths. Additionally, households may have been reticent to share such information with strangers, wary of upsetting community leaders or the bereaved families themselves, or acting based on other cultural and/or religious beliefs and practices.

Both key informants and households may also have deliberately withheld information on deaths due to sensitive causes, such as suicide or drug addiction. This issue would however affect alternative mortality measurement methods and hence is not a particular weakness of this method. Furthermore, the main Theoretically, the informant method presents considerable advantages over both surveillance and surveys for the purposes of real-time mortality measurement. It is a one-off activity that can be repeated on a regular basis by staff with limited research skills. The small number of interviews conducted (i.e. only in households with deaths) greatly reduces data entry requirements; a simple analysis can be done with pen and paper, as there is no need for weighting and design effect adjustment; and the questionnaire can be expanded to explore the timing/location of the death (e.g. if the death occurred when residing in the study site or beforehand) and the causes and circumstances of death through methods such as verbal autopsy, since considerably more interview time can be devoted to each household than for other methods. The findings on issues of feasibility of the new method (e.g. time, financial, use of verbal autopsy questionnaires, and ethical implications) will be presented elsewhere. A potential disadvantage is the need for accurate population estimation. However, population estimation is frequently required by humanitarian agencies for operational response in crisis affected and resource poor settings, and so is itself a useful measurement activity. Like retrospective surveys (but not surveillance), the informant method is also affected by potential survival bias (households disintegrating during the course of the recall period), though the potential effect of this bias may be small in our case due to the short recall period. The method's sensitivity did not reach 80%, the level we aimed for when developing this method. Based upon our experience of developing and testing the method, we speculate that it may be possible to increase sensitivity by: (i) ensuring that the FGD participants represent a mix of men and women, ages, occupations and economic and social hierarchies; (ii) using multiple rounds of FGDs; (iii) using other discussions; (iv) using more than two informants: in this study, we easily collected additional lists of deaths; and (v) using less formal informants (e.g. groups of people at gathering points such as water sources, shops, etc.).

Limitations of this validation study

Capture-recapture analysis may not have provided an accurate estimate of mortality. The method is highly sensitive to accuracy in matching individuals who appear in more than one list. In this study we matched on the basis of place of residence, sex, age, and name: the latter two variables are often problematic in settings without vital registration. While some over-or under-matching may have occurred (resulting in under-or over-estimation of the true number of deaths, respectively), we did not note instances of equivocal combinations of the above characteristics, probably because the chance of repetition was very low (there were almost always less than five deaths reported on any list, for a given sector). Other problems with establishing the lists occurred, however. In Kabul, hospital records were often incomplete, leaving us with a very small list that decreased the precision of the capture-recapture estimate. In Mae La and Chiradzulu respectively, data were collected through religious leaders and Health Surveillance Assistants and we were not able to closely supervise data quality: if some of the deaths reported by these sources did not in fact meet our inclusion criteria, the total number of deaths would have been overestimated and sensitivity underestimated.

Establishing the correct date of death was challenging. It was apparent that many household respondents and informants had difficulties pinpointing the actual date of death. We attempted to minimise error through a series of questions about each death and a detailed calendar containing local salient events.

The sensitivity of the informant method was lower for 60 days compared to 30 days, which suggests that Non-systematic misclassification of date of death, particularly around the 60 day mark, would generally have led to less overlap among lists, and thus underestimation of sensitivity and over-estimation of the true number of deaths: this may explain the curious finding that, in all four sites, more deaths were estimated to have occurred during the period 60 to 30 days before the survey, than in the last 30 days (Table 6, last column). This finding, if not spurious, may also be due to a simultaneous seasonal effect across all four sites, though this is less plausible.

Capture-recapture analysis also entails arbitrary decisions in the choice of statistical models, though we attempted to minimise these through Bayesian averaging. Alternative choices of models did not significantly alter the main sensitivity findings (data not shown).

Despite these potential biases, the mortality rates we estimated through capture-recapture reflect expected patterns given the underlying epidemiological and demographic profile in the study sites, and were broadly consistent with recent survey data from neighbouring areas within the study countries [START_REF] Bartlett | Where giving birth is a forecast of death: maternal mortality in four districts of Afghanistan, 1999-2002[END_REF][START_REF] Mashal | Factors associated with the health and nutritional status of children under 5 years of age in Afghanistan: family behaviour related to women and past experience of warrelated hardships[END_REF][START_REF] Morgan | Thailand demographic surveillance[END_REF][START_REF]Complex Emergency Database[END_REF] and also national-level estimates [START_REF]International Data Base[END_REF][START_REF]Country Information[END_REF][START_REF]Tanzania] and ORC Macro. Tanzania Demographic and Health Survey 2004-5[END_REF]. In Afghanistan, we investigated an urban neighbourhood of Kabul which is well served by health facilities and reasonably well covered by preventive services and where mortality would be expected to be lower than the nationwide average which includes under-served rural areas. The camp-based populations we investigated in Thailand and Tanzania enjoyed long-standing humanitarian assistance and comparatively good health services which helps explain why their mortality rates are similar to or lower than the local and national mortality rates. In Chiradzulu district of Malawi, the crude mortality rate recorded by the informant method falls within the range of survey data from the local region and is slightly higher than the national rate which is to be expected as the national rate includes urban areas which generally have lower mortality rates. Alternative best estimate options included prospective surveillance systems, but these either did not exist in the study sites or were suspected to have poor sensitivity, as suggested by a review of camp-based surveillance. (3) We did not compare our estimates to retrospective household surveys as to our knowledge these have not been validated sufficiently to provide a reliable best estimate comparator.(2, 4, 7, 27) Moreover, retrospective mortality surveys typically estimate mortality rates averaged over several months and are therefore not directly comparable to mortality estimates from the informant method, whose main objective is to use very short recall periods.

Lastly, we were unable to test the method in settings of high mortality, and further studies are required to understand how well it would perform in such settings.

Conclusions

This study suggests that a new method to estimate recent mortality, based on information provided by community informants, while performing at least as well as other available methods in crisis affected population, still missed a substantial proportion of all deaths, particularly among children. Overall, the method's sensitivity did not reach 80%, the level we aimed for when developing this method. However, we believe the method shows sufficient promise to warrant further development given the paramount importance of real-time mortality measurement, particularly in crisis-affected populations.

Variants of our method are already being used to identify cases of maternal mortality and severe acute malnutrition in the community [START_REF]Immpact Toolkit: A guide and tools for maternal mortality programme assessment (Module 4, tool 9) -Maternal Death from Informants and Maternal Death Follow-on Review (MADE-IN / MADE-FOR)[END_REF][START_REF] Aggarwal | Risk factors for maternal mortality in Delhi slums: A communitybased case-control study[END_REF][START_REF] Singh | House-to-house survey vs. snowball technique for capturing maternal deaths in India: a search for a cost-effective method[END_REF][START_REF] Myatt | A field trial of a survey method for estimating the coverage of selective feeding programmes[END_REF], and further applications may be envisaged. These could cover supplementing or evaluating existing mortality surveillance systems (including cause-specific mortality such as maternal mortality, as recommended elsewhere [START_REF] Kim | Evaluating completeness of maternal mortality reporting in a rural health and social affairs unit in Vellore, India, 2004[END_REF] Response rate (households found and giving consent) 100% 100% 100% 98% † Two interviews were conducted on 27 July due to previous inability to contact the household. ‡ Data were not collected systematically due to the large number of mullahs contacted: a rough estimate is provided. ¶ In practice, we consulted with the section leader as well as members of his/her office, who were usually present during the visit. * In 5 (5.2%) of the 96 villages sampled in Chiradzulu district, the fumukazi was also the acting headman as the headman was away. In a further 7 villages (7.3%) the fumukazi was also the headman on a permanent basis (these are included among the 91 found). ** In 6 villages (6.3%) a deputy fumukazi was contacted as the fumukazi was away.

§ The three remaining primary informants were contacted by other primary informants and provided the required information.

  the informant method against a best estimate of mortality provided by capture-recapture analysis.

  were thus fitted (no interactions; one interaction among any two lists; two two-way interactions; and three two-way interactions, i.e. saturated). Each model predicts m abc , i.e. the quantity inside each cell of the three-list contingency table, where m is the estimator of x (the true value in the cell). From these predictions,

  were no instances in which a household was empty. One household in Tanzania did not consent.

  children under five years. The informant method also cannot be used to record data on exposure to mortality risk-factors whereas retrospective surveys can potentially record such data.

  would not be affected if these deaths were mentioned, but falsely attributed to a more socially acceptable cause.

  alongside FGDs, such as in-depth individual interviews or informal

  was worse over time: a well known problem for retrospective epidemiological studies.

Table 1 . Details of FGD participants, by study site.

 1 context of vertical disease programmes; rapidly identifying persons living with disabilities; or investigating outbreaks of diseases with very recognisable symptoms. Improving sensitivity through strengthened methods of selecting appropriate informants and eliciting their knowledge should be the focus of future research and development of this method.

	); monitoring cause-specific proportionate

Table 3 . Timeframe, population covered by the exhaustive search, and response rate, by study site.

 3

		District 1, Kabul	Mae La camp	Chiradzulu District	Tanzania camps
	Population size (age	76 476 (13 790)	43 794 (5 384)	54 418 (9 462)	80 136 (16 028)
	<5 years)
	Timeframe of data	14-27 July 2008	11-17 July 2008 †	26 August-16	3-9 October 2008
	collection			September 2008
	Number (%) of	26/26 (100%)	22/22 (100%) ¶	91/96 (94.8%)*	15/18 (83.3%) §
	primary informants
	found
	Number (%) of	≈80% ‡	22/22 (100%)	90/96 (93.8%)**	n/a (see Table 2)
	secondary informants
	found

Table 4 . Number (percentage) of deaths captured using the informant method, by informant type, recall period, and study site.

 4

	Deaths	District 1, Kabul	Mae La camp	Chiradzulu District	Tanzania camps
	captured, by
	type of
	informant
		60 days	30 days	60 days	30 days	60 days	30 days	60 days	30 days
		recall	recall	recall	recall	recall	recall	recall	recall
	Primary and	55 (82.1)	11	20	11	90 (96.8) 34	42	20
	secondary (%*)		(100.0)	(74.1)	(68.8)		(94.4)	(95.5)	(95.2)
	Respondent	11 (16.4)	0 (0.0)	9 (33.3)	4 (25.0)	6 (6.5)	5 (13.9) 9 (21.4)	4 (20.0)
	households (%*)
	Others (%*)	4 (6.0)	0 (0.0)	1 (3.7)	1 (6.3)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)
	Total deaths,	67	11	27	16	93	36	44	21
	excluding multiple reports		(16.4)		(59.2)		(38.7)		(47.7)
	(% †)

* Percentages of total deaths (excluding multiple reports) in each column †Percentage of all deaths reported in the site .

Table 5 . Demographic profile of deaths captured by the informant method and alternative lists, by recall period and study site.

 5 ) †Percentage of all deaths reported for the individual list in the study site. ‡Health Surveillance Assistants. ¶Tanzania Red Cross Society.

	Site	List 1	List 2	List 3	All lists combined
		60 days	30 days	60 days	30 days	60 days	30 days	60 days	30 days
		recall	recall	recall	recall	recall	recall	recall	recall
	District 1, Kabul	Informant method	Stores, bakeries	Hospital records	Total
	Number of deaths	67	11	33	12 (36.3)	8	6 (75.0)	82	18
	(%) †		(16.4)						(22.0)
	Median age, years	35	50	50	48	3	1	35	40
	(range)	(0-99)	(0-99)	(0-99)	(0-99)	(0-36)	(0-6)	(0-99)	(0-99)
	Number aged < 5	20	1	9	2	5 (62.5)	4	26	5
	years (% of total	(29.9)	(9.1)	(27.3)	(16.7)		(66.7)	(31.7)	(27.8)
	deaths)
	Male sex (% of total	32	4	16	6	8	6	43	10
	deaths)	(47.8)	(36.4)	(48.5)	(50.0)	(100.0)	(100.0)	(52.4)	(55.6)
	Mae La camp	Informant method	Religious leaders	Surveillance	Total
						system
	Number of deaths	27	16	23	15 (65.2)	41	18	52	23
	(%) †		(59.2)				(43.9)		(44.2)
	Median age years	55	63	55	63	47	54	49	63
	(range)	(1-87)	(21-87)	(1-85)	(45-85)	(0-87)	(0-87)	(0-87)	(0-87)
	Number aged< 5	2	0	2	0	15	4	16	4
	years (% of total	(7.4)	(0.0)	(8.7)	(0.0)	(36.6)	(22.2)	(30.8)	(17.4)
	deaths)
	Male sex (% of total	15	6	14	8	27	11	33	13
	deaths)	(55.6)	(37.5)	(60.9)	(53.3)	(65.9)	(61.1)	(63.4)	(56.5)
	Chiradzulu District	Informant method	Graveyard	Health structures,	Total
				chairmen	HSAs ‡
	Number of deaths	93	37	72	27 (37.5) 44	18	110	43
	(%) †		(39.8)				(40.9)		(39.0)
	Median age, years	35	35	34	34	35	29	35	34
	(range)	(0-96)	(0-85)	(0-96)	(0-85)	(0-96)	(0-85)	(0-96)	(0-85)
	Number aged < 5	26	12	20	9 (33.3)	11	7	30	15
	years (% of total	(28.0)	(32.4)	(27.8)		(25.0)	(38.9)	(27.3)	(35.7)
	deaths)
	Male sex (% of total	47	16	34	9 (33.3)	25	9	55	19
	deaths)	(50.5)	(43.2)	(47.2)		(56.8)	(50.0)	(50.0)	(45.2)
	Tanzania camps	Informant method	Camp register	TRCS ¶	Total
						surveillance
	Number of deaths	44	21	35	16 (45.7) 29	13	63	28
	(%) †		(47.7)				(44.8)		(44.3)
	Median age, years	4	25	2	22	2	27	4	15
	(range)	(0-98)	(0-98)	(0-87)	(0-87)	(0-78)	(0-78)	(0-98)	(0-98)
	Number aged < 5	22	8	17	6 (37.5)	17	4	33	12
	years (% of total	(50.0)	(38.1)	(48.6)		(58.6)	(30.8)	(52.4)	(42.9)
	deaths)
	Male sex (% of total	22	10	22	11 (68.8) 16	9	34	16
	deaths)	(50.0)	(47.6)	(62.9)		(55.2)	(69.2)	(54.0)	(57.1

Table 8 . Recent estimates of mortality (as deaths per 10,000 person-days) in the regions surrounding the four study sites and nationally Crude mortality rate Under 5 Mortality Rate Afghanistan Kabul

 8 District surveys (includes residents or returnees only, and rural areas in the district):

Acknowledgments

We thank the field staff and residents in the four sites. We are grateful to Megan Deitchler at FANTA; the Ministry of Public Health in Afghanistan; Annemarie ter Veen, Aurore Debellut and colleagues at Aide Médicale Internationale in Thailand; Médecins Sans Frontières-France in Malawi; the United Nations High Commissioner for Refugees; the Tanzania Red Cross Society, and Patrick Ball and colleagues at the Benetech Initiative.

Funding

This study was funded by the Office of Health, Infectious Disease, and Nutrition, Bureau for Global Health, United States Agency for International Development (USAID), through both the FANTA-2 Project under the terms of Cooperative Agreement Number GHN-A-00-08-00001-00, and the FANTA Project (1998-2008) under the terms of Cooperative Agreement Number HRN-A-00-98-00046-00, managed by the Academy for Educational Development (AED). The contents do not necessarily reflect the views of USAID or the United States Government. 1999-2002: 0.16 (Bartlett et al (20)) 2001: 0.20 2003: 0.50 2004: 0.30 2006: 0.76 National in 2008: 0.49*, 0.55** This study: 0.24 2001: 0.54 2003: 0.59 2004: 0.63 2006: 0.73 2001-2006: 0.45 (calculated based on data in Mashal et al (21)) This study: 0.49 Thailand Tak Province in 2007: 0.17 (Thailand demographic surveillance; pers. comm., Oliver Morgan (22)). National in 2008: 0.16*, 0.25** This study: 0.23 [no survey found] This study: ≥0.49 Malawi Southern and Central Regions (surveys): 2005: 0.45 2006: 0.10, 0.26, 0.30, 0.40, 0.40, 0.41, 0.41, 0.80, 0.90, 1.90, 2.20, 2.50 (median: 0.41) 2007: 0.38 National in 2008: 0.41*, 0.33** This study: 0.51 2003: 1.57 2005: 1.20 2006: 0.40, 0.78, 1.60, 2.10, 3.40, 3.80 2007: 0.71, 0.82 This study: 0.84 Tanzania [no regional surveys found] National in 2008: 0.36*, 0.30** This study: 0.18 2004: 0.72*** This study: 0.43 Unless otherwise referenced, data are as reported by the Centre for Research on the Epidemiology of Disasters' Complex Emergency Database (www.cedat.be), after excluding reports of zero mortality on plausibility grounds. * US Census Bureau International Data Base (24). ** UNICEF data using data from the UN Population Division's 'United Nations World Population Prospects: The 2008 Revision' (25). ***Data converted from rate per 1000 live births to rate per 10,000 person-days using the following calculation: (under 5 mortality rate per 1000 live births [112] * 1000 * birth rate per 1000 persons [42.4]) * (10,000 / 365) / (proportion of under 5s in the population [18.2%]). All data from the Tanzania Demographic and Health Survey 2004-5 (26).

Conflict of interest

None declared

Abbreviations

CMR Crude Mortality Rate FGD Focus Group Discussion