Validating the indicator "maternal death review coverage" to improve maternal mortality data: A retrospective analysis of district, facility, and individual medical record data.

BACKGROUND: Understanding causes and contributors to maternal mortality is critical from a quality improvement perspective to inform decision making and monitor progress toward ending preventable maternal mortality. The indicator "maternal death review coverage" is defined as the percentage of maternal deaths occurring in a facility that are audited. Both the numerator and denominator of this indicator are subject to misclassification errors, underreporting, and bias. This study assessed the validity of the indicator by examining both its numerator-the number and quality of death reviews-and denominator-the number of facility-based maternal deaths and comparing estimates of the indicator obtained from facility- versus district-level data. METHODS AND FINDINGS: We collected data on the number of maternal deaths and content of death reviews from all health facilities serving as birthing sites in 12 districts in three countries: Argentina, Ghana, and India. Additional data were extracted from health management information systems on the number and dates of maternal deaths and maternal death reviews reported from health facilities to the district-level. We tabulated the percentage of facility deaths with evidence of a review, the percentage of reviews that met the World Health Organization defined standard for maternal and perinatal death surveillance and response. Results were stratified by sociodemographic characteristics of women and facility location and type. We compared these estimates to that obtained using district-level data. and looked at evidence of the review at the district/provincial level. Study teams reviewed facility records at 34 facilities in Argentina, 51 facilities in Ghana, and 282 facilities in India. In total, we found 17 deaths in Argentina, 14 deaths in Ghana, and 58 deaths in India evidenced at facilities. Overall, >80% of deaths had evidence of a review at facilities. In India, a much lower percentage of deaths occurring at secondary-level facilities (61.1%) had evidence of a review compared to deaths in tertiary-level facilities (92.1%). In all three countries, only about half of deaths in each country had complete reviews: 58.8% (n = 10) in Argentina, 57.2% (n = 8) in Ghana, and 41.1% (n = 24) in India. Dramatic reductions in indicator value were seen in several subnational geographic areas, including Gonda and Meerut in India and Sunyani in Ghana. For example, in Gonda only three of the 18 reviews conducted at facilities met the definitional standard (16.7%), which caused the value of the indicator to decrease from 81.8% to 13.6%. Stratification by women's sociodemographic factors suggested systematic differences in completeness of reviews by women's age, place of residence, and timing of death. CONCLUSIONS: Our study assessed the validity of an important indicator for ending preventable deaths: the coverage of reviews of maternal deaths occurring in facilities in three study settings. We found discrepancies in deaths recorded at facilities and those reported to districts from facilities. Further, few maternal death reviews met global quality standards for completeness. The value of the calculated indicator masked inaccuracies in counts of both deaths and reviews and gave no indication of completeness, thus undermining the ultimate utility of the measure in achieving an accurate measure of coverage.

Validating indicators for monitoring availability and geographic distribution of emergency obstetric and newborn care (EmoNC) facilities: A study triangulating health system, facility, and geospatial data.

Availability of emergency obstetric and newborn care (EmONC) is a strong supply side measure of essential health system capacity that is closely and causally linked to maternal mortality reduction and fundamentally to achieving universal health coverage. The World Health Organization's indicator "Availability of EmONC facilities" was prioritized as a core indicator to prevent maternal death. The indicator focuses on whether there are sufficient emergency care facilities to meet the population need, but not all facilities designated as providing EmONC function as such. This study seeks to validate "Availability of EmONC" by comparing the value of the indicator after accounting for key aspects of facility functionality and an alternative measure of geographic distribution. This study takes place in four subnational geographic areas in Argentina, Ghana, and India using a census of all birthing facilities. Performance of EmONC in the 90 days prior to data collection was assessed by examining facility records. Data were collected on facility operating hours, staffing, and availability of essential medications. Population estimates were generated using ArcGIS software using WorldPop to estimate the total population, and the number of women of reproductive age (WRA), pregnancies and births in the study areas. In addition, we estimated the population within two-hours travel time of an EmONC facility by incorporating data on terrain from Open Street Map. Using these data sources, we calculated and compared the value of the indicator after incorporating data on facility performance and functionality while varying the reference population used. Further, we compared its value to the proportion of the population within two-hours travel time of an EmONC facility. Included in our study were 34 birthing facilities in Argentina, 51 in Ghana, and 282 in India. Facility performance of basic EmONC (BEmONC) and comprehensive EmONC (CEmONC) signal functions varied considerably. One facility (4.8%) in Ghana and no facility in India designated as BEmONC had performed all seven BEmONC signal functions. In Argentina, three (8.8%) CEmONC-designated facilities performed all nine CEmONC signal functions, all located in Buenos Aires Region V. Four CEmONC-designated facilities in Ghana (57.1%) and the three CEmONC-designated facilities in India (23.1%) evidenced full CEmONC performance. No sub-national study area in Argentina or India reached the target of 5 BEmONC-level facilities per 20,000 births after incorporating facility functionality yet 100% did in Argentina and 50% did in India when considering only facility designation. Demographic differences also accounted for important variation in the indicator's value. In Ghana, the total population in Tolon within 2 hours travel time of a designated EmONC facility was estimated at 99.6%; however, only 91.1% of women of reproductive age were within 2 hours travel time. Comparing the value of the indicator when calculated using different definitions reveals important inconsistencies, resulting in conflicting information about whether the threshold for sufficient coverage is met. This raises important questions related to the indicator's validity. To provide a valid measure of effective coverage of EmONC, the construct for measurement should extend beyond the most narrow definition of availability and account for functionality and geographic accessibility.

Authorization of midwives to perform basic emergency obstetric and newborn care signal functions in Argentina, Ghana, and India: A multi-country validation study of a key global maternal and newborn health indicator.

BACKGROUND: Midwives' authorization to deliver the seven basic emergency obstetric and newborn care (BEmONC) functions is a core policy indicator in global monitoring frameworks, yet little evidence supports whether such data are captured accurately, or whether authorization demonstrates convergence with midwives' skills and actual provision of services. In this study, we aimed to validate the data reported in global monitoring frameworks (criterion validity) and to determine whether a measure of authorization is a valid indicator for BEmONC availability (construct validity). METHODS: We conducted a validation study in Argentina, Ghana, and India. To assess accuracy of the reported data on midwives' authorization to provide BEmONC services, we reviewed national regulatory documents and compared with reported country-specific data in Countdown to 2030 and the World Health Organization Maternal, Newborn, Child and Adolescent Health Policy Survey. To assess whether authorization demonstrates convergent validity with midwives' skills, training, and performance of BEmONC signal functions, we surveyed 1257 midwives/midwifery professionals and assessed variance. RESULTS: We detected discrepancies between data reported in the global monitoring frameworks and the national regulatory framework in all three countries. We found wide variations between midwives' authorization to perform signal functions and their self-reported skills and actual performance within the past 90 days. The percentage of midwives who reported performing all signal functions for which they were authorized per country-specific regulations was 17% in Argentina, 23% in Ghana, and 31% in India. Additionally, midwives in all three countries reported performing some signal functions that the national regulations did not authorize. CONCLUSION: Our findings suggest limitations in criterion and construct validity for this indicator in Argentina, Ghana, and India. Some signal functions such as assisted vaginal delivery may be obsolete based on current practice patterns. Findings suggest the need to re-examine the emergency interventions that should be included as BEmONC signal functions.

Improving measures of access to legal abortion: A validation study triangulating multiple data sources to assess a global indicator.

BACKGROUND: Global mechanisms have been established to monitor and facilitate state accountability regarding the legal status of abortion. However, there is little evidence describing whether these mechanisms capture accurate data. Moreover, it is uncertain whether the "legal status of abortion" is a valid proxy measure for access to safe abortion, pursuant to the global goals of reducing preventable maternal mortality and advancing reproductive rights. Therefore, this study sought to assess the accuracy of reported monitoring data, and to determine whether evidence supports the consistent application of domestic law by health care professionals such that legality of abortion functions as a valid indicator of access. METHODS AND FINDINGS: We conducted a validation study using three countries as illustrative case examples: Argentina, Ghana, and India. We compared data reported by two global monitoring mechanisms (Countdown to 2030 and the Global Abortion Policies Database) against domestic source documents collected through in-depth policy review. We then surveyed health care professionals authorized to perform abortions about their knowledge of abortion law in their countries and their personal attitudes and practices regarding provision of legal abortion. We compared professionals' responses to the domestic legal frameworks described in the source documents to establish whether professionals consistently applied the law as written. This analysis revealed weaknesses in the criterion validity and construct validity of the "legal status of abortion" indicator. We detected discrepancies between data reported by the global monitoring and accountability mechanisms and the domestic policy reviews, even though all referenced the same source documents. Further, provider surveys unearthed important context-specific barriers to legal abortion not captured by the indicator, including conscientious objection and imposition of restrictions at the provider's discretion. CONCLUSIONS: Taken together, these findings denote weaknesses in the indicator "legal status of abortion" as a proxy for access to safe abortion, as well as inaccuracies in data reported to global monitoring mechanisms. This information provides important groundwork for strengthening indicators for monitoring access to abortion and for renewed advocacy to assure abortion rights worldwide.

Multisite, mixed methods study to validate 10 maternal health system and policy indicators in Argentina, Ghana and India: a research protocol.

INTRODUCTION: Most efforts to assess maternal health indicator validity focus on measures of service coverage. Fewer measures focus on the upstream enabling environment, and such measures are typically not research validated. Thus, methods for validating system and policy-level indicators are not well described. This protocol describes original multicountry research to be conducted in Argentina, Ghana and India, to validate 10 indicators from the monitoring framework for the 'Strategies toward Ending Preventable Maternal Mortality' (EPMM). The overall aim is to improve capacity to drive and track progress towards achieving the priority recommendations in the EPMM strategies. This work is expected to contribute new knowledge on validation methodology and reveal important information about the indicators under study and the phenomena they target for monitoring. Validating the indicators in three diverse settings will explore the external validity of results. METHODS AND ANALYSIS: This observational study explores the validity of 10 indicators from the EPMM monitoring framework via seven discrete validation exercises that will use mixed methods: (1) cross-sectional review of policy data, (2) retrospective review of facility-level patient and administrative data and (3) collection of primary quantitative and qualitative cross-sectional data from health service providers and clients. There is a specific methodological approach and analytic plan for each indicator, directed by unique, relevant validation research questions. ETHICS AND DISSEMINATION: The protocol was approved by the Office of Human Research Administration at Harvard University in November 2019. Individual study sites received approval via local institutional review boards by January 2020 except La Pampa, Argentina, approved June 2020. Our dissemination plan enables unrestricted access and reuse of all published research, including data sets. We expect to publish at least one peer-reviewed publication per validation exercise. We will disseminate results at conferences and engage local stakeholders in dissemination activities in each study country.

Measuring coverage in MNCH: validating women's self-report of emergency cesarean sections in Ghana and the Dominican Republic.

BACKGROUND: Cesarean section is the only surgery for which we have nearly global population-based data. However, few surveys provide additional data related to cesarean sections. Given weaknesses in many health information systems, health planners in developing countries will likely rely on nationally representative surveys for the foreseeable future. The objective is to validate self-reported data on the emergency status of cesarean sections among women delivering in teaching hospitals in the capitals of two contrasting countries: Accra, Ghana and Santo Domingo, Dominican Republic (DR). METHODS AND FINDINGS: This study compares hospital-based data, considered the reference standard, against women's self-report for two definitions of emergency cesarean section based on the timing of the decision to operate and the timing of the cesarean section relative to onset of labor. Hospital data were abstracted from individual medical records, and hospital discharge interviews were conducted with women who had undergone cesarean section in two hospitals. The study assessed sensitivity, specificity, and positive predictive value of responses to questions regarding emergency versus non-emergency cesarean section and estimated the percent of emergency cesarean sections that would be obtained from a survey, given the observed prevalence, sensitivity, and specificity from this study. Hospital data were matched with exit interviews for 659 women delivered via cesarean section for Ghana and 1,531 for the Dominican Republic. In Ghana and the Dominican Republic, sensitivity and specificity for emergency cesarean section defined by decision time were 79% and 82%, and 50% and 80%, respectively. The validity of emergency cesarean defined by operation time showed less favorable results than decision time in Ghana and slightly more favorable results in the Dominican Republic. CONCLUSIONS: Questions used in this study to identify emergency cesarean section are promising but insufficient to promote for inclusion in international survey questionnaires. Additional studies which confirm the accuracy of key facility-based indicators in advance of data collection and which use a longer recall period are warranted.