Cost-effectiveness of a direct to beneficiary mobile communication programme in improving reproductive and child health outcomes in India.

INTRODUCTION: Kilkari is the largest maternal messaging programme of its kind globally. Between its initiation in 2012 in Bihar and its transition to the government in 2019, Kilkari was scaled to 13 states across India and reached over 10 million new and expectant mothers and their families. This study aims to determine the cost-effectiveness of exposure to Kilkari as compared with no exposure across 13 states in India. METHODS: The study was conducted from a programme perspective using an analytic time horizon aligned with national scale-up efforts from December 2014 to April 2019. Economic costs were derived from the financial records of implementing partners. Data on incremental changes in the practice of reproductive maternal newborn and child health (RMNCH) outcomes were drawn from an individually randomised controlled trial in Madhya Pradesh and inputted into the Lives Saved Tool to yield estimates of maternal and child lives saved. One-way and probabilistic sensitivity analyses were carried out to assess uncertainty. RESULTS: Inflation adjusted programme costs were US$8.4 million for the period of December 2014-April 2019, corresponding to an average cost of US$264 298 per year of implementation in each state. An estimated 13 842 lives were saved across 13 states, 96% among children and 4% among mothers. The cost per life saved ranged by year of implementation and with the addition of new states from US$392 ($385-$393) to US$953 ($889-$1092). Key drivers included call costs and incremental changes in coverage for key RMNCH practices. CONCLUSION: Kilkari is highly cost-effective using a threshold of India's national gross domestic product of US$1998. Study findings provide important evidence on the cost-effectiveness of a national maternal messaging programme in India. TRIAL REGISTRATION: NCT03576157.

Can we design the next generation of digital health communication programs by leveraging the power of artificial intelligence to segment target audiences, bolster impact and deliver differentiated services? A machine learning analysis of survey data from rural India.

OBJECTIVES: Direct to beneficiary (D2B) mobile health communication programmes have been used to provide reproductive, maternal, neonatal and child health information to women and their families in a number of countries globally. Programmes to date have provided the same content, at the same frequency, using the same channel to large beneficiary populations. This manuscript presents a proof of concept approach that uses machine learning to segment populations of women with access to phones and their husbands into distinct clusters to support differential digital programme design and delivery. SETTING: Data used in this study were drawn from cross-sectional survey conducted in four districts of Madhya Pradesh, India. PARTICIPANTS: Study participant included pregnant women with access to a phone (n=5095) and their husbands (n=3842) RESULTS: We used an iterative process involving K-Means clustering and Lasso regression to segment couples into three distinct clusters. Cluster 1 (n=1408) tended to be poorer, less educated men and women, with low levels of digital access and skills. Cluster 2 (n=666) had a mid-level of digital access and skills among men but not women. Cluster 3 (n=1410) had high digital access and skill among men and moderate access and skills among women. Exposure to the D2B programme 'Kilkari' showed the greatest difference in Cluster 2, including an 8% difference in use of reversible modern contraceptives, 7% in child immunisation at 10 weeks, 3% in child immunisation at 9 months and 4% in the timeliness of immunisation at 10 weeks and 9 months. CONCLUSIONS: Findings suggest that segmenting populations into distinct clusters for differentiated programme design and delivery may serve to improve reach and impact. TRIAL REGISTRATION NUMBER: NCT03576157.

Optimising the reach of mobile health messaging programmes: an analysis of system generated data for the Kilkari programme across 13 states in India.

Kilkari is an outbound service that makes weekly, stage-based, prerecorded calls about reproductive, maternal, neonatal and child health directly to families' mobile phones, starting from the second trimester of pregnancy and until the child is 1 year old. Since its initiation in 2012-2013, Kilkari has scaled to 13 states across India. In this analysis article, we explored the subscriber's journey from entry to programme to engagement with calls. Data sources included call data records and household survey data from the 2015 National Family Health Survey. In 2018, of the 13.6 million records received by MOTECH, the technology platform that powers Kilkari, 9.5 million (~70%) were rejected and 4.1 million new subscribers were created. Overall, 21% of pregnant women across 13 states were covered by the programme in 2018, with West Bengal and Himachal Pradesh reaching a coverage of over 50%. Among new subscriptions in 2018, 63% were subscribed during pregnancy and 37% after childbirth. Of these, over 80% were ever reached by Kilkari calls and 39% retained in the programme. The main causes for deactivation of subscribers from the system were low listenership and calls going unanswered for six continuous weeks. Globally, Kilkari is the largest maternal mobile messaging programme of its kind in terms of number of subscribers but the coverage among pregnant women remains low. While call reach appears to be on the higher side, subscriber retention is low; this highlights broader challenges with providing mobile health services at scale across India.

The impact of a direct to beneficiary mobile communication program on reproductive and child health outcomes: a randomised controlled trial in India.

BACKGROUND: Direct-to-beneficiary communication mobile programmes are among the few examples of digital health programmes to have scaled widely in low-resource settings. Yet, evidence on their impact at scale is limited. This study aims to assess whether exposure to mobile health information calls during pregnancy and postpartum improved infant feeding and family planning practices. METHODS: We conducted an individually randomised controlled trial in four districts of Madhya Pradesh, India. Study participants included Hindi speaking women 4-7 months pregnant (n=5095) with access to a mobile phone and their husbands (n=3842). Women were randomised to either an intervention group where they received up to 72 Kilkari messages or a control group where they received none. Intention-to-treat (ITT) and instrumental variable (IV) analyses are presented. RESULTS: An average of 65% of the 2695 women randomised to receive Kilkari listened to ≥50% of the cumulative content of calls answered. Kilkari was not observed to have a significant impact on the primary outcome of exclusive breast feeding (ITT, relative risk (RR): 1.04, 95% CI 0.88 to 1.23, p=0.64; IV, RR: 1.10, 95% CI 0.67 to 1.81, p=0.71). Across study arms, Kilkari was associated with a 3.7% higher use of modern reversible contraceptives (RR: 1.12, 95% CI 1.03 to 1.21, p=0.007), and a 2.0% lower proportion of men or women sterilised since the birth of the child (RR: 0.85, 95% CI 0.74 to 0.97, p=0.016). Higher reversible method use was driven by increases in condom use and greatest among those women exposed to Kilkari with any male child (9.9% increase), in the poorest socioeconomic strata (15.8% increase), and in disadvantaged castes (12.0% increase). Immunisation at 10 weeks was higher among the children of Kilkari listeners (2.8% higher; RR: 1.03, 95% CI 1.00 to 1.06, p=0.048). Significant differences were not observed for other maternal, newborn and child health outcomes assessed. CONCLUSION: Study findings provide evidence to date on the effectiveness of the largest mobile health messaging programme in the world. TRIAL REGISTRATION NUMBER: Trial registration clinicaltrials.gov; ID 90075552, NCT03576157.

'[We] learned how to speak with love': a qualitative exploration of accredited social health activist (ASHA) community health worker experiences of the Mobile Academy refresher training in Rajasthan, India.

INTRODUCTION: Mobile Academy is a mobile-based training course for India's accredited social health activist (ASHA) community health workers (CHW). The course, which ASHAs access by dialling a number from their phones, totals 4 hours of audio content. It consists of 11 chapters, each with their own quiz, and provides a cumulative pass or fail score at the end. This qualitative study of Mobile Academy explores how the programme was accessed and experienced by CHWs, and how they perceive it to have influenced their work. METHODS: We conducted in-depth interviews (n=25) and focus group discussions (n=5) with ASHAs and other health system actors. Open-ended questions explored ASHA perspectives on Mobile Academy, the course's perceived influence on ASHAs and preferences for future training programmes. After applying a priori codes to the transcripts, we identified emergent themes and grouped them according to our CHW mLearning framework. RESULTS: ASHAs reported enjoying Mobile Academy, specifically praising its friendly tone and the ability to repeat content. They, and higher level health systems actors, conceived it to primarily be a test not a training. ASHAs reported that they found the quizzes easy but generally did not consider the course overly simplistic. ASHAs considered Mobile Academy's content to be a useful knowledge refresher but said its primary benefit was in modelling a positive communications approach, which inspired them to adopt a kinder, more 'loving' communication style when speaking to beneficiaries. ASHAs and health system actors wanted follow-on mLearning courses that would continue to compliment but not replace face-to-face training. CONCLUSION: This mLearning programme for CHWs in India was well received by ASHAs across a wide range of education levels and experience. Dial-in audio training has the potential to reinforce topical knowledge and showcase positive ways to communicate.

Assessing the reliability of phone surveys to measure reproductive, maternal and child health knowledge among pregnant women in rural India: a feasibility study.

OBJECTIVES: Efforts to understand the factors influencing the uptake of reproductive, maternal, newborn, child health and nutrition (RMNCH&N) services in high disease burden low-resource settings have often focused on face-to-face surveys or direct observations of service delivery. Increasing access to mobile phones has led to growing interest in phone surveys as a rapid, low-cost alternatives to face-to-face surveys. We assess determinants of RMNCH&N knowledge among pregnant women with access to phones and examine the reliability of alternative modalities of survey delivery. PARTICIPANTS: Women 5-7 months pregnant with access to a phone. SETTING: Four districts of Madhya Pradesh, India. DESIGN: Cross-sectional surveys administered face-to-face and within 2 weeks, the same surveys were repeated among two random subsamples of the original sample: face-to-face (n=205) and caller-attended telephone interviews (n=375). Bivariate analyses, multivariable linear regression, and prevalence and bias-adjusted kappa scores are presented. RESULTS: Knowledge scores were low across domains: 52% for maternal nutrition and pregnancy danger signs, 58% for family planning, 47% for essential newborn care, 56% infant and young child feeding, and 58% for infant and young child care. Higher knowledge (≥1 composite score) was associated with older age; higher levels of education and literacy; living in a nuclear family; primary health decision-making; greater attendance in antenatal care and satisfaction with accredited social health activist services. Survey questions had low inter-rater and intermodal reliability (kappa<0.70) with a few exceptions. Questions with the lowest reliability included true/false questions and those with unprompted, multiple response options. Reliability may have been hampered by the sensitivity of the content, lack of privacy, enumerators' and respondents' profile differences, rapport, social desirability bias, and/or enumerator's ability to adequately convey concepts or probe. CONCLUSIONS: Phone surveys are a reliable modality for generating population-level estimates data about pregnant women's knowledge, however, should not be used for individual-level tracking. TRIAL REGISTRATION NUMBER: NCT03576157.

At the frontlines of digitisation: a qualitative study on the challenges and opportunities in maintaining accurate, complete and timely digital health records in India's government health system.

OBJECTIVES: To understand factors underpinning the accuracy and timeliness of mobile phone numbers and other health information captured in India's government registry for pregnant and postpartum women. Accurate and timely registration of mobile phone numbers is necessary for beneficiaries to receive mobile health services. SETTING: Madhya Pradesh and Rajasthan states in India at the community, clinical, and administrative levels of the health system. PARTICIPANTS: Interviews (n=59) with frontline health workers (FLHWs), data entry operators, and higher level officials. Focus group discussions (n=12) with pregnant women to discuss experiences with sharing data in the health system. Observations (n=9) of the process of digitization and of interactions between stakeholders for data collection. PRIMARY AND SECONDARY OUTCOME MEASURES: Thematic analysis identified how key actors experienced the data collection and digitisation process, reasons for late or inaccurate data, and mechanisms that can bolster timeliness and accuracy. RESULTS: Pregnant women were comfortable sharing mobile numbers with health workers, but many were unaware that their data moved beyond their FLHW. FLHWs valued knowing up-to-date beneficiary mobile numbers, but felt little incentive to ensure accuracy in the digital record system. Delays in registering pregnant women in the online portal were attributed to slow movement of paper records into the digital system and difficulties in gathering required documents from beneficiaries. Data, including women's phone numbers, were handwritten and copied multiple times by beneficiaries and health workers with variable literacy. Supervision tended to focus on completeness rather than accuracy. Health system actors noted challenges with the digital system but valued the broader project of digitisation. CONCLUSIONS: Increased focus on training, supportive supervision, and user-friendly data processes that prioritise accuracy and timeliness should be considered. These inputs can build on existing positive patient-provider relationships and health system actors' enthusiasm for digitisation.

Using behavioural design and theories of change to integrate communication solutions into health systems in India: evolution, evidence and learnings from practice.

Between 2011 and 2019, an integrated communication programme to address reproductive, maternal, neonatal and child health was implemented in the Indian state of Bihar. Along with mass media, community events and listening groups, four mobile health services were co-designed with the government of Bihar. These were Mobile Academy-a training course for frontline health workers (FLHWs) supporting them as the last mile of the health system; Mobile Kunji-a job aid to support FLHWs' interactions with families; Kilkari-a maternal messaging service delivering information directly to families' mobile phones, encouraging families to seek public health services through their FLHWs; and GupShup Potli-mobile audio stimulus used by FLHWs in community events. While Mobile Kunji and GupShup Potli scaled to other states (two and one, respectively), neither was adopted nationally. The Government of India adopted Kilkari and Mobile Academy and scaled to 12 additional states by 2019. In this article, we describe the programme's overarching person-centred theory of change, reflect on how the mHealth services supported integration with the health system and discuss implications for the role of health communication solutions in supporting families to navigate healthcare systems. Evaluations of Kunji, Academy and GupShup Potli were conducted in Bihar between 2013 and 2017. Between 2018-2020, an independent evaluation was conducted involving a randomised controlled trial for Kilkari in Madhya Pradesh; qualitative research on Kilkari and Academy and secondary analyses of call record data. While the findings from these evaluations are described elsewhere, this article collates key findings for all the services and offers implications for the role digital and non-digital communication solutions can play in supporting joined-up healthcare and improving health outcomes.

Avoiding the Road to Nowhere: Policy Insights on Scaling up and Sustaining Digital Health.

Digital health solutions offer tremendous potential to enhance the reach and quality of health services and population-level outcomes in low- and middle-income countries (LMICs). While the number of programs reaching scale increases yearly, the long-term sustainability for most remains uncertain. In this article, as researchers and implementors, we draw on experiences of designing, implementing and evaluating digital health solutions at scale in Africa and Asia, and provide examples from India and South Africa to illustrate ten considerations to support scale and sustainability of digital health solutions in LMICs. Given the investments being made in digital health solutions and the urgent concurrent needs to strengthen health systems to ensure their responsiveness to marginalized populations in LMICs, we cannot afford to go down roads that 'lead to nowhere'. These ten considerations focus on drivers of equity and innovation, the foundations for a digital health ecosystem, and the elements for systems integration. We urge technology enthusiasts to consider these issues before and during the roll-out of large-scale digital health initiatives to navigate the complexities of achieving scale and enabling sustainability.

Freedom within a cage: how patriarchal gender norms limit women's use of mobile phones in rural central India.

INTRODUCTION: India has one of the highest gender gaps in mobile phone access in the world. As employment opportunities, health messaging (mHealth), access to government entitlements, banking, civic participation and social engagement increasingly take place in the digital sphere, this gender gap risks further exacerbating women's disadvantage in Indian society. This study identifies the factors driving women's unequal use of phones in rural Madhya Pradesh, India. METHODS: We interviewed mothers of 1-year-old children (n=29) who reported that they had at least some access to a mobile phone. Whenever possible, we also spoke to their husbands (n=23) and extended family members (n=34) through interviews or family group discussions about the use of phones in their households, as well as their perspectives on gender and phone use more broadly. Our analysis involved comparing wife-husband pairs to assess differences in phone access and use, and thematic coding on the determinants of women's phone use using an iteratively developed conceptual framework. RESULTS: While respondents reported that women could use the phone without needing permission, this apparent 'freedom' existed in a context that severely constrained women's actual use, most directly through: (1) narrow expectations and desires around how women would use phones, (2) women's dependence on men for phone ownership and lower proximity to phones, (3) the poorer functionality of women's phones; (4) women's limited digital skills, and (5) time allocation constraints, wherein women had less leisure time and were subject to social norms that discouraged using a phone for leisure. CONCLUSION: Our framework, presenting the distal and proximate determinants of women's phone use, enables more nuanced understanding of India's digital divide. Addressing these determinants is vital to shift from re-entrenching unequal gender relations to transforming them through digital technology.

Examining the reach and exposure of a mobile phone-based training programme for frontline health workers (ASHAs) in 13 states across India.

Mobile phones are increasingly used to facilitate in-service training for frontline health workers (FLHWs). Mobile learning (mLearning) programmes have the potential to provide FLHWs with high quality, inexpensive, standardised learning at scale, and at the time and location of their choosing. However, further research is needed into FLHW engagement with mLearning content at scale, a factor which could influence knowledge and service delivery. Mobile Academy is an interactive voice response training course for FLHWs in India, which aims to improve interpersonal communication skills and refresh knowledge of preventative reproductive, maternal, neonatal and child health. FLHWs dial in to an audio course consisting of 11 chapters, each with a 4-question true/false quiz, resulting in a cumulative pass/fail score. In this paper, we analyse call data records from the national version of Mobile Academy to explore coverage, user engagement and completion. Over 158 596 Accredited Social Health Activists (ASHAs) initiated the national version, while 111 994 initiated the course on state-based platforms. Together, this represents 41% of the estimated total number of ASHAs registered in the government database across 13 states. Of those who initiated the national version, 81% completed it; and of those, over 99% passed. The initiation and completion rates varied by state, with Rajasthan having the highest initiation rate. Many ASHAs made multiple calls in the afternoons and evenings but called in for longer durations earlier in the day. Findings from this analysis provide important insights into the differential reach and uptake of the programme across states.

Assessing exposure to Kilkari: a big data analysis of a large maternal mobile messaging service across 13 states in India.

The Kilkari programme is being implemented by the Government of India in 13 states. Designed by BBC Media Action and scaled in collaboration with the Ministry of Health and Family Welfare from January 2016, Kilkari had provided mobile health information to over 10 million subscribers by the time BBC Media Action transitioned the service to the government in April 2019. Despite the reach of Kilkari in terms of the absolute number of subscribers, no longitudinal analysis of subscriber exposure to health information content over time has been conducted, which may underpin effectiveness and changes in health outcomes. In this analysis, we draw from call data records to explore exposure to the Kilkari programme in India for the 2018 cohort of subscribers. We start by assessing the timing of the first successful call answered by subscribers on entry to the programme during pregnancy or postpartum, and then assess call volume, delivery, answering and listening rates over time. Findings suggest that over half of subscribers answer their first call after childbirth, with the remaining starting in the pregnancy period. The system handles upwards of 1.2 million calls per day on average. On average, 50% of calls are picked up on the first call attempt, 76% by the third and 99.5% by the ninth call attempt. Among calls picked up, over 48% were listened to for at least 50% of the total content duration and 43% were listened to for at least 75%. This is the first analysis of its kind of a maternal mobile messaging programme at scale in India. Study analyses suggest that multiple call attempts may be required to reach subscribers. However, once answered, subscribers tend to listen the majority of the call-a figure consistent across states, over time, and by health content area.

Ten lessons learnt: scaling and transitioning one of the largest mobile health communication programmes in the world to a national government.

There has been exponential growth in the numbers of 'digital development' programmes seeking to leverage technology to solve systemic challenges. However, despite promising results and a shift from pilots to scale-ups, many have failed to realise their full potential. This paper reflects on lessons learnt from scaling and transitioning one of the largest mobile health programmes in the world to the Indian government. The complementary suite of services was designed by BBC Media Action to strengthen families' reproductive, maternal, neonatal and child health behaviours. Mobile Academy was a training course to refresh frontline health workers' (FLHWs) knowledge and improve their interpersonal communication skills. Mobile Kunji was a job aid to support FLHWs' interactions with families. Kilkari delivered weekly audio information to families' phones to reinforce FLHWs' counselling. As of April 2019, when Mobile Academy and Kilkari were transitioned to the government, 206 000 FLHWs had graduated and Kilkari had reached 10 million subscribers. Lessons learnt include the following: (1) private sector business models are challenging in low-resource settings; (2) you may pilot 'apples' but scale 'oranges'; (3) trade-offs are required between ideal solution design and affordability; (4) programme components should be reassessed before scaling; (5) operational viability at scale is a prerequisite for sustainability; (6) consider the true cost of open-source software; (7) taking informed consent in low-resource settings is challenging; (8) big data offer promise, but social norms and SIM change constrain use; (9) successful government engagements require significant capacity; (10) define governance structures and roadmaps up front.

Does exposure to health information through mobile phones increase immunisation knowledge, completeness and timeliness in rural India?

INTRODUCTION: Immunisation plays a vital role in reducing child mortality and morbidity against preventable diseases. As part of a randomised controlled trial in rural Madhya Pradesh, India to assess the impact of Kilkari, a maternal messaging programme, we explored determinants of parental immunisation knowledge and immunisation practice (completeness and timeliness) for children 0-12 months of age from four districts in Madhya Pradesh. METHODS: Data were drawn from a cross-sectional survey of women (n=4423) with access to a mobile phone and their spouses (n=3781). Parental knowledge about immunisation and their child's receipt of vaccines, including timeliness and completeness, was assessed using self-reports and vaccination cards. Ordered logistic regressions were used to analyse the factors associated with parental immunisation knowledge. A Heckman two-stage probit model was used to analyse completeness and timeliness of immunisation after correcting for selection bias from being able to produce the immunisation card. RESULTS: One-third (33%) of women and men knew the timing for the start of vaccinations, diseases linked to immunisations and the benefits of Vitamin-A. Less than half of children had received the basic package of 8 vaccines (47%) and the comprehensive package of 19 vaccines (44%). Wealth was the most significant determinant of men's knowledge and of the child receiving complete and timely immunisation for both basic and comprehensive packages. Exposure to Kilkari content on immunisation was significantly associated with an increase in men's knowledge (but not women's) about child immunisation (OR: 1.23, 95% CI 1.02 to1.48) and an increase in the timeliness of the child receiving vaccination at birth (Probit coefficient: 0.08, 95% CI 0.08 to 0.24). CONCLUSION: Gaps in complete and timely immunisation for infants persist in rural India. Mobile messaging programmes, supported by mass media messages, may provide one important source for bolstering awareness, uptake and timeliness of immunisation services. TRIAL REGISTRATION NUMBER: NCT03576157.

Can health information through mobile phones close the divide in health behaviours among the marginalised? An equity analysis of Kilkari in Madhya Pradesh, India.

Kilkari is one of the largest maternal mobile messaging programmes in the world. It makes weekly prerecorded calls to new and expectant mothers and their families from the fourth month of pregnancy until 1-year post partum. The programme delivers reproductive, maternal, neonatal and child health information directly to subscribers' phones. However, little is known about the reach of Kilkari among different subgroups in the population, or the differentiated benefits of the programme among these subgroups. In this analysis, we assess differentials in eligibility, enrolment, reach, exposure and impact across well-known proxies of socioeconomic position-that is, education, caste and wealth. Data are drawn from a randomised controlled trial (RCT) in Madhya Pradesh, India, including call data records from Kilkari subscribers in the RCT intervention arm, and the National Family Health Survey-4, 2015. The analysis identifies that disparities in household phone ownership and women's access to phones create inequities in the population eligible to receive Kilkari, and that among enrolled Kilkari subscribers, marginalised caste groups and those without education are under-represented. An analysis of who is left behind by such interventions and how to reach those groups through alternative communication channels and platforms should be undertaken at the intervention design phase to set reasonable expectations of impact. Results suggest that exposure to Kilkari has improved levels of some health behaviours across marginalised groups but has not completely closed pre-existing gaps in indicators such as wealth and education.

Implementing health communication tools at scale: mobile audio messaging and paper-based job aids for front-line workers providing community health education to mothers in Bihar, India.

INTRODUCTION: As part of an investment by the Bill & Melinda Gates Foundation to support the Government of Bihar to improve reproductive, maternal, newborn and child health and nutrition (RMNCHN) statewide, BBC Media Action implemented multiple communication tools to support front-line worker (FLW) outreach. We analyse the impacts of a package of mHealth audio messaging and paper-based job aids used by FLWs during government-sponsored village health, sanitation and nutrition days (VHSNDs) on knowledge and practices of childbearing women across the RMNCHN continuum of care. METHODS: Data from two surveys collected between July and September 2016 were analysed using logistic regression to compare health-related knowledge and behaviours between women who had been exposed at VHSNDs to the mHealth GupShup Potli (GSP) audio recordings or interpersonal communication (IPC) tools versus those who were unexposed. RESULTS: Exposure to GSP recordings (n=2608) was associated with improved knowledge across all continuum-of-care domains, as well as improved health-related behaviours in some domains. The odds of having taken iron-folic acid (IFA) tablets were significantly higher in exposed women (OR 1.5, 95% CI 1.1 to 2.2), as was contraceptive use (OR 2.0, 95% CI 1.2 to 3.2). There were no differences in birth preparedness or complementary feeding practices between groups. Exposure to IPC paper-based tools (n=2002) was associated with a twofold increased odds of IFA consumption (OR 2.3, 95% CI 1.7 to 3.2) and contraceptive use (OR 1.8, 95% CI 1.2 to 2.8). Women exposed to both tools were generally at least twice as likely to subsequently discuss the messages with others. CONCLUSION: BBC Media Action's mHealth audio messaging job aids and paper-based IPC tools were associated with improved knowledge and practices of women who were exposed to them across multiple domains, suggesting their important potential for improving health outcomes for beneficiaries at scale in low-resource settings. TRIAL REGISTRATION NUMBER: NCT02726230.

Another voice in the crowd: the challenge of changing family planning and child feeding practices through mHealth messaging in rural central India.

INTRODUCTION: Kilkari is one of the world's largest mobile phone-based health messaging programmes. Developed by BBC Media Action, it provides weekly stage-based information to pregnant and postpartum women and their families, including on infant and young child feeding (IYCF) and family planning, to compliment the efforts of frontline health workers. The quantitative component of a randomised controlled trial (RCT) in the Indian state of Madhya Pradesh found that exposure to Kilkari increased modern contraceptive uptake but did not change IYCF practices. This qualitative research complements the RCT to explore why these findings may have emerged. METHODS: We used system generated data to identify households within the RCT with very high to medium Kilkari listenership. Mothers (n=29), as well as husbands and extended family members (n=25 interviews/family group discussions) were interviewed about IYCF and family planning, including their reactions to Kilkari's calls on these topics. Analysis was informed by the theory of reciprocal determinism, which positions behaviour change within the interacting domains of individual attributes, social and environmental determinants, and existing practices. RESULTS: While women who owned and controlled their own phones were the Kilkari listeners, among women who did not own their own phones, it was often their husbands who listened. Spouses did not discuss Kilkari messages. Respondents retained and appreciated Kilkari messages that aligned with their pre-existing worldviews, social norms, and existing practices. However, they overlooked or de-emphasised content that did not. In this way, they reported agreeing with and trusting Kilkari while persisting with practices that went against Kilkari's recommendations, particularly non-exclusive breastfeeding and inappropriate complementary feeding. CONCLUSION: To deepen impact, digital direct to beneficiary services need to be complimented by wider communication efforts (e.g., sustained face-to-face, media, community engagement) to change social norms, taking into account the role of socio-environmental, behavioural, and individual determinants.

Novel approaches to measuring knowledge among frontline health workers in India: Are phone surveys a reliable option?

BACKGROUND: In 2017, India was home to nearly 20% of maternal and child deaths occurring globally. Accredited social health activists (ASHAs) act as the frontline for health services delivery in India, providing a range of reproductive, maternal, newborn, child health, and nutrition (RMNCH&N) services. Empirical evidence on ASHAs' knowledge is limited, yet is a critical determinant of the quality of health services provided. We assessed the determinants of RMNCH&N knowledge among ASHAs and examined the reliability of alternative modalities of survey delivery, including face-to-face and caller attended telephone interviews (phone surveys) in 4 districts of Madhya Pradesh, India. METHODS: We carried out face-to-face surveys among a random cross-sectional sample of ASHAs (n = 1,552), and administered a follow-up test-retest survey within 2 weeks of the initial survey to a subsample of ASHAs (n = 173). We interviewed a separate sub-sample of ASHAs 2 weeks of the face-to-face interview over the phone (n = 155). Analyses included bivariate analyses, multivariable linear regression, and prevalence and bias adjusted kappa analyses. FINDINGS: The average ASHA knowledge score was 64% and ranged across sub-domains from 71% for essential newborn care, 71% for WASH/ diarrhea, 64% for infant feeding, 61% for family planning, and 60% for maternal health. Leading determinants of knowledge included geographic location, age <30 years of age, education, experience as an ASHA, completion of seven or more client visits weekly, phone ownership and use as a communication tool for work, as well as the ability to navigate interactive voice response prompts (a measure of digital literacy). Efforts to develop a phone survey tool for measuring knowledge suggest that findings on inter-rater and inter-modal reliability were similar. Reliability was higher for shorter, widely known questions, including those about timing of exclusive breastfeeding or number of tetanus shots during pregnancy. Questions with lower reliability included those on sensitive topics such as family planning; questions with multiple response options; or which were difficult for the enumerator to convey. CONCLUSIONS: Overall results highlight important gaps in the knowledge of ASHAs. Findings on the reliability of phone surveys led to the development of a tool, which can be widely used for the routine, low cost measurement of ASHA RMNCH&N knowledge in India.

Impact of mHealth interventions for reproductive, maternal, newborn and child health and nutrition at scale: BBC Media Action and the Ananya program in Bihar, India.

BACKGROUND: Mobile health (mHealth) tools have potential for improving the reach and quality of health information and services through community health workers in low- and middle-income countries. This study evaluates the impact of an mHealth tool implemented at scale as part of the statewide reproductive,maternal, newborn and child health and nutrition (RMNCHN) program in Bihar, India. METHODS: Three survey-based data sets were analysed to compare the health-related knowledge, attitudes and behaviours amongst childbearing women exposed to the Mobile Kunji and Dr. Anita mHealth tools during their visits with frontline workers compared with those who were unexposed. RESULTS: An evaluation by Mathematica (2014) revealed that exposure to Mobile Kunji and Dr. Anita recordings were associated with significantly higher odds of consuming iron-folic acid tablets (odds ratio (OR) = 2.3, 95% confidence interval (CI) = 1.8-3.1) as well as taking a set of three measures for delivery preparedness (OR = 2.8, 95% CI = 1.9-4.2) and appropriate infant complementary feeding (OR = 1.9, 95% CI = 1.0-3.5). CARE India's Community-based Household Surveys (2012-2017) demonstrated significant improvements in early breastfeeding (OR = 1.64, 95% CI = 1.5-1.78) and exclusive breastfeeding (OR = 1.46, 95% CI = 1.33-1.62) in addition to birth preparedness practices. BBC Media Action's Usage & Engagement Survey (2014) demonstrated a positive association between exposure to Mobile Kunji and Dr. Anita and exclusive breastfeeding (58% exposed vs 43% unexposed, P < 0.01) as well as maternal respondents' trust in their frontline worker. CONCLUSIONS: Significant improvements in RMNCHN-related knowledge and behaviours were observed for Bihari women who were exposed to Mobile Kunji and Dr. Anita. This analysis is unique in its rigorous evaluation across multiple data sets of mHealth interventions implemented at scale. These results can help inform global understanding of how best to use mHealth tools, for whom, and in what contexts. STUDY REGISTRATION: ClinicalTrials.gov number NCT02726230.

Improving primary health care delivery in Bihar, India: Learning from piloting and statewide scale-up of Ananya.

In 2010, the Bill and Melinda Gates Foundation (BMGF) partnered with the Government of Bihar (GoB), India to launch the Ananya program to improve reproductive, maternal, newborn and child health and nutrition (RMNCHN) outcomes. The program sought to address supply- and demand-side barriers to the adoption, coverage, quality, equity and health impact of select RMNCHN interventions. Approaches included strengthening frontline worker service delivery; social and behavior change communications; layering of health, nutrition and sanitation into women's self-help groups (SHGs); and quality improvement in maternal and newborn care at primary health care facilities. Ananya program interventions were piloted in approximately 28 million population in eight innovation districts from 2011-2013, and then beginning in 2014, were scaled up by the GoB across the rest of the state's population of 104 million. A Bihar Technical Support Program provided techno-managerial support to governmental Health as well as Integrated Child Development Services, and the JEEViKA Technical Support Program supported health layering and scale-up of the GoB's SHG program. The level of support at the block level during statewide scale-up in 2014 onwards was approximately one-fourth that provided in the pilot phase of Ananya in 2011-2013. This paper - the first manuscript in an 11-manuscript and 2-viewpoint collection on Learning from Ananya: Lessons for primary health care performance improvement - seeks to provide a broad description of Ananya and subsequent statewide adaptation and scale-up, and capture the background and context, key objectives, interventions, delivery approaches and evaluation methods of this expansive program. Subsequent papers in this collection focus on specific intervention delivery platforms. For the analyses in this series, Stanford University held key informant interviews and worked with the technical support and evaluation grantees of the Ananya program, as well as leadership from the India Country Office of the BMGF, to analyse and synthesise data from multiple sources. Capturing lessons from the Ananya pilot program and statewide scale-up will assist program managers and policymakers to more effectively design and implement RMNCHN programs at scale through technical assistance to governments.