ABSTRACT
BACKGROUND: Small-scale community health worker (CHW) programs provide basic health services and strengthen health systems in resource-poor settings. This paper focuses on improving CHW performance by providing individual feedback to CHWs working with an mHealth program to address malnutrition in children younger than 5 years. OBJECTIVE: The paper aims to evaluate the immediate and retention effects of providing performance feedback and supportive supervision on CHW motivation and performance for CHWs working with an mHealth platform to reduce malnutrition in five districts of Madhya Pradesh, India. We expected a positive impact on CHW performance for the indicator they received feedback on. Performance on indicators the CHW did not receive feedback on was not expected to change. METHODS: In a randomized controlled trial, 60 CHWs were randomized into three treatment groups based on overall baseline performance ranks to achieve balanced treatment groups. Data for each treatment indicator were analyzed with the other two treatments acting as the control. In total, 10 CHWs were lost to follow-up. There were three performance indicators: case activity, form submissions, and duration of counseling. Each group received weekly calls to provide performance targets and discuss their performance on the specific indicator they were allocated to as well as any challenges or technical issues faced during the week for a 6-week period. Data were collected for a further 4 weeks to assess intertemporal sustained effects of the intervention. RESULTS: We found positive and significant impacts on duration of counseling, whereas case activity and number of form submissions did not show significant improvements as a result of the intervention. We found a moderate to large effect (Glass's delta=0.97, P=.004) of providing performance feedback on counseling times in the initial 6 weeks. These effects were sustained in the postintervention period (Glass's delta=1.69, P<.001). The counseling times decreased slightly from the intervention to postintervention period by 2.14 minutes (P=.01). Case activity improved for all CHWs after the intervention. We also performed the analysis by replacing the CHWs lost to follow-up with those in their treatment groups with the closest ranks in baseline performance and found similar results. CONCLUSIONS: Calls providing performance feedback are effective in improving CHW motivation and performance. Providing feedback had a positive effect on performance in the case of duration of counseling. The results suggest that difficulty in achieving the performance target can affect results of performance feedback. Regardless of the performance information disclosed, calls can improve performance due to elements of supportive supervision included in the calls encouraging CHW motivation.
ABSTRACT
BACKGROUND: mHealth apps are deployed with the aim of improving access, quality, and experience of health care. It is possible that any mHealth intervention can yield differential impacts for different types of users. Mediating and determining factors, including personal and socioeconomic factors, affect technology adoption, the way health workers leverage and use the technology, and subsequently the quality and experience of care they provide. OBJECTIVE: To develop a framework to assess whether mHealth platforms affect the quality and experience of care provided by frontline workers, and whether these effects on quality and experience are different depending on the level of technology adoption and individual characteristics of the health worker. Literacy, education, age, and previous mobile experience are identified as individual factors that affect technology adoption and use, as well as factors that affect the quality and experience of care directly and via the technology. METHODS: Formative research was conducted with 15 community health workers (CHWs) using CommCare, an mHealth app for maternal and newborn care, in Bihar, India. CHWs were first classified on the level of CommCare adoption using data from CommCareHQ and were then shadowed on home visits to evaluate their levels of technology proficiency, and the quality and experience of care provided. Regression techniques were employed to test the relationships. Out of all the CHWs, 2 of them refused to participate in the home visits, however, we did have information on their levels of technology adoption and background characteristics, which were included in the analysis as relevant. RESULTS: Level of technology adoption was important for both quality and experience of care. The quality score for high users of CommCare was higher by 33.4% (P=.04), on average, compared to low users of CommCare. Those who scored higher on CommCare proficiency also provided significantly higher quality and experience of care, where an additional point in CommCare proficiency score increased the quality score by around half a point (0.541, P=.07), and experience score by around a third of a point (0.308, P=.03). Age affected CommCare user type negatively, with an increase in age increasing the likelihood of belonging to a lower category of CommCare adoption (-0.105, P=.08). Other individual characteristics did not affect adoption or the predicted values estimating the relationship between adoption and quality and experience of care, although illiteracy was able to affect the relationship negatively. CONCLUSIONS: mHealth technology adoption by frontline workers can positively impact the quality and experience of care they provide. Individual characteristics, especially literacy and age, can be important elements affecting technology adoption and the way users leverage the technology for their work. Our formative study provides informed hypotheses and methods for further research.