Measuring the performance of computer vision artificial intelligence to interpret images of HIV self-testing results.

Introduction: HIV self-testing (HIVST) is highly sensitive and specific, addresses known barriers to HIV testing (such as stigma), and is recommended by the World Health Organization as a testing option for the delivery of HIV pre-exposure prophylaxis (PrEP). Nevertheless, HIVST remains underutilized as a diagnostic tool in community-based, differentiated HIV service delivery models, possibly due to concerns about result misinterpretation, which could lead to inadvertent onward transmission of HIV, delays in antiretroviral therapy (ART) initiation, and incorrect initiation on PrEP. Ensuring that HIVST results are accurately interpreted for correct clinical decisions will be critical to maximizing HIVST's potential. Early evidence from a few small pilot studies suggests that artificial intelligence (AI) computer vision and machine learning could potentially assist with this task. As part of a broader study that task-shifted HIV testing to a new setting and cadre of healthcare provider (pharmaceutical technologists at private pharmacies) in Kenya, we sought to understand how well AI technology performed at interpreting HIVST results. Methods: At 20 private pharmacies in Kisumu, Kenya, we offered free blood-based HIVST to clients ≥18 years purchasing products indicative of sexual activity (e.g., condoms). Trained pharmacy providers assisted clients with HIVST (as needed), photographed the completed HIVST, and uploaded the photo to a web-based platform. In real time, each self-test was interpreted independently by the (1) client and (2) pharmacy provider, with the HIVST images subsequently interpreted by (3) an AI algorithm (trained on lab-captured images of HIVST results) and (4) an expert panel of three HIVST readers. Using the expert panel's determination as the ground truth, we calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for HIVST result interpretation for the AI algorithm as well as for pharmacy clients and providers, for comparison. Results: From March to June 2022, we screened 1,691 pharmacy clients and enrolled 1,500 in the study. All clients completed HIVST. Among 854 clients whose HIVST images were of sufficient quality to be interpretable by the AI algorithm, 63% (540/854) were female, median age was 26 years (interquartile range: 22-31), and 39% (335/855) reported casual sexual partners. The expert panel identified 94.9% (808/854) of HIVST images as HIV-negative, 5.1% (44/854) as HIV-positive, and 0.2% (2/854) as indeterminant. The AI algorithm demonstrated perfect sensitivity (100%), perfect NPV (100%), and 98.8% specificity, and 81.5% PPV (81.5%) due to seven false-positive results. By comparison, pharmacy clients and providers demonstrated lower sensitivity (93.2% and 97.7% respectively) and NPV (99.6% and 99.9% respectively) but perfect specificity (100%) and perfect PPV (100%). Conclusions: AI computer vision technology shows promise as a tool for providing additional quality assurance of HIV testing, particularly for catching Type II error (false-negative test interpretations) committed by human end-users. We discuss possible use cases for this technology to support differentiated HIV service delivery and identify areas for future research that is needed to assess the potential impacts-both positive and negative-of deploying this technology in real-world HIV service delivery settings.

Voluntary medical male circumcision in selected provinces in South Africa: Outcomes from a programmatic setting.

INTRODUCTION: Voluntary medical male circumcision (VMMC) remains an effective biomedical intervention for HIV prevention in high HIV prevalence countries. In South Africa, United States Agency for International Development VMMC partners provide technical assistance to the Department of Health, at national and provincial levels in support of the establishment of VMMC sites as well as in providing direct VMMC services at site level since April 2012. We describe the outcomes of the Right to Care (RTC) VMMC program implemented in South Africa from 2012 to 2017. METHODS: This retrospective study was undertaken at RTC supported facilities across six provinces. Young males aged ≥10 years who presented at these facilities from 1 July 2012 to 31 September 2017 were included. Outcomes were VMMC uptake, HIV testing uptake and rate of adverse events (AEs). Using a de-identified observational database of these clients, summary statistics of the demographic characteristics and outcomes were calculated. RESULTS: There were a total 1,001,226 attendees of which 998,213 (99.7%) were offered VMMC and had a median age of 15 years (IQR = 12-23 years). Of those offered VMMC, 99.6% (994,293) consented, 96.7% (965,370) were circumcised and the majority (46.3%) were from Gauteng province. HIV testing uptake was 71% with a refusal rate of 15%. Of the newly diagnosed HIV positives, 64% (6,371 / 9,972) referrals were made. The rate of AEs, defined as bleeding, infection, and insufficient skin removal) declined from 3.26% in 2012 to 1.17% in 2017. There was a reduction in infection-related AEs from 2,448 of the 2,602 adverse events (94.08%) in 2012 to 129 of the 2,069 adverse events (6.23%) in 2017. CONCLUSION: There was a high VMMC uptake with a decline in AEs over time. Adolescent men contributed the most to the circumcised population, an indication that the young population accesses medical circumcision more. VMMC programs need to implement innovative demand creation strategies to encourage older males (20-34 years) at higher risk of HIV acquisition to get circumcised for immediate impact in reduction of HIV incidence. HIV prevalence in the total population increased with increasing age, notably in clients above 25 years.

Advertising for Demand Creation for Voluntary Medical Male Circumcision.

OBJECTIVE: To measure the effects of information, a challenge, and a conditional cash transfer on take-up of voluntary medical male circumcision (VMMC). DESIGN: A randomized, controlled experiment with 4000 postcard recipients in Soweto (Johannesburg), South Africa. METHODS: We examined differences in take-up of several decisions in the VMMC cascade between the control arm and each of several intervention arms using logistic regression. RESULTS: Logistic regression analysis indicated that the group offered US $10 as compensation and the group challenged with "Are you tough enough?" had significantly higher take-up of the VMMC procedure than did the control group [odds ratios, respectively, 5.30 (CI: 2.20 to 12.76) and 2.70 (CI: 1.05 to 6.91)]. Similarly, the compensation group had significantly higher take-up of the VMMC counseling session than did the control group [odds ratio 3.76 (CI: 1.79 to 7.89)]. The analysis did not reveal significantly different take-up of either the VMMC counseling session or the procedure in the partner preference information group compared with the control group [odds ratios, respectively, 1.23 (CI: 0.51 to 2.97) and 1.67 (CI: 0.61 to 4.62)]. The analysis did not reveal significantly higher take-up of the VMMC nurse hotline in any intervention group compared with the control group [odds ratios for US $10, information, and challenge, respectively, 1.17 (CI: 0.67 to 2.07), 0.69 (CI: 0.36 to 1.32), and 0.60 (0.31 to 1.18)]. CONCLUSIONS: Among adult males in Soweto, South Africa, compensation of US $10 provided conditional on completing the VMMC counseling session compared with no compensation offer and a postcard with a challenge, "Are you tough enough?" compared with no challenge, resulted in moderate increases in take-up of circumcision.

Quality of voluntary medical male circumcision services during scale-up: a comparative process evaluation in Kenya, South Africa, Tanzania and Zimbabwe.

BACKGROUND: The rapid expansion of voluntary medical male circumcision (VMMC) has raised concerns whether health systems can deliver and sustain VMMC according to minimum quality criteria. METHODS AND FINDINGS: A comparative process evaluation was used to examine data from SYMMACS, the Systematic Monitoring of the Voluntary Medical Male Circumcision Scale-Up, among health facilities providing VMMC across two years of program scale-up. Site-level assessments examined the availability of guidelines, supplies and equipment, infection control, and continuity of care services. Direct observation of VMMC surgeries were used to assess care quality. Two sample tests of proportions and t-tests were used to examine differences in the percent of facilities meeting requisite preparedness standards and the mean number of directly-observed surgical tasks performed correctly. Results showed that safe, high quality VMMC can be implemented and sustained at-scale, although substantial variability was observed over time. In some settings, facility preparedness and VMMC service quality improved as the number of VMMC facilities increased. Yet, lapses in high performance and expansion of considerably deficient services were also observed. Surgical tasks had the highest quality scores, with lower performance levels in infection control, pre-operative examinations, and post-operative patient monitoring and counseling. The range of scale-up models used across countries additionally underscored the complexity of delivering high quality VMMC. CONCLUSIONS: Greater efforts are needed to integrate VMMC scale-up and quality improvement processes in sub-Saharan African settings. Monitoring of service quality, not just adverse events reporting, will be essential in realizing the full health impact of VMMC for HIV prevention.

Implications of the fast-evolving scale-up of adult voluntary medical male circumcision for quality of services in South Africa.

BACKGROUND: The scale-up of voluntary medical male circumcision (VMMC) services in South Africa has been rapid, in an attempt to achieve the national government target of 4.3 million adult male circumcisions for HIV prevention by 2016. This study assesses the effect of the scale-up on the quality of the VMMC program. METHODS AND FINDINGS: This analysis compares the quality of services at 15 sites operational in 2011 to (1) the same 15 sites in 2012 and (2) to a set of 40 sites representing the expanded program in 2012. Trained clinicians scored each site on 29 items measuring readiness to provide quality services (abbreviated version of the WHO Quality Assessment [QA] Guide) and 29 items to assess quality of surgical care provided (pre-op, surgical technique and post-op) based on the observation of VMMC procedures at each site. Declines in quality far outnumbered improvements. The negative effects in terms of readiness to provide quality services were most evident in expanded sites, whereas the declines in provision of quality services tended to affect both repeat sites and expanded sites equally. Areas of notable concern included the monitoring of adverse events, external supervision, post-operative counselling, and some infection control issues. Scores on quality of surgical technique tended to be among the highest across the 58 items observed, and the South Africa program has clearly institutionalized three "best practices" for surgical efficiency. CONCLUSIONS: These findings demonstrate the challenges of rapidly developing large numbers of new VMMC sites with the necessary equipment, supplies, and protocols. The scale-up in South Africa has diluted human resources, with negative effects for both the original sites and the expanded program.

Systematic Monitoring of Voluntary Medical Male Circumcision Scale-up: adoption of efficiency elements in Kenya, South Africa, Tanzania, and Zimbabwe.

BACKGROUND: SYMMACS, the Systematic Monitoring of the Voluntary Medical Male Circumcision Scale-up, tracked the implementation and adoption of six elements of surgical efficiency-use of multiple surgical beds, pre-bundled kits, task shifting, task sharing, forceps-guided surgical method, and electrocautery--as standards of surgical efficiency in Kenya, South Africa, Tanzania, and Zimbabwe. METHODS AND FINDINGS: This multi-country study used two-staged sampling. The first stage sampled VMMC sites: 73 in 2011, 122 in 2012. The second stage involved sampling providers (358 in 2011, 591 in 2012) and VMMC procedures for observation (594 in 2011, 1034 in 2012). The number of VMMC sites increased significantly between 2011 and 2012; marked seasonal variation occurred in peak periods for VMMC. Countries adopted between three and five of the six elements; forceps-guided surgery was the only element adopted by all countries. Kenya and Tanzania routinely practiced task-shifting. South Africa and Zimbabwe used pre-bundled kits with disposable instruments and electrocautery. South Africa, Tanzania, and Zimbabwe routinely employed multiple surgical bays. CONCLUSIONS: SYMMACS is the first study to provide data on the implementation of VMMC programs and adoption of elements of surgical efficiency. Findings have contributed to policy change on task-shifting in Zimbabwe, a review of the monitoring system for adverse events in South Africa, an increased use of commercially bundled VMMC kits in Tanzania, and policy dialogue on improving VMMC service delivery in Kenya. This article serves as an overview for five other articles following this supplement.

Provider attitudes toward the voluntary medical male circumcision scale-up in Kenya, South Africa, Tanzania and Zimbabwe.

BACKGROUND: Countries participating in voluntary medical male circumcision (VMMC) scale-up have adopted most of six elements of surgical efficiency, depending on national policy. However, effective implementation of these elements largely depends on providers' attitudes and subsequent compliance. We explored the concordance between recommended practices and providers' perceptions toward the VMMC efficiency elements, in part to inform review of national policies. METHODS AND FINDINGS: As part of Systematic Monitoring of the VMMC Scale-up (SYMMACS), we conducted a survey of VMMC providers in Kenya, South Africa, Tanzania, and Zimbabwe. SYMMACS assessed providers' attitudes and perceptions toward these elements in 2011 and 2012. A restricted analysis using 2012 data to calculate unadjusted odds ratios and 95% confidence intervals for the country effect on each attitudinal outcome was done using logistic regression. As only two countries allow more than one cadre to perform the surgical procedure, odds ratios looking at country effect were adjusted for cadre effect for these two countries. Qualitative data from open-ended responses were used to triangulate with quantitative analyses. This analysis showed concordance between each country's policies and provider attitudes toward the efficiency elements. One exception was task-shifting, which is not authorized in South Africa or Zimbabwe; providers across all countries approved this practice. CONCLUSIONS: The decision to adopt efficiency elements is often based on national policies. The concordance between the policies of each country and provider attitudes bodes well for compliance and effective implementation. However, study findings suggest that there may be need to consult providers when developing national policies.

Work experience, job-fulfillment and burnout among VMMC providers in Kenya, South Africa, Tanzania and Zimbabwe.

BACKGROUND: Human resource capacity is vital to the scale-up of voluntary medical male circumcision (VMMC) services. VMMC providers are at risk of "burnout" from performing a single task repeatedly in a high volume work environment that produces long work hours and intense work effort. METHODS AND FINDINGS: The Systematic Monitoring of the Voluntary Medical Male Circumcision Scale-up (SYMMACS) surveyed VMMC providers in Kenya, South Africa, Tanzania, and Zimbabwe in 2011 (n = 357) and 2012 (n = 591). Providers self-reported on their training, work experience, levels of job-fulfillment and work fatigue/burnout. Data analysis included a descriptive analysis of VMMC provider characteristics, and both bivariate and multivariate analyses of factors associated with provider work fatigue/burnout. In 2012, Kenyan providers had worked in VMMC for a median of 31 months compared to South Africa (10 months), Tanzania (15 months), and Zimbabwe (11 months). More than three-quarters (78 - 99%) of providers in all countries in 2012 reported that VMMC is a personally fulfilling job. However, 67% of Kenyan providers reported starting to experience work fatigue/burnout compared to South Africa (33%), Zimbabwe (17%), and Tanzania (15%). Despite the high level of work fatigue/burnout in Kenya, none of the measured factors (i.e., gender, age, full-time versus part-time status, length of service, number of operations performed, or cadre) were significantly associated with work fatigue/burnout in 2011. In 2012, logistic regression found increases in age (p<.05) and number of months working in VMMC (p<.01) were associated with an increased likelihood of experiencing work fatigue/burnout, while higher career total VMMCs decreased the likelihood of experiencing burnout. CONCLUSION: Given cross-country differences, further elucidation of cultural and other contextual factors that may influence provider burnout is required. Continuing to emphasize the contribution that providers make in the fight against HIV/AIDS is important.

Surgical efficiencies and quality in the performance of voluntary medical male circumcision (VMMC) procedures in Kenya, South Africa, Tanzania, and Zimbabwe.

INTRODUCTION: This analysis explores the association between elements of surgical efficiency in voluntary medical male circumcision (VMMC), quality of surgical technique, and the amount of time required to conduct VMMC procedures in actual field settings. Efficiency outcomes are defined in terms of the primary provider's time with the client (PPTC) and total elapsed operating time (TEOT). METHODS: Two serial cross-sectional surveys of VMMC sites were conducted in Kenya, Republic of South Africa, Tanzania and Zimbabwe in 2011 and 2012. Trained clinicians observed quality of surgical technique and timed 9 steps in the VMMC procedure. Four elements of efficiency (task-shifting, task-sharing [of suturing], rotation among multiple surgical beds, and use of electrocautery) and quality of surgical technique were assessed as explanatory variables. Mann Whitney and Kruskal Wallis tests were used in the bivariate analysis and linear regression models for the multivariate analyses to test the relationship between these five explanatory variables and two outcomes: PPTC and TEOT. The VMMC procedure TEOT and PPTC averaged 23-25 minutes and 6-15 minutes, respectively, across the four countries and two years. The data showed time savings from task-sharing in suturing and use of electrocautery in South Africa and Zimbabwe (where task-shifting is not authorized). After adjusting for confounders, results demonstrated that having a secondary provider complete suturing and use of electrocautery reduced PPTC. Factors related to TEOT varied by country and year, but task-sharing of suturing and/or electrocautery were significant in two countries. Quality of surgical technique was not significantly related to PPTC or TEOT, except for South Africa in 2012 where higher quality was associated with lower TEOT. CONCLUSIONS: SYMMACS data confirm the efficiency benefits of task-sharing of suturing and use of electrocautery for decreasing TEOT. Reduced TEOT and PPTC in high volume setting did not result in decreased quality of surgical care.

Association of the ANRS-12126 male circumcision project with HIV levels among men in a South African township: evaluation of effectiveness using cross-sectional surveys.

BACKGROUND: Randomized controlled trials have shown that voluntary medical male circumcision (VMMC) reduces HIV infection by 50% to 60% in sub-Saharan African populations; however, little is known about the population-level effect of adult male circumcision (MC) as an HIV prevention method. We assessed the effectiveness of VMMC roll-out on the levels of HIV in the South African township of Orange Farm where the first randomized controlled trial (RCT) to test the effect of VMMC on HIV acquisition was conducted in 2002-2005. METHODS AND FINDINGS: The Bophelo Pele project is a community-based campaign against HIV, which includes the roll-out of free VMMC. A baseline cross-sectional biomedical survey was conducted in 2007-2008 among a random sample of 1,998 men aged 15 to 49 (survey response rate 80.7%). In 2010-2011, we conducted a follow-up random survey among 3,338 men aged 15 to 49 (survey response rate 79.6%) to evaluate the project. Participants were interviewed, blood samples were collected and tested for HIV and recent HIV infection (using the BED HIV incidence assay), and MC status was assessed through a clinical examination. Data were analyzed using multivariate and propensity statistical methods. Owing to the VMMCs performed in the context of the RCT and the Bophelo Pele project, the prevalence rate of adult MC increased from 0.12 (95% CI 0.10-0.14) to 0.53 (95% CI 0.51-0.55). Without these VMMCs, the HIV prevalence rate in 2010-2011 would have been 19% (95% CI 12%-26%) higher (0.147 instead of 0.123). When comparing circumcised and uncircumcised men, no association of MC status with sexual behavior was detected. Among circumcised and uncircumcised men, the proportion consistently using condoms with non-spousal partners in the past 12 months was 44.0% (95% CI 41.7%-46.5%) versus 45.4% (95% CI 42.2%-48.6%) with weighted prevalence rate ratio (wPRR) = 0.94 (95% CI 0.85-1.03). The proportion having two or more non-spousal partners was 50.4% (95% CI 47.9%-52.9%) versus 44.2% (95% CI 41.3%-46.9%) with wPRR = 1.03 (95% CI 0.95-1.10). We found a reduction of BED-estimated HIV incidence rate ranging from 57% (95% CI 29%-76%) to 61% (95% CI 14%-83%) among circumcised men in comparison with uncircumcised men. CONCLUSIONS: Findings suggest that the roll-out of VMMC in Orange Farm is associated with a significant reduction of HIV levels in the community. The main limitation of the study is that it was not randomized and cannot prove a causal association. The roll-out of VMMC among adults in sub-Saharan Africa should be an international priority and needs to be accelerated to effectively combat the spread of HIV. Please see later in the article for the Editors' Summary.

Voluntary medical male circumcision: logistics, commodities, and waste management requirements for scale-up of services.

BACKGROUND: The global HIV prevention community is implementing voluntary medical male circumcision (VMMC) programs across eastern and southern Africa, with a goal of reaching 80% coverage in adult males by 2015. Successful implementation will depend on the accessibility of commodities essential for VMMC programming and the appropriate allocation of resources to support the VMMC supply chain. For this, the United States President's Emergency Plan for AIDS Relief, in collaboration with the World Health Organization and the Joint United Nations Programme on HIV/AIDS, has developed a standard list of commodities for VMMC programs. METHODS AND FINDINGS: This list of commodities was used to inform program planning for a 1-y program to circumcise 152,000 adult men in Swaziland. During this process, additional key commodities were identified, expanding the standard list to include commodities for waste management, HIV counseling and testing, and the treatment of sexually transmitted infections. The approximate costs for the procurement of commodities, management of a supply chain, and waste disposal, were determined for the VMMC program in Swaziland using current market prices of goods and services. Previous costing studies of VMMC programs did not capture supply chain costs, nor the full range of commodities needed for VMMC program implementation or waste management. Our calculations indicate that depending upon the volume of services provided, supply chain and waste management, including commodities and associated labor, contribute between US$58.92 and US$73.57 to the cost of performing one adult male circumcision in Swaziland. CONCLUSIONS: Experience with the VMMC program in Swaziland indicates that supply chain and waste management add approximately US$60 per circumcision, nearly doubling the total per procedure cost estimated previously; these additional costs are used to inform the estimate of per procedure costs modeled by Njeuhmeli et al. in "Voluntary Medical Male Circumcision: Modeling the Impact and Cost of Expanding Male Circumcision for HIV Prevention in Eastern and Southern Africa." Program planners and policy makers should consider the significant contribution of supply chain and waste management to VMMC program costs as they determine future resource needs for VMMC programs.

Adult male circumcision as an intervention against HIV: an operational study of uptake in a South African community (ANRS 12126).

BACKGROUND: To evaluate the knowledge, attitudes and beliefs about adult male circumcision (AMC), assess the association of AMC with HIV incidence and prevalence, and estimate AMC uptake in a Southern African community. METHODS: A cross-sectional biomedical survey (ANRS-12126) conducted in 2007-2008 among a random sample of 1198 men aged 15 to 49 from Orange Farm (South Africa). Face-to-face interviews were conducted by structured questionnaire. Recent HIV infections were evaluated using the BED incidence assay. Circumcision status was self-reported and clinically assessed. Adjusted HIV incidence rate ratios (aIRR) and prevalence ratios (aPR) were calculated using Poisson regression. RESULTS: The response rate was 73.9%. Most respondents agreed that circumcised men could become HIV infected and needed to use condoms, although 19.3% (95%CI: 17.1% to 21.6%) asserted that AMC protected fully against HIV. Among self-reported circumcised men, 44.9% (95%CI: 39.6% to 50.3%) had intact foreskins. Men without foreskins had lower HIV incidence and prevalence than men with foreskins (aIRR = 0.35; 95%CI: 0.14 to 0.88; aPR = 0.45, 95%CI: 0.26 to 0.79). No significant difference was found between self-reported circumcised men with foreskins and other uncircumcised men. Intention to undergo AMC was associated with ethnic group and partner and family support of AMC. Uptake of AMC was 58.8% (95%CI: 55.4% to 62.0%). CONCLUSIONS: AMC uptake in this community is high but communication and counseling should emphasize what clinical AMC is and its effect on HIV acquisition. These findings suggest that AMC roll-out is promising but requires careful implementation strategies to be successful against the African HIV epidemic.

A model for the roll-out of comprehensive adult male circumcision services in African low-income settings of high HIV incidence: the ANRS 12126 Bophelo Pele Project.

BACKGROUND: World Health Organization (WHO)/Joint United Nations Programme on AIDS (UNAIDS) has recommended adult male circumcision (AMC) for the prevention of heterosexually acquired HIV infection in men from communities where HIV is hyperendemic and AMC prevalence is low. The objective of this study was to investigate the feasibility of the roll-out of medicalized AMC according to UNAIDS/WHO operational guidelines in a targeted African setting. METHODS AND FINDINGS: The ANRS 12126 "Bophelo Pele" project was implemented in 2008 in the township of Orange Farm (South Africa). It became functional in 5 mo once local and ethical authorizations were obtained. Project activities involved community mobilization and outreach, as well as communication approaches aimed at both men and women incorporating broader HIV prevention strategies and promoting sexual health. Free medicalized AMC was offered to male residents aged 15 y and over at the project's main center, which had been designed for low-income settings. Through the establishment of an innovative surgical organization, up to 150 AMCs under local anesthesia, with sterilized circumcision disposable kits and electrocautery, could be performed per day by three task-sharing teams of one medical circumciser and five nurses. Community support for the project was high. As of November 2009, 14,011 men had been circumcised, averaging 740 per month in the past 12 mo, and 27.5% of project participants agreed to be tested for HIV. The rate of adverse events, none of which resulted in permanent damage or death, was 1.8%. Most of the men surveyed (92%) rated the services provided positively. An estimated 39.1% of adult uncircumcised male residents have undergone surgery and uptake is steadily increasing. CONCLUSION: This study demonstrates that a quality AMC roll-out adapted to African low-income settings is feasible and can be implemented quickly and safely according to international guidelines. The project can be a model for the scale-up of comprehensive AMC services, which could be tailored for other rural and urban communities of high HIV prevalence and low AMC rates in Eastern and Southern Africa. Please see later in the article for the Editors' Summary.