Measuring the performance of computer vision artificial intelligence to interpret images of HIV self-testing results.

Introduction: HIV self-testing (HIVST) is highly sensitive and specific, addresses known barriers to HIV testing (such as stigma), and is recommended by the World Health Organization as a testing option for the delivery of HIV pre-exposure prophylaxis (PrEP). Nevertheless, HIVST remains underutilized as a diagnostic tool in community-based, differentiated HIV service delivery models, possibly due to concerns about result misinterpretation, which could lead to inadvertent onward transmission of HIV, delays in antiretroviral therapy (ART) initiation, and incorrect initiation on PrEP. Ensuring that HIVST results are accurately interpreted for correct clinical decisions will be critical to maximizing HIVST's potential. Early evidence from a few small pilot studies suggests that artificial intelligence (AI) computer vision and machine learning could potentially assist with this task. As part of a broader study that task-shifted HIV testing to a new setting and cadre of healthcare provider (pharmaceutical technologists at private pharmacies) in Kenya, we sought to understand how well AI technology performed at interpreting HIVST results. Methods: At 20 private pharmacies in Kisumu, Kenya, we offered free blood-based HIVST to clients ≥18 years purchasing products indicative of sexual activity (e.g., condoms). Trained pharmacy providers assisted clients with HIVST (as needed), photographed the completed HIVST, and uploaded the photo to a web-based platform. In real time, each self-test was interpreted independently by the (1) client and (2) pharmacy provider, with the HIVST images subsequently interpreted by (3) an AI algorithm (trained on lab-captured images of HIVST results) and (4) an expert panel of three HIVST readers. Using the expert panel's determination as the ground truth, we calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for HIVST result interpretation for the AI algorithm as well as for pharmacy clients and providers, for comparison. Results: From March to June 2022, we screened 1,691 pharmacy clients and enrolled 1,500 in the study. All clients completed HIVST. Among 854 clients whose HIVST images were of sufficient quality to be interpretable by the AI algorithm, 63% (540/854) were female, median age was 26 years (interquartile range: 22-31), and 39% (335/855) reported casual sexual partners. The expert panel identified 94.9% (808/854) of HIVST images as HIV-negative, 5.1% (44/854) as HIV-positive, and 0.2% (2/854) as indeterminant. The AI algorithm demonstrated perfect sensitivity (100%), perfect NPV (100%), and 98.8% specificity, and 81.5% PPV (81.5%) due to seven false-positive results. By comparison, pharmacy clients and providers demonstrated lower sensitivity (93.2% and 97.7% respectively) and NPV (99.6% and 99.9% respectively) but perfect specificity (100%) and perfect PPV (100%). Conclusions: AI computer vision technology shows promise as a tool for providing additional quality assurance of HIV testing, particularly for catching Type II error (false-negative test interpretations) committed by human end-users. We discuss possible use cases for this technology to support differentiated HIV service delivery and identify areas for future research that is needed to assess the potential impacts-both positive and negative-of deploying this technology in real-world HIV service delivery settings.

Measuring the performance of HIV self-testing at private pharmacies in Kenya: a cross-sectional study.

INTRODUCTION: HIV self-testing (HIVST) has the potential to support daily oral pre-exposure prophylaxis (PrEP) delivery in private pharmacies, but many national guidelines have not approved HIVST for PrEP dispensing. In Kenya, pharmacy providers are permitted to deliver HIVST, but often do not have the required certification to deliver rapid diagnostic testing (RDT). We estimated the performance of provider-delivered HIVST compared to RDT, the standard of care for PrEP delivery, at private pharmacies in Kenya to inform decisions on the use of HIVST for PrEP scale-up. METHODS: At 20 pharmacies in Kisumu County, we trained pharmacy providers (pharmacists and pharmaceutical technologists) on blood-based HIVST use and client assistance (if requested). We recruited pharmacy clients purchasing sexual and reproductive health-related products (e.g. condoms) and enrolled those ≥18 years with self-reported behaviours associated with HIV risk. Enrolled clients received HIVST with associated provider counselling, followed by RDT by a certified HIV testing services (HTS) counsellor. Pharmacy providers and clients independently interpreted HIVST results prior to RDT (results interpreted only by the HTS counsellor). We calculated the sensitivity and specificity of pharmacy provider-delivered HIVST compared to HTS counsellor-administered RDT. RESULTS: Between March and June 2022, we screened 1691 clients and enrolled 1500; 64% (954/1500) were female and the median age was 26 years (IQR 22-31). We additionally enrolled 40 providers; 42% (17/40) were pharmacy owners and their median years of experience was 6 (IQR 4-10). The majority (79%, 1190/1500) of clients requested provider assistance with HIVST and providers spent a median of 20 minutes (IQR 15-43) with each HIVST client. The sensitivity of provider-delivered HIVST at the pharmacy was high when interpreted by providers (98.5%, 95% CI 97.8%, 99.1%) and clients (98.8%, 95% CI 98.0%, 99.3%), as was the specificity of HIVST in this setting (provider-interpretation: 96.9%, 95% CI 89.2%, 99.6%; client-interpretation: 93.8%, 95% CI 84.8%, 98.3%). CONCLUSIONS: When compared to the national HIV testing algorithm, provider-delivered blood-based HIVST at private pharmacies in Kenya performed well. These findings suggest that blood-based HIVST may be a useful tool to support PrEP initiation and continuation at private pharmacies and potentially other community-based delivery settings.

HIV prevention programme with young women who sell sex in Mombasa, Kenya: learnings for scale-up.

INTRODUCTION: In 2018, the National AIDS and sexually transmitted infection (STI) Control Programme developed a national guidelines to facilitate the inclusion of young women who sell sex (YWSS) in the HIV prevention response in Kenya. Following that, a 1-year pilot intervention, where a package of structural, behavioural and biomedical services was provided to 1376 cisgender YWSS to address their HIV-related risk and vulnerability, was implemented. METHODS: Through a mixed-methods, pre/post study design, we assessed the effectiveness of the pilot, and elucidated implementation lessons learnt. The three data sources used included: (1) monthly routine programme monitoring data collected between October 2019 and September 2020 to assess the reach and coverage; (2) two polling booth surveys, conducted before and after implementation, to determine the effectiveness; and (3) focus group discussions and key informant interviews conducted before and after intervention to assess the feasibility of the intervention. Descriptive analysis was performed to produce proportions and comparative statistics. RESULTS: During the intervention, 1376 YWSS were registered in the programme, 28% were below 19 years of age and 88% of the registered YWSS were active in the last month of intervention. In the survey, respondents reported increases in HIV-related knowledge (61.7% vs. 90%, p <0.001), ever usage of pre-exposure prophylaxis (8.5% vs. 32.2%, p < 0.001); current usage of pre-exposure prophylaxis (5.3% vs. 21.1%, p<0.002); ever testing for HIV (87.2% vs. 95.6%, p <0.04) and any clinic visit (35.1 vs. 61.1, p <0.001). However, increase in harassment by family (11.7% vs. 23.3%, p<0.04) and discrimination at educational institutions (5.3% vs. 14.4%, p<0.04) was also reported. In qualitative assessment, respondents reported early signs of success, and identified missed opportunities and made recommendations for scale-up. CONCLUSIONS: Our intervention successfully rolled out HIV prevention services for YWSS in Mombasa, Kenya, and demonstrated that programming for YWSS is feasible and can effectively be done through YWSS peer-led combination prevention approaches. However, while reported uptake of treatment and prevention services increased, there was also an increase in reported harassment and discrimination requiring further attention. Lessons learnt from the pilot intervention can inform replication and scale-up of such interventions in Kenya.

Using a HIV prevention cascade for identifying missed opportunities in PrEP delivery in Kenya: results from a programmatic surveillance study.

INTRODUCTION: HIV prevention cascades have been systematically evaluated in high-income countries, but steps in the pre-exposure prophylaxis (PrEP) service delivery cascade have not been systematically quantified in sub-Saharan Africa. We analysed missed opportunities in the PrEP cascade in a large-scale project serving female sex workers (FSW), men who have sex with men (MSM) and adolescent girls and young women (AGYW) in Kenya. METHODS: Programmatic surveillance was conducted using routine programme data from 89 project-supported sites from February 2017 to December 2019, and complemented by qualitative data. Healthcare providers used nationally approved tools to document service statistics. The analyses examined proportions of people moving onto the next step in the PrEP continuum, and identified missed opportunities. Missed opportunities were defined as implementation gaps exemplified by the proportion of individuals who could have potentially accessed each step of the PrEP cascade and did not. We also assessed trends in the cascade indicators at monthly intervals. Qualitative data were collected through 28 focus group discussions with 241 FSW, MSM, AGYW and healthcare providers, and analysed thematically to identify reasons underpinning the missed opportunities. RESULTS: During the study period, 299,798 individuals tested HIV negative (211,927 FSW, 47,533 MSM and 40,338 AGYW). Missed opportunities in screening for PrEP eligibility was 58% for FSW, 45% for MSM and 78% for AGYW. Of those screened, 28% FSW, 25% MSM and 65% AGYW were ineligible. Missed opportunities for PrEP initiation were lower among AGYW (8%) compared to FSW (72%) and MSM (75%). Continuation rates were low across all populations at Month-1 (ranging from 29% to 32%) and Month-3 (6% to 8%). Improvements in average annual Month-1 (from 26% to 41%) and Month-3 (from 4% to 15%) continuation rates were observed between 2017 and 2019. While initiation rates were better among younger FSW, MSM and AGYW (<30 years), the reverse was true for continuation. CONCLUSIONS: The application of a PrEP cascade framework facilitated this large-scale oral PrEP programme to conduct granular programmatic analysis, detecting "leaks" in the cascade. These informed programme adjustments to mitigate identified gaps resulting in improvement of selected programmatic outcomes. PrEP programmes are encouraged to introduce the cascade analysis framework into new and existing programming to optimize HIV prevention outcomes.