Historical visit attendance as predictor of treatment interruption in South African HIV patients: Extension of a validated machine learning model.

Retention of antiretroviral (ART) patients is a priority for achieving HIV epidemic control in South Africa. While machine-learning methods are being increasingly utilised to identify high risk populations for suboptimal HIV service utilisation, they are limited in terms of explaining relationships between predictors. To further understand these relationships, we implemented machine learning methods optimised for predictive power and traditional statistical methods. We used routinely collected electronic medical record (EMR) data to evaluate longitudinal predictors of lost-to-follow up (LTFU) and temporal interruptions in treatment (IIT) in the first two years of treatment for ART patients in the Gauteng and North West provinces of South Africa. Of the 191,162 ART patients and 1,833,248 visits analysed, 49% experienced at least one IIT and 85% of those returned for a subsequent clinical visit. Patients iteratively transition in and out of treatment indicating that ART retention in South Africa is likely underestimated. Historical visit attendance is shown to be predictive of IIT using machine learning, log binomial regression and survival analyses. Using a previously developed categorical boosting (CatBoost) algorithm, we demonstrate that historical visit attendance alone is able to predict almost half of next missed visits. With the addition of baseline demographic and clinical features, this model is able to predict up to 60% of next missed ART visits with a sensitivity of 61.9% (95% CI: 61.5-62.3%), specificity of 66.5% (95% CI: 66.4-66.7%), and positive predictive value of 19.7% (95% CI: 19.5-19.9%). While the full usage of this model is relevant for settings where infrastructure exists to extract EMR data and run computations in real-time, historical visits attendance alone can be used to identify those at risk of disengaging from HIV care in the absence of other behavioural or observable risk factors.

Validation and Improvement of a Machine Learning Model to Predict Interruptions in Antiretroviral Treatment in South Africa.

INTRODUCTION: Machine learning algorithms are increasingly being used to inform HIV prevention and detection strategies. We validated and extended a previously developed machine learning model for patient retention on antiretroviral therapy in a new geographic catchment area in South Africa. METHODS: We compared the ability of an adaptive boosting algorithm to predict interruption in treatment (IIT) in 2 South African cohorts from the Free State and Mpumalanga and Gauteng and North West (GA/NW) provinces. We developed a novel set of predictive features for the GA/NW cohort using a categorical boosting model. We evaluated the ability of the model to predict IIT over all visits and across different periods within a patient's treatment trajectory. RESULTS: When predicting IIT, the GA/NW and Free State and Mpumalanga models demonstrated a sensitivity of 60% and 61%, respectively, able to correctly predict nearly two-thirds of all missed visits with a positive predictive value of 18% and 19%. Using predictive features generated from the GA/NW cohort, the categorical boosting model correctly predicted 22,119 of a total of 35,985 missed next visits, yielding a sensitivity of 62%, specificity of 67%, and positive predictive value of 20%. Model performance was highest when tested on visits within the first 6 months. CONCLUSIONS: Machine learning algorithms may be useful in informing tools to increase antiretroviral therapy patient retention and efficiency of HIV care interventions. This is particularly relevant in developing countries where health data systems are being strengthened to collect data on a scale that is large enough to apply novel analytical methods.

Applying machine learning and predictive modeling to retention and viral suppression in South African HIV treatment cohorts.

HIV treatment programs face challenges in identifying patients at risk for loss-to-follow-up and uncontrolled viremia. We applied predictive machine learning algorithms to anonymised, patient-level HIV programmatic data from two districts in South Africa, 2016-2018. We developed patient risk scores for two outcomes: (1) visit attendance ≤ 28 days of the next scheduled clinic visit and (2) suppression of the next HIV viral load (VL). Demographic, clinical, behavioral and laboratory data were investigated in multiple models as predictor variables of attending the next scheduled visit and VL results at the next test. Three classification algorithms (logistical regression, random forest and AdaBoost) were evaluated for building predictive models. Data were randomly sampled on a 70/30 split into a training and test set. The training set included a balanced set of positive and negative examples from which the classification algorithm could learn. The predictor variable data from the unseen test set were given to the model, and each predicted outcome was scored against known outcomes. Finally, we estimated performance metrics for each model in terms of sensitivity, specificity, positive and negative predictive value and area under the curve (AUC). In total, 445,636 patients were included in the retention model and 363,977 in the VL model. The predictive metric (AUC) ranged from 0.69 for attendance at the next scheduled visit to 0.76 for VL suppression, suggesting that the model correctly classified whether a scheduled visit would be attended in 2 of 3 patients and whether the VL result at the next test would be suppressed in approximately 3 of 4 patients. Variables that were important predictors of both outcomes included prior late visits, number of prior VL tests, time since their last visit, number of visits on their current regimen, age, and treatment duration. For retention, the number of visits at the current facility and the details of the next appointment date were also predictors, while for VL suppression, other predictors included the range of the previous VL value. Machine learning can identify HIV patients at risk for disengagement and unsuppressed VL. Predictive modeling can improve the targeting of interventions through differentiated models of care before patients disengage from treatment programmes, increasing cost-effectiveness and improving patient outcomes.