Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning.

BACKGROUND: Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue. METHODS: A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. RESULTS: In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9-92.7), and specificity 75.6% (95% CI 73.1-78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2-91.5), but specificity increased to 85.1% (95%CI 82.6-87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200-200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69-0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66-0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides. CONCLUSIONS: The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678.

Automated liquid handling robot for rapid lateral flow assay development.

The lateral flow assay (LFA) is one of the most popular technologies on the point-of-care diagnostics market due to its low cost and ease of use, with applications ranging from pregnancy to environmental toxins to infectious disease. While the use of these tests is relatively straightforward, significant development time and effort are required to create tests that are both sensitive and specific. Workflows to guide the LFA development process exist but moving from target selection to an LFA that is ready for field testing can be labor intensive, resource heavy, and time consuming. To reduce the cost and the duration of the LFA development process, we introduce a novel development platform centered on the flexibility, speed, and throughput of an automated robotic liquid handling system. The system comprises LFA-specific hardware and software that enable large optimization experiments with discrete and continuous variables such as antibody pair selection or reagent concentration. Initial validation of the platform was demonstrated during development of a malaria LFA but was readily expanded to encompass development of SARS-CoV-2 and Mycobacterium tuberculosis LFAs. The validity of the platform, where optimization experiments are run directly on LFAs rather than in solution, was based on a direct comparison between the robotic system and a more traditional ELISA-like method. By minimizing hands-on time, maximizing experiment size, and enabling improved reproducibility, the robotic system improved the quality and quantity of LFA assay development efforts.

Automated microscopy for routine malaria diagnosis: a field comparison on Giemsa-stained blood films in Peru.

BACKGROUND: Microscopic examination of Giemsa-stained blood films remains a major form of diagnosis in malaria case management, and is a reference standard for research. However, as with other visualization-based diagnoses, accuracy depends on individual technician performance, making standardization difficult and reliability poor. Automated image recognition based on machine-learning, utilizing convolutional neural networks, offers potential to overcome these drawbacks. A prototype digital microscope device employing an algorithm based on machine-learning, the Autoscope, was assessed for its potential in malaria microscopy. Autoscope was tested in the Iquitos region of Peru in 2016 at two peripheral health facilities, with routine microscopy and PCR as reference standards. The main outcome measures include sensitivity and specificity of diagnosis of malaria from Giemsa-stained blood films, using PCR as reference. METHODS: A cross-sectional, observational trial was conducted at two peripheral primary health facilities in Peru. 700 participants were enrolled with the criteria: (1) age between 5 and 75 years, (2) history of fever in the last 3 days or elevated temperature on admission, (3) informed consent. The main outcome measures included sensitivity and specificity of diagnosis of malaria from Giemsa-stained blood films, using PCR as reference. RESULTS: At the San Juan clinic, sensitivity of Autoscope for diagnosing malaria was 72% (95% CI 64-80%), and specificity was 85% (95% CI 79-90%). Microscopy performance was similar to Autoscope, with sensitivity 68% (95% CI 59-76%) and specificity 100% (95% CI 98-100%). At San Juan, 85% of prepared slides had a minimum of 600 WBCs imaged, thus meeting Autoscope's design assumptions. At the second clinic, Santa Clara, the sensitivity of Autoscope was 52% (95% CI 44-60%) and specificity was 70% (95% CI 64-76%). Microscopy performance at Santa Clara was 42% (95% CI 34-51) and specificity was 97% (95% CI 94-99). Only 39% of slides from Santa Clara met Autoscope's design assumptions regarding WBCs imaged. CONCLUSIONS: Autoscope's diagnostic performance was on par with routine microscopy when slides had adequate blood volume to meet its design assumptions, as represented by results from the San Juan clinic. Autoscope's diagnostic performance was poorer than routine microscopy on slides from the Santa Clara clinic, which generated slides with lower blood volumes. Results of the study reflect both the potential for artificial intelligence to perform tasks currently conducted by highly-trained experts, and the challenges of replicating the adaptiveness of human thought processes.