PhthisisBioMed Artificial Medical Intelligence: Software for Automated Analysis of Digital Chest X-ray/Fluorograms.

The scope of diagnostic medical examinations increases from year to year causing a reasonable desire to develop and implement new technologies to diagnostics and medical data analysis. Artificial intelligence (AI) algorithms became one of the most promising solutions to this problem and proved themselves in the course of mass practical application. During the three-year Moscow experiment started in 2020, the possibility was achieved to develop methodologies of AI use and to successfully implement it into the regional level healthcare system. In this article, the authors share their experience in developing a medical AI service using the example of PhthisisBioMed AI service and the results of its application in real clinical activities environment. This AI service has shown its quality and reliability confirmed by technological monitoring. Clinical trials of PhthisisBioMed AI service were conducted on a specially prepared verified data set (n=1536) considering epidemiological indicators of the thoracic organs major diseases prevalence. The mean sensitivity of the service was 0.975 (95% CI: 0.966-0.984). PhthisisBioMed medical AI service is registered as a medical device (medical device registration certificate No.RZN 2022/17406 dated May 31, 2022) and is actively used in the Russian Federation as a diagnostic tool to reduce the burden on radiologists and to accelerate the process of medical report obtaining.

Methodology for Conducting Post-Marketing Surveillance of Software as a Medical Device Based on Artificial Intelligence Technologies.

The aim of the study was to develop a methodology for conducting post-registration clinical monitoring of software as a medical device based on artificial intelligence technologies (SaMD-AI). Materials and Methods: The methodology of post-registration clinical monitoring is based on the requirements of regulatory legal acts issued by the Board of the Eurasian Economic Commission. To comply with these requirements, the monitoring involves submission of the review of adverse events reports, the review of developers' routine reports on the safety and efficiency of SaMD-AI, and the assessment of the system for collecting and analyzing developers' post-registration data on the safety and efficiency of medical devices. The methodology was developed with regard to the recommendations of the International Medical Device Regulators Forum and the documents issued by the Food and Drug Administration (USA). Field-testing of this methodology was carried out using SaMD-AI designed for diagnostic imaging. Results: The post-registration monitoring of SaMD-AI consists of three key stages: collecting user feedback, technical monitoring and clinical validation. Technical monitoring involves routine evaluation of SaMD-AI output data quality to detect and remove flaws in a timely manner, and to secure the product stability. Major outcomes include an ordered list of technical flaws in SaMD-AI and their classification using evidence from diagnostic imaging studies. The application of this methodology resulted in a gradual reduction in the number of studies with flaws due to timely improvements in artificial intelligence algorithms: the number of flaws decreased to 5% in various aspects during subsequent testing. Clinical validation confirmed that SaMD-AI is capable of producing clinically meaningful outputs related to its intended use within the functionality determined by the developer. The testing procedure and the baseline testing framework were established during the field testing. Conclusion: The developed methodology will ensure the safety and efficiency of SaMD-AI taking into account its specifics as intangible medical devices. The methodology presented in this paper can be used by SaMD-AI developers to plan and carry out the post-registration clinical monitoring.