Augmented reality visualisation and interaction for COVID-19 Ct-scan NN automated segmentation: A validation study
IEEE Sensors Journal
; : 1-1, 2023.
Article
in English
| Scopus | ID: covidwho-2291171
ABSTRACT
Although medical imaging technology has persisted in evolving over the last decades, the techniques and technologies used for analytical and visualisation purposes have remained constant. Manual or semi-automatic segmentation is, in many cases, complicated. It requires the intervention of a specialist and is time-consuming, especially during the Coronavirus disease (COVID-19) pandemic, which has had devastating medical and economic consequences. Processing and visualising medical images with advanced techniques represent medical professionals’breakthroughs. This paper studies how augmented reality (AR) and artificial intelligence (AI) can transform medical practice during COVID-19 and post-COVID-19 pandemic. Here we report an augmented reality visualisation and interaction platform;it covers the whole process from uploading chest Ct-scan images to automatic segmentation-based deep learning, 3D reconstruction, 3D visualisation, and manipulation. AR provides a more realistic 3D visualisation system, allowing doctors to effectively interact with the generated 3D model of segmented lungs and COVID-19 lesions. We use the U-Net Neural Network (NN) for automated segmentation. The statistical measures obtained using the Dice score, pixel accuracy, sensitivity, G-mean, and specificity are 0.749, 0.949, 0.956, 0.955, and 0.954, respectively. The user-friendliness and usability are objectified by a formal user study that compared our augmented reality-assisted design to the standard diagnosis setup. One hundred and six doctors and medical students, including eight senior medical lecturers, volunteered to assess our platform. The platform could be used as an aid diagnosis tool to identify and analyse the COVID-19 infectious or as a training tool for residents and medical students. The prototype can be extended to other pulmonary pathologies. IEEE
AR Interaction; AR Visualisation; Augmented Reality; Automated Segmentation; COVID-19; Deep Learning; Medical imaging; U-Net; 3D modeling; Automation; Computerized tomography; Diagnosis; Image segmentation; Three dimensional computer graphics; Visualization; Augmented reality interaction; Augmented reality visualization; CT-scan; Medical students; Neural-networks; Reality visualization; Validation study
Full text:
Available
Collection:
Databases of international organizations
Database:
Scopus
Type of study:
Prognostic study
Language:
English
Journal:
IEEE Sensors Journal
Year:
2023
Document Type:
Article
Similar
MEDLINE
...
LILACS
LIS