Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning.
IEEE Trans Neural Netw Learn Syst
; 32(9): 3786-3797, 2021 09.
Article
in English
| MEDLINE | ID: covidwho-1348109
ABSTRACT
Medical imaging technologies, including computed tomography (CT) or chest X-Ray (CXR), are largely employed to facilitate the diagnosis of the COVID-19. Since manual report writing is usually too time-consuming, a more intelligent auxiliary medical system that could generate medical reports automatically and immediately is urgently needed. In this article, we propose to use the medical visual language BERT (Medical-VLBERT) model to identify the abnormality on the COVID-19 scans and generate the medical report automatically based on the detected lesion regions. To produce more accurate medical reports and minimize the visual-and-linguistic differences, this model adopts an alternate learning strategy with two procedures that are knowledge pretraining and transferring. To be more precise, the knowledge pretraining procedure is to memorize the knowledge from medical texts, while the transferring procedure is to utilize the acquired knowledge for professional medical sentences generations through observations of medical images. In practice, for automatic medical report generation on the COVID-19 cases, we constructed a dataset of 368 medical findings in Chinese and 1104 chest CT scans from The First Affiliated Hospital of Jinan University, Guangzhou, China, and The Fifth Affiliated Hospital of Sun Yat-sen University, Zhuhai, China. Besides, to alleviate the insufficiency of the COVID-19 training samples, our model was first trained on the large-scale Chinese CX-CHR dataset and then transferred to the COVID-19 CT dataset for further fine-tuning. The experimental results showed that Medical-VLBERT achieved state-of-the-art performances on terminology prediction and report generation with the Chinese COVID-19 CT dataset and the CX-CHR dataset. The Chinese COVID-19 CT dataset is available at https//covid19ct.github.io/.
Full text:
Available
Collection:
International databases
Database:
MEDLINE
Main subject:
Research Report
/
Machine Learning
/
COVID-19
Type of study:
Observational study
/
Prognostic study
Limits:
Humans
Country/Region as subject:
Asia
Language:
English
Journal:
IEEE Trans Neural Netw Learn Syst
Year:
2021
Document Type:
Article
Similar
MEDLINE
...
LILACS
LIS