Use of Natural Language Processing (NLP) in Evaluation of Radiology Reports: An Update on Applications and Technology Advances.

Natural language processing (NLP) is focused on the computer interpretation of human language and can be used to evaluate radiology reports and has demonstrated useful applications in essentially all aspects of medical imaging delivery: interpretation of imaging data, improving image acquisition, image analysis, and increasing efficiency of imaging services. This manuscript reviews general technologic approaches to NLP at a level hopefully understandable by clinical radiologists, discusses recent advancements in NLP techniques, and discusses current and potential applications of NLP in radiology.

Using a Natural Language Processing and Machine Learning Algorithm Program to Analyze Inter-Radiologist Report Style Variation and Compare Variation Between Radiologists When Using Highly Structured Versus More Free Text Reporting.

PURPOSE: To use a natural language processing and machine learning algorithm to evaluate inter-radiologist report variation and compare variation between radiologists using highly structured versus more free text reporting. MATERIALS AND METHODS: 28,615 radiology reports were analyzed for 4 metrics: verbosity, observational terms only, unwarranted negative findings, and repeated language in different sections. Radiology reports for two imaging examinations were analyzed and compared - one which was more templated (ultrasound - appendicitis) and one which relied on more free text (chest radiograph - single view). For each metric, the mean and standard deviation for defined outlier results for all dictations (individual and group mean) was calculated. The mean number of outlier metrics per reader per study was calculated and compared between radiologists and between the two report types. Wilcoxon rank test and paired Wilcoxon signed rank test were applied. The radiologists were also ranked based on the number of outlier metrics identified per study. RESULTS: There was great variability in radiologist dictation styles - outlier metrics per report varied greatly between radiologists with the maximum 10 times higher than the minimum score. Metric values were greater (P < 0.0001) on the standardized reports using free text than the more structured reports. CONCLUSIONS: The algorithm successfully evaluated metrics showing variability in reporting profiles particularly when there is free text. This variability can be an obstacle to providing effective communication and reliability of care.

Comparison Between Manual Auditing and a Natural Language Process With Machine Learning Algorithm to Evaluate Faculty Use of Standardized Reports in Radiology.

PURPOSE: When implementing or monitoring department-sanctioned standardized radiology reports, feedback about individual faculty performance has been shown to be a useful driver of faculty compliance. Most commonly, these data are derived from manual audit, which can be both time-consuming and subject to sampling error. The purpose of this study was to evaluate whether a software program using natural language processing and machine learning could accurately audit radiologist compliance with the use of standardized reports compared with performed manual audits. METHODS: Radiology reports from a 1-month period were loaded into such a software program, and faculty compliance with use of standardized reports was calculated. For that same period, manual audits were performed (25 reports audited for each of 42 faculty members). The mean compliance rates calculated by automated auditing were then compared with the confidence interval of the mean rate by manual audit. RESULTS: The mean compliance rate for use of standardized reports as determined by manual audit was 91.2% with a confidence interval between 89.3% and 92.8%. The mean compliance rate calculated by automated auditing was 92.0%, within that confidence interval. CONCLUSION: This study shows that by use of natural language processing and machine learning algorithms, an automated analysis can accurately define whether reports are compliant with use of standardized report templates and language, compared with manual audits. This may avoid significant labor costs related to conducting the manual auditing process.