Automating Scoliosis Measurements in Radiographic Studies with Machine Learning: Comparing Artificial Intelligence and Clinical Reports.

Scoliosis is a condition of abnormal lateral spinal curvature affecting an estimated 2 to 3% of the US population, or seven million people. The Cobb angle is the standard measurement of spinal curvature in scoliosis but is known to have high interobserver and intraobserver variability. Thus, the objective of this study was to build and validate a system for automatic quantitative evaluation of the Cobb angle and to compare AI generated and human reports in the clinical setting. After IRB was obtained, we retrospectively collected 2150 frontal view scoliosis radiographs at a tertiary referral center (January 1, 2019, to January 1, 2021, ≥ 16 years old, no hardware). The dataset was partitioned into 1505 train (70%), 215 validation (10%), and 430 test images (20%). All thoracic and lumbar vertebral bodies were segmented with bounding boxes, generating approximately 36,550 object annotations that were used to train a Faster R-CNN Resnet-101 object detection model. A controller algorithm was written to localize vertebral centroid coordinates and derive the Cobb properties (angle and endplate) of dominant and secondary curves. AI-derived Cobb angle measurements were compared to the clinical report measurements, and the Spearman rank-order demonstrated significant correlation (0.89, p < 0.001). Mean difference between AI and clinical report angle measurements was 7.34° (95% CI: 5.90-8.78°), which is similar to published literature (up to 10°). We demonstrate the feasibility of an AI system to automate measurement of level-by-level spinal angulation with performance comparable to radiologists.

Clinical utility of accelerated MAVRIC-SL with robust-PCA compared to conventional MAVRIC-SL in evaluation of total hip arthroplasties.

OBJECTIVE: To compare the diagnostic performance of a conventional metal artifact suppression sequence MAVRIC-SL (multi-acquisition variable-resonance image combination selective) and a novel 2.6-fold faster sequence employing robust principal component analysis (RPCA), in the MR evaluation of hip implants at 3 T. MATERIALS AND METHODS: Thirty-six total hip implants in 25 patients were scanned at 3 T using a conventional MAVRIC-SL proton density-weighted sequence and an RPCA MAVRIC-SL proton density-weighted sequence. Comparison was made of image quality, geometric distortion, visualization around acetabular and femoral components, and conspicuity of abnormal imaging findings using the Wilcoxon signed-rank test and a non-inferiority test. Abnormal findings were correlated with subsequent clinical management and intraoperative findings if the patient underwent subsequent surgery. RESULTS: Mean scores for conventional MAVRIC-SL were better than RPCA MAVRIC-SL for all qualitative parameters (p < 0.05), although the probability of RPCA MAVRIC-SL being clinically useful was non-inferior to conventional MAVRIC-SL (within our accepted 10% difference, p < 0.05), except for visualization around the acetabular component. Abnormal imaging findings were seen in 25 hips, and either equally visible or visible but less conspicuous on RPCA MAVRIC-SL in 21 out of 25 cases. In 4 cases, a small joint effusion was queried on MAVRIC-SL but not RPCA MAVRIC-SL, but the presence or absence of a small effusion did not affect subsequent clinical management and patient outcome. CONCLUSION: While the overall image quality is reduced, RPCA MAVRIC-SL allows for significantly reduced scan time and maintains almost equal diagnostic performance.

Combined 5-minute double-echo in steady-state with separated echoes and 2-minute proton-density-weighted 2D FSE sequence for comprehensive whole-joint knee MRI assessment.

BACKGROUND: Clinical knee MRI protocols require upwards of 15 minutes of scan time. PURPOSE/HYPOTHESIS: To compare the imaging appearance of knee abnormalities depicted with a 5-minute 3D double-echo in steady-state (DESS) sequence with separate echo images, with that of a routine clinical knee MRI protocol. A secondary goal was to compare the imaging appearance of knee abnormalities depicted with 5-minute DESS paired with a 2-minute coronal proton-density fat-saturated (PDFS) sequence. STUDY TYPE: Prospective. SUBJECTS: Thirty-six consecutive patients (19 male) referred for a routine knee MRI. FIELD STRENGTH/SEQUENCES: DESS and PDFS at 3T. ASSESSMENT: Five musculoskeletal radiologists evaluated all images for the presence of internal knee derangement using DESS, DESS+PDFS, and the conventional imaging protocol, and their associated diagnostic confidence of the reading. STATISTICAL TESTS: Differences in positive and negative percent agreement (PPA and NPA, respectively) and 95% confidence intervals (CIs) for DESS and DESS+PDFS compared with the conventional protocol were calculated and tested using exact McNemar tests. The percentage of observations where DESS or DESS+PDFS had equivalent confidence ratings to DESS+Conv were tested with exact symmetry tests. Interreader agreement was calculated using Krippendorff's alpha. RESULTS: DESS had a PPA of 90% (88-92% CI) and NPA of 99% (99-99% CI). DESS+PDFS had increased PPA of 99% (95-99% CI) and NPA of 100% (99-100% CI) compared with DESS (both P < 0.001). DESS had equivalent diagnostic confidence to DESS+Conv in 94% of findings, whereas DESS+PDFS had equivalent diagnostic confidence in 99% of findings (both P < 0.001). All readers had moderate concordance for all three protocols (Krippendorff's alpha 47-48%). DATA CONCLUSION: Both 1) 5-minute 3D-DESS with separated echoes and 2) 5-minute 3D-DESS paired with a 2-minute coronal PDFS sequence depicted knee abnormalities similarly to a routine clinical knee MRI protocol, which may be a promising technique for abbreviated knee MRI. LEVEL OF EVIDENCE: 2 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2018.

Deep-learning-assisted diagnosis for knee magnetic resonance imaging: Development and retrospective validation of MRNet.

BACKGROUND: Magnetic resonance imaging (MRI) of the knee is the preferred method for diagnosing knee injuries. However, interpretation of knee MRI is time-intensive and subject to diagnostic error and variability. An automated system for interpreting knee MRI could prioritize high-risk patients and assist clinicians in making diagnoses. Deep learning methods, in being able to automatically learn layers of features, are well suited for modeling the complex relationships between medical images and their interpretations. In this study we developed a deep learning model for detecting general abnormalities and specific diagnoses (anterior cruciate ligament [ACL] tears and meniscal tears) on knee MRI exams. We then measured the effect of providing the model's predictions to clinical experts during interpretation. METHODS AND FINDINGS: Our dataset consisted of 1,370 knee MRI exams performed at Stanford University Medical Center between January 1, 2001, and December 31, 2012 (mean age 38.0 years; 569 [41.5%] female patients). The majority vote of 3 musculoskeletal radiologists established reference standard labels on an internal validation set of 120 exams. We developed MRNet, a convolutional neural network for classifying MRI series and combined predictions from 3 series per exam using logistic regression. In detecting abnormalities, ACL tears, and meniscal tears, this model achieved area under the receiver operating characteristic curve (AUC) values of 0.937 (95% CI 0.895, 0.980), 0.965 (95% CI 0.938, 0.993), and 0.847 (95% CI 0.780, 0.914), respectively, on the internal validation set. We also obtained a public dataset of 917 exams with sagittal T1-weighted series and labels for ACL injury from Clinical Hospital Centre Rijeka, Croatia. On the external validation set of 183 exams, the MRNet trained on Stanford sagittal T2-weighted series achieved an AUC of 0.824 (95% CI 0.757, 0.892) in the detection of ACL injuries with no additional training, while an MRNet trained on the rest of the external data achieved an AUC of 0.911 (95% CI 0.864, 0.958). We additionally measured the specificity, sensitivity, and accuracy of 9 clinical experts (7 board-certified general radiologists and 2 orthopedic surgeons) on the internal validation set both with and without model assistance. Using a 2-sided Pearson's chi-squared test with adjustment for multiple comparisons, we found no significant differences between the performance of the model and that of unassisted general radiologists in detecting abnormalities. General radiologists achieved significantly higher sensitivity in detecting ACL tears (p-value = 0.002; q-value = 0.019) and significantly higher specificity in detecting meniscal tears (p-value = 0.003; q-value = 0.019). Using a 1-tailed t test on the change in performance metrics, we found that providing model predictions significantly increased clinical experts' specificity in identifying ACL tears (p-value < 0.001; q-value = 0.006). The primary limitations of our study include lack of surgical ground truth and the small size of the panel of clinical experts. CONCLUSIONS: Our deep learning model can rapidly generate accurate clinical pathology classifications of knee MRI exams from both internal and external datasets. Moreover, our results support the assertion that deep learning models can improve the performance of clinical experts during medical imaging interpretation. Further research is needed to validate the model prospectively and to determine its utility in the clinical setting.

Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: Application to bone tumor radiographs.

BACKGROUND: The majority of current medical CBIR systems perform retrieval based only on "imaging signatures" generated by extracting pixel-level quantitative features, and only rarely has a feedback mechanism been incorporated to improve retrieval performance. In addition, current medical CBIR approaches do not routinely incorporate semantic terms that model the user's high-level expectations, and this can limit CBIR performance. METHOD: We propose a retrieval framework that exploits a hybrid feature space (HFS) that is built by integrating low-level image features and high-level semantic terms, through rounds of relevance feedback (RF) and performs similarity-based retrieval to support semi-automatic image interpretation. The novelty of the proposed system is that it can impute the semantic features of the query image by reformulating the query vector representation in the HFS via user feedback. We implemented our framework as a prototype that performs the retrieval over a database of 811 radiographic images that contains 69 unique types of bone tumors. RESULTS: We evaluated the system performance by conducting independent reading sessions with two subspecialist musculoskeletal radiologists. For the test set, the proposed retrieval system at fourth RF iteration of the sessions conducted with both the radiologists achieved mean average precision (MAP) value ∼0.90 where the initial MAP with baseline CBIR was 0.20. In addition, we also achieved high prediction accuracy (>0.8) for the majority of the semantic features automatically predicted by the system. CONCLUSION: Our proposed framework addresses some limitations of existing CBIR systems by incorporating user feedback and simultaneously predicting the semantic features of the query image. This obviates the need for the user to provide those terms and makes CBIR search more efficient for inexperience users/trainees. Encouraging results achieved in the current study highlight possible new directions in radiological image interpretation employing semantic CBIR combined with relevance feedback of visual similarity.

Is it painful to be different? Sciatic nerve anatomical variants on MRI and their relationship to piriformis syndrome.

OBJECTIVES: To investigate the purported relationship between sciatic nerve variant anatomy and piriformis syndrome. METHODS: Over 49 months, 1039 consecutive noncontrast adult hip MRIs were completed for various clinical indications. Repeat and technically insufficient studies were excluded. Radiologists categorized sciatic nerve anatomy into Beaton and Anson anatomical types. Chart review using our institution's cohort search and navigation tool determined the prevalence of the explicit clinical diagnosis of piriformis syndrome (primary endpoint) and sciatica and buttock pain (secondary endpoints). A Z-test compared the prevalence of each diagnosis in the variant anatomy and normal groups. RESULTS: Seven hundred eighty-three studies were included, with sciatic nerve variants present in 150 hips (19.2%). None of the diagnoses had a statistically significant difference in prevalence between the variant and normal hip groups. Specifically, piriformis syndrome was present in 11.3% of variant hips compared with 9.0% of normal hips (p = 0.39). CONCLUSIONS: There were no significant differences in the prevalence of piriformis syndrome, buttock pain, or sciatica between normal and variant sciatic nerve anatomy. This large-scale correlative radiologic study into the relationship between sciatic nerve variants and piriformis syndrome calls into question this purported relationship. KEY POINTS: • Large retrospective study relating variant sciatic nerve anatomy, present in 19.2% of hip MRIs, and piriformis syndrome • While sciatic nerve variant anatomy has previously been implicated in piriformis syndrome in small studies, no relationship was identified between sciatic nerve variants and piriformis syndrome.

Bone Tumor Diagnosis Using a Naïve Bayesian Model of Demographic and Radiographic Features.

Because many bone tumors have a variety of appearances and are uncommon, few radiologists develop sufficient expertise to guide optimal management. Bayesian inference can guide decision-making by computing probabilities of multiple diagnoses to generate a differential. We built and validated a naïve Bayes machine (NBM) that processes 18 demographic and radiographic features. We reviewed over 1664 analog radiographic cases of bone tumors and selected 811 cases (66 diagnoses) for annotation using a quantitative imaging platform. Leave-one-out cross validation was performed. Primary accuracy was defined as the correct pathological diagnosis as the top machine prediction. Differential accuracy was defined as whether the correct pathological diagnosis was within the top three predictions. For the 29 most common diagnoses (710 cases), primary accuracy was 44%, and differential accuracy was 60%. For the top 10 most common diagnoses (478 cases), primary accuracy was 62%, and differential accuracy was 80%. The machine returned relevant diagnoses for the majority of unknown test cases and may be a feasible alternative to machine learning approaches such as deep neural networks or support vector machines that typically require larger training data (our model required a minimum of five samples per diagnosis) and are "black boxes" (our model can provide details of probability calculations to identify features that most significantly contribute to truth diagnoses). Finally, our Bayes model was designed to scale and "learn" from external data, enabling incorporation of outside knowledge such as Dahlin's Bone Tumors, a reference of anatomic and demographic statistics of more than 10,000 tumors.

Computerized Prediction of Radiological Observations Based on Quantitative Feature Analysis: Initial Experience in Liver Lesions.

We propose a computerized framework that, given a region of interest (ROI) circumscribing a lesion, not only predicts radiological observations related to the lesion characteristics with 83.2% average prediction accuracy but also derives explicit association between low-level imaging features and high-level semantic terms by exploiting their statistical correlation. Such direct association between semantic concepts and low-level imaging features can be leveraged to build a powerful annotation system for radiological images that not only allows the computer to infer the semantics from diverse medical images and run automatic reasoning for making diagnostic decision but also provides "human-interpretable explanation" of the system output to facilitate better end user understanding of computer-based diagnostic decisions. The core component of our framework is a radiological observation detection algorithm that maximizes the low-level imaging feature relevancy for each high-level semantic term. On a liver lesion CT dataset, we have implemented our framework by incorporating a large set of state-of-the-art low-level imaging features. Additionally, we included a novel feature that quantifies lesion(s) present within the liver that have a similar appearance as the primary lesion identified by the radiologist. Our framework achieved a high prediction accuracy (83.2%), and the derived association between semantic concepts and imaging features closely correlates with human expectation. The framework has been only tested on liver lesion CT images, but it is capable of being applied to other imaging domains.

Detection and prevalence of variant sciatic nerve anatomy in relation to the piriformis muscle on MRI.

OBJECTIVE: To determine whether known variant anatomical relationships between the sciatic nerve and piriformis muscle can be identified on routine MRI studies of the hip and to establish their imaging prevalence. METHODS: Hip MRI studies acquired over a period of 4 years at two medical centers underwent retrospective interpretation. Anatomical relationship between the sciatic nerve and the piriformis muscle was categorized according to the Beaton and Anson classification system. The presence of a split sciatic nerve at the level of the ischial tuberosity was also recorded. RESULTS: A total of 755 consecutive scans were reviewed. Conventional anatomy (type I), in which an undivided sciatic nerve passes below the piriformis muscle, was identified in 87% of cases. The remaining 13% of cases demonstrated a type II pattern in which one division of the sciatic nerve passes through the piriformis whereas the second passes below. Only two other instances of variant anatomy were identified (both type III). Most variant cases were associated with a split sciatic nerve at the level of the ischial tuberosity (73 out of 111, 65.8%). By contrast, only 6% of cases demonstrated a split sciatic nerve at this level in the context of otherwise conventional anatomy. CONCLUSION: Anatomical variations of the sciatic nerve course in relation to the piriformis muscle are frequently identified on routine MRI of the hips, occurring in 12-20% of scans reviewed. Almost all variants identified were type II. The ability to recognize variant sciatic nerve courses on MRI may prove useful in optimal treatment planning.

Adaptive local window for level set segmentation of CT and MRI liver lesions.

We propose a novel method, the adaptive local window, for improving level set segmentation technique. The window is estimated separately for each contour point, over iterations of the segmentation process, and for each individual object. Our method considers the object scale, the spatial texture, and the changes of the energy functional over iterations. Global and local statistics are considered by calculating several gray level co-occurrence matrices. We demonstrate the capabilities of the method in the domain of medical imaging for segmenting 233 images with liver lesions. To illustrate the strength of our method, those lesions were screened by either Computed Tomography or Magnetic Resonance Imaging. Moreover, we analyzed images using three different energy models. We compared our method to a global level set segmentation, to a local framework that uses predefined fixed-size square windows and to a local region-scalable fitting model. The results indicate that our proposed method outperforms the other methods in terms of agreement with the manual marking and dependence on contour initialization or the energy model used. In case of complex lesions, such as low contrast lesions, heterogeneous lesions, or lesions with a noisy background, our method shows significantly better segmentation with an improvement of 0.25 ± 0.13 in Dice similarity coefficient, compared with state of the art fixed-size local windows (Wilcoxon, p < 0.001).

Chondral Rib Fractures in Professional American Football: Two Cases and Current Practice Patterns Among NFL Team Physicians.

BACKGROUND: Although a recognized and discussed injury, chondral rib fractures in professional American football have not been previously reported in the literature. There currently exists no consensus on how to identify and treat these injuries or the expected return to play for the athlete. PURPOSE: To present 2 cases of chondral rib injuries in the National Football League (NFL) and discuss the current practice patterns for management of these injuries among the NFL team physicians. STUDY DESIGN: Case series; Level of evidence, 4. METHODS: Two cases of NFL players with chondral rib injuries are presented. A survey regarding work-up and treatment of these injuries was completed by team physicians at the 2014 NFL Combine. Our experience in identifying and treating these injuries is presented in conjunction with a survey of NFL team physicians' experiences. RESULTS: Two cases of rib chondral injuries were diagnosed by computed tomography (CT) and treated with rest and protective splinting. Return to play was 2 to 4 weeks. NFL Combine survey results show that NFL team physicians see a mean of 4 costal cartilage injuries per 5-year period, or approximately 1 case per year per team. Seventy percent of team physicians use CT scanning and 43% use magnetic resonance imaging for diagnosis of these injuries. An anesthetic block is used acutely in 57% and only electively in subsequent games by 39%. CONCLUSION: A high index of suspicion is necessary to diagnose chondral rib injuries in American football. CT scan is most commonly used to confirm diagnosis. Return to play can take up to 2 to 4 weeks with a protective device, although anesthetic blocks can be used to potentially expedite return. CLINICAL RELEVANCE: Chondral rib injuries are common among NFL football players, while there is no literature to support proper diagnosis and treatment of these injuries or expected duration of recovery. These injuries are likely common in other contact sports and levels of competition as well. Our series combined with NFL team physician survey results can aid team physicians in identifying these injuries, obtaining useful imaging, and counseling players and coaches and the expected time of recovery.

Improved Patch-Based Automated Liver Lesion Classification by Separate Analysis of the Interior and Boundary Regions.

The bag-of-visual-words (BoVW) method with construction of a single dictionary of visual words has been used previously for a variety of classification tasks in medical imaging, including the diagnosis of liver lesions. In this paper, we describe a novel method for automated diagnosis of liver lesions in portal-phase computed tomography (CT) images that improves over single-dictionary BoVW methods by using an image patch representation of the interior and boundary regions of the lesions. Our approach captures characteristics of the lesion margin and of the lesion interior by creating two separate dictionaries for the margin and the interior regions of lesions ("dual dictionaries" of visual words). Based on these dictionaries, visual word histograms are generated for each region of interest within the lesion and its margin. For validation of our approach, we used two datasets from two different institutions, containing CT images of 194 liver lesions (61 cysts, 80 metastasis, and 53 hemangiomas). The final diagnosis of each lesion was established by radiologists. The classification accuracy for the images from the two institutions was 99% and 88%, respectively, and 93% for a combined dataset. Our new BoVW approach that uses dual dictionaries shows promising results. We believe the benefits of our approach may generalize to other application domains within radiology.

Piriformis Syndrome With Variant Sciatic Nerve Anatomy: A Case Report.

A 68-year-old male long distance runner presented with low back and left buttock pain, which eventually progressed to severe and debilitating pain, intermittently radiating to the posterior thigh and foot. A comprehensive workup ruled out possible spine or hip causes of his symptoms. A pelvic magnetic resonance imaging neurogram with complex oblique planes through the piriformis demonstrated variant anatomy of the left sciatic nerve consistent with the clinical diagnosis of piriformis syndrome. The patient ultimately underwent neurolysis with release of the sciatic nerve and partial resection of the piriformis muscle. After surgery the patient reported significant pain reduction and resumed running 3 months later. Piriformis syndrome is uncommon but should be considered in the differential diagnosis for buttock pain. Advanced imaging was essential to guide management.

Content-based image retrieval in radiology: analysis of variability in human perception of similarity.

We aim to develop a better understanding of perception of similarity in focal computed tomography (CT) liver images to determine the feasibility of techniques for developing reference sets for training and validating content-based image retrieval systems. In an observer study, four radiologists and six nonradiologists assessed overall similarity and similarity in 5 image features in 136 pairs of focal CT liver lesions. We computed intra- and inter-reader agreements in these similarity ratings and viewed the distributions of the ratings. The readers' ratings of overall similarity and similarity in each feature primarily appeared to be bimodally distributed. Median Kappa scores for intra-reader agreement ranged from 0.57 to 0.86 in the five features and from 0.72 to 0.82 for overall similarity. Median Kappa scores for inter-reader agreement ranged from 0.24 to 0.58 in the five features and were 0.39 for overall similarity. There was no significant difference in agreement for radiologists and nonradiologists. Our results show that developing perceptual similarity reference standards is a complex task. Moderate to high inter-reader variability precludes ease of dividing up the workload of rating perceptual similarity among many readers, while low intra-reader variability may make it possible to acquire large volumes of data by asking readers to view image pairs over many sessions.

Weighted locality-constrained linear coding for lesion classification in CT images.

Computed tomography is a popular imaging modality for detecting abnormalities associated with abdominal organs such as the liver, kidney and uterus. In this paper, we propose a novel weighted locality-constrained linear coding (LLC) method followed by a weighted max-pooling method to classify liver lesions into three classes: cysts, metastases, hemangiomas. We first divide the lesions into same-size patches. Then, we extract the raw features in all patches followed by Principal Components Analysis (PCA) and apply K means to obtain a single LLC dictionary. Since the interior lesion patches and the boundary patches contribute different information in the image, we assign different weights on these two types of patches to obtain the LLC codes. Moreover, a weighted max pooling approach is also proposed to further evaluate the importance of these two types of patches in feature pooling. Experiments on 109 images of liver lesions were carried out to validate the proposed method. The proposed method achieves a best lesion classification accuracy of 96.33%, which appears to be superior compared with traditional image coding methods: LLC method and Bag-of-words method (BoW) and traditional features: Local Binary Pattern (LBP) features, uniform LBP and complete LBP, demonstrating that the proposed method provides better classification.

Classification of hypervascular liver lesions based on hepatic artery and portal vein blood supply coefficients calculated from triphasic CT scans.

Perfusion CT of the liver typically involves scanning the liver at least 20 times, resulting in a large radiation dose. We developed and validated a simplified model of tumor blood supply that can be applied to standard triphasic scans and evaluated whether this can be used to distinguish benign and malignant liver lesions. Triphasic CTs of 46 malignant and 32 benign liver lesions were analyzed. For each phase, regions of interest were drawn in the arterially enhancing portion of each lesion, as well as the background liver, aorta, and portal vein. Hepatic artery and portal vein blood supply coefficients for each lesion were then calculated by expressing the enhancement curve of the lesion as a linear combination of the enhancement curves of the aorta and portal vein. Hepatocellular carcinoma (HCC) and hypervascular metastases, on average, both had increased hepatic artery coefficients compared to the background liver. Compared to HCC, benign lesions, on average, had either a greater hepatic artery coefficient (hemangioma) or a greater portal vein coefficient (focal nodular hyperplasia or transient hepatic attenuation difference). Hypervascularity with washout is a key diagnostic criterion for HCC, but it had a sensitivity of 72 % and specificity of 81 % for diagnosing malignancy in our diverse set of liver lesions. The sensitivity for malignancy was increased to 89 % by including enhancing lesions that were hypodense on all phases. The specificity for malignancy was increased to 97 % (p = 0.039) by also examining hepatic artery and portal vein blood supply coefficients, while maintaining a sensitivity of 76 %.

Ultrasound-guided musculoskeletal interventions in American football: 18 years of experience.

OBJECTIVE: Myotendinous strains, contusions, and hematomas are common injuries in American football. Along with ligament sprains and inflammatory disorders, musculoskeletal injuries often result in lost participation time. This article summarizes 18 years of experience with 128 ultrasound-guided drainages and injections in 69 football players with 88 injuries. CONCLUSION: When performed by an operator with sufficient expertise in diagnostic and procedural skills, ultrasound-guided musculoskeletal interventions are minimally invasive, are safe, and can play an integral role in injury management.

On combining image-based and ontological semantic dissimilarities for medical image retrieval applications.

Computer-assisted image retrieval applications can assist radiologists by identifying similar images in archives as a means to providing decision support. In the classical case, images are described using low-level features extracted from their contents, and an appropriate distance is used to find the best matches in the feature space. However, using low-level image features to fully capture the visual appearance of diseases is challenging and the semantic gap between these features and the high-level visual concepts in radiology may impair the system performance. To deal with this issue, the use of semantic terms to provide high-level descriptions of radiological image contents has recently been advocated. Nevertheless, most of the existing semantic image retrieval strategies are limited by two factors: they require manual annotation of the images using semantic terms and they ignore the intrinsic visual and semantic relationships between these annotations during the comparison of the images. Based on these considerations, we propose an image retrieval framework based on semantic features that relies on two main strategies: (1) automatic "soft" prediction of ontological terms that describe the image contents from multi-scale Riesz wavelets and (2) retrieval of similar images by evaluating the similarity between their annotations using a new term dissimilarity measure, which takes into account both image-based and ontological term relations. The combination of these strategies provides a means of accurately retrieving similar images in databases based on image annotations and can be considered as a potential solution to the semantic gap problem. We validated this approach in the context of the retrieval of liver lesions from computed tomographic (CT) images and annotated with semantic terms of the RadLex ontology. The relevance of the retrieval results was assessed using two protocols: evaluation relative to a dissimilarity reference standard defined for pairs of images on a 25-images dataset, and evaluation relative to the diagnoses of the retrieved images on a 72-images dataset. A normalized discounted cumulative gain (NDCG) score of more than 0.92 was obtained with the first protocol, while AUC scores of more than 0.77 were obtained with the second protocol. This automatical approach could provide real-time decision support to radiologists by showing them similar images with associated diagnoses and, where available, responses to therapies.

A hierarchical knowledge-based approach for retrieving similar medical images described with semantic annotations.

Computer-assisted image retrieval applications could assist radiologist interpretations by identifying similar images in large archives as a means to providing decision support. However, the semantic gap between low-level image features and their high level semantics may impair the system performances. Indeed, it can be challenging to comprehensively characterize the images using low-level imaging features to fully capture the visual appearance of diseases on images, and recently the use of semantic terms has been advocated to provide semantic descriptions of the visual contents of images. However, most of the existing image retrieval strategies do not consider the intrinsic properties of these terms during the comparison of the images beyond treating them as simple binary (presence/absence) features. We propose a new framework that includes semantic features in images and that enables retrieval of similar images in large databases based on their semantic relations. It is based on two main steps: (1) annotation of the images with semantic terms extracted from an ontology, and (2) evaluation of the similarity of image pairs by computing the similarity between the terms using the Hierarchical Semantic-Based Distance (HSBD) coupled to an ontological measure. The combination of these two steps provides a means of capturing the semantic correlations among the terms used to characterize the images that can be considered as a potential solution to deal with the semantic gap problem. We validate this approach in the context of the retrieval and the classification of 2D regions of interest (ROIs) extracted from computed tomographic (CT) images of the liver. Under this framework, retrieval accuracy of more than 0.96 was obtained on a 30-images dataset using the Normalized Discounted Cumulative Gain (NDCG) index that is a standard technique used to measure the effectiveness of information retrieval algorithms when a separate reference standard is available. Classification results of more than 95% were obtained on a 77-images dataset. For comparison purpose, the use of the Earth Mover's Distance (EMD), which is an alternative distance metric that considers all the existing relations among the terms, led to results retrieval accuracy of 0.95 and classification results of 93% with a higher computational cost. The results provided by the presented framework are competitive with the state-of-the-art and emphasize the usefulness of the proposed methodology for radiology image retrieval and classification.