Improving the Reliability of Peer Review Without a Gold Standard.

Peer review plays a crucial role in accreditation and credentialing processes as it can identify outliers and foster a peer learning approach, facilitating error analysis and knowledge sharing. However, traditional peer review methods may fall short in effectively addressing the interpretive variability among reviewing and primary reading radiologists, hindering scalability and effectiveness. Reducing this variability is key to enhancing the reliability of results and instilling confidence in the review process. In this paper, we propose a novel statistical approach called "Bayesian Inter-Reviewer Agreement Rate" (BIRAR) that integrates radiologist variability. By doing so, BIRAR aims to enhance the accuracy and consistency of peer review assessments, providing physicians involved in quality improvement and peer learning programs with valuable and reliable insights. A computer simulation was designed to assign predefined interpretive error rates to hypothetical interpreting and peer-reviewing radiologists. The Monte Carlo simulation then sampled (100 samples per experiment) the data that would be generated by peer reviews. The performances of BIRAR and four other peer review methods for measuring interpretive error rates were then evaluated, including a method that uses a gold standard diagnosis. Application of the BIRAR method resulted in 93% and 79% higher relative accuracy and 43% and 66% lower relative variability, compared to "Single/Standard" and "Majority Panel" peer review methods, respectively. Accuracy was defined by the median difference of Monte Carlo simulations between measured and pre-defined "actual" interpretive error rates. Variability was defined by the 95% CI around the median difference of Monte Carlo simulations between measured and pre-defined "actual" interpretive error rates. BIRAR is a practical and scalable peer review method that produces more accurate and less variable assessments of interpretive quality by accounting for variability within the group's radiologists, implicitly applying a standard derived from the level of consensus within the group across various types of interpretive findings.

Variability in diagnostic error rates of 10 MRI centers performing lumbar spine MRI examinations on the same patient within a 3-week period.

BACKGROUND CONTEXT: In today's health-care climate, magnetic resonance imaging (MRI) is often perceived as a commodity-a service where there are no meaningful differences in quality and thus an area in which patients can be advised to select a provider based on price and convenience alone. If this prevailing view is correct, then a patient should expect to receive the same radiological diagnosis regardless of which imaging center he or she visits, or which radiologist reviews the examination. Based on their extensive clinical experience, the authors believe that this assumption is not correct and that it can negatively impact patient care, outcomes, and costs. PURPOSE: This study is designed to test the authors' hypothesis that radiologists' reports from multiple imaging centers performing a lumbar MRI examination on the same patient over a short period of time will have (1) marked variability in interpretive findings and (2) a broad range of interpretive errors. STUDY DESIGN: This is a prospective observational study comparing the interpretive findings reported for one patient scanned at 10 different MRI centers over a period of 3 weeks to each other and to reference MRI examinations performed immediately preceding and following the 10 MRI examinations. PATIENT SAMPLE: The sample is a 63-year-old woman with a history of low back pain and right L5 radicular symptoms. OUTCOME MEASURES: Variability was quantified using percent agreement rates and Fleiss kappa statistic. Interpretive errors were quantified using true-positive counts, false-positive counts, false-negative counts, true-positive rate (sensitivity), and false-negative rate (miss rate). METHODS: Interpretive findings from 10 study MRI examinations were tabulated and compared for variability and errors. Two of the authors, both subspecialist spine radiologists from different institutions, independently reviewed the reference examinations and then came to a final diagnosis by consensus. Errors of interpretation in the study examinations were considered present if a finding present or not present in the study examination's report was not present in the reference examinations. RESULTS: Across all 10 study examinations, there were 49 distinct findings reported related to the presence of a distinct pathology at a specific motion segment. Zero interpretive findings were reported in all 10 study examinations and only one finding was reported in nine out of 10 study examinations. Of the interpretive findings, 32.7% appeared only once across all 10 of the study examinations' reports. A global Fleiss kappa statistic, computed across all reported findings, was 0.20±0.06, indicating poor overall agreement on interpretive findings. The average interpretive error count in the study examinations was 12.5±3.2 (both false-positives and false-negatives). The average false-negative count per examination was 10.9±2.9 out of 25 and the average false-positive count was 1.6±0.9, which correspond to an average true-positive rate (sensitivity) of 56.4%±11.7 and miss rate of 43.6%±11.7. CONCLUSIONS: This study found marked variability in the reported interpretive findings and a high prevalence of interpretive errors in radiologists' reports of an MRI examination of the lumbar spine performed on the same patient at 10 different MRI centers over a short time period. As a result, the authors conclude that where a patient obtains his or her MRI examination and which radiologist interprets the examination may have a direct impact on radiological diagnosis, subsequent choice of treatment, and clinical outcome.

Image-guided and -monitored renal artery stenting using only MRI.

PURPOSE: To demonstrate the ability of a unique interventional MR system to be used safely and effectively as the only imaging modality for all phases of MR-guided stent-supported angioplasty. MATERIALS AND METHODS: An experimental disease model of renal stenosis was created in six pigs. An interventional MR system, which employed previously reported tools for real-time catheter tracking with automated scan-plane positioning, adaptive image parameters, and radial true-FISP imaging with steady-state precession (True-FISP) imaging coupled with a high-speed reconstruction technique, was then used to guide all phases of the intervention, including: guidewire and catheter insertion, stent deployment, and confirmation of therapeutic success. Pre- and postprocedural X-ray imaging was used as a gold standard to validate the experimental results. RESULTS: All of the stent-supported angioplasty interventions were a technical success and were performed without complications. The average postoperative residual stenosis was 14.9%. The image guidance enabled the stents to be deployed with an accuracy of 0.98 +/- 0.69 mm. Additionally, using this interventional MRI system to guide renal artery stenting significantly reduces the procedure time, as compared to using X-ray fluoroscopy. CONCLUSION: This study has clearly demonstrated the first successful treatment of renal artery stenting in an experimental animal model solely under MRI guidance and monitoring.

An augmented reality system for MR image-guided needle biopsy: initial results in a swine model.

PURPOSE: To evaluate an augmented reality (AR) system in combination with a 1.5-T closed-bore magnetic resonance (MR) imager as a navigation tool for needle biopsies. MATERIALS AND METHODS: The experimental protocol had institutional animal care and use committee approval. Seventy biopsies were performed in phantoms by using 20 tube targets, each with a diameter of 6 mm, and 50 virtual targets. The position of the needle tip in AR and MR space was compared in multiple imaging planes, and virtual and real needle tip localization errors were calculated. Ten AR-guided biopsies were performed in three pigs, and the duration of each procedure was determined. After successful puncture, the distance to the target was measured on MR images. The confidence limits for the achieved in-plane hit rate and for lateral deviation were calculated. A repeated measures analysis of variance was used to determine whether the placement error in a particular dimension (x, y, or z) differed from the others. RESULTS: For the 50 virtual targets, a mean error of 1.1 mm +/- 0.5 (standard deviation) was calculated. A repeated measures analysis of variance indicated no statistically significant difference (P > .99) in the errors in any particular orientation. For the real targets, all punctures were inside the 6-mm-diameter tube in the transverse plane. The needle depth was within the target plane in 11 biopsy procedures; the mean distance to the center of the target was 2.55 mm (95% confidence interval: 1.77 mm, 3.34 mm). For nine biopsy procedures, the needle tip was outside the target plane, with a mean distance to the edge of the target plane of 1.5 mm (range, 0.07-3.46 mm). In the animal experiments, the puncture was successful in all 10 cases, with a mean target-needle distance of 9.6 mm +/- 4.85. The average procedure time was 18 minutes per puncture. CONCLUSION: Biopsy procedures performed with a combination of a closed-bore MR system and an AR system are feasible and accurate.

A review of technical advances in interventional magnetic resonance imaging.

Initial research in the development of interventional magnetic resonance (MR) imaging in the late 1980s and early to mid-1990s focused on pulse sequences, devices, and clinical applications. This focus was largely a result of the limited number of areas in which the academic research community leading the development could provide innovation on the MR systems of the time. However, during the past decade, computational power, higher bandwidth graphical displays, faster computer networks, improved pulse sequence architectures, and improved technical specifications have accelerated the pace of development on modern MR systems. Today, it is the combination of multiple system factors that are enabling the future of interventional MR. These developments, their impact on the field, and newly emerging applications are described.

MR imaging-guided percutaneous angioplasty and stent placement in a swine model comparison of open- and closed-bore scanners.

RATIONALE AND OBJECTIVES: The purpose of this study is to compare the feasibility and precision of renal artery angioplasty and stent placement using two different MR scanners. MATERIALS AND METHODS: MR imaging-guided angioplasty and stent placements were performed on seven pigs using 0.2 and 1.5 T scanners (Magnetom Open and Magnetom Sonata, Siemens Medical Solutions, Erlangen, Germany). For guidance of catheters, guide wires and stents susceptibility artifact-based tracking was used. The end point of each intervention was to position a stent in the renal artery with its proximal end at the level of the aortic wall. Procedure time and stent position were evaluated. RESULTS: Catheterization, angioplasty, and stent placement were feasible using MRI guidance at both 0.2 and 1,5 Tesla. At 1.5 T all catheter manipulations and interventions were performed in less than 30 minutes. At 0.2 T the interventions took up to 90 minutes. No significant difference in the stent deviation was noted between the two scanners. CONCLUSION: The use of a high-performance 1.5 T scanner helped to reduce the procedure time to half of that of a low-field system. Since no difference in stent placement precision was noted, a dedicated MR-stent might be mandatory for more precise stent placement.

The catheter-driven MRI scanner: a new approach to intravascular catheter tracking and imaging-parameter adjustment for interventional MRI.

OBJECTIVE: Our aim was to test the feasibility of a hands-free approach to MRI that allows the interventionalist to track an angiographic catheter in real time throughout the procedure and to automatically change imaging parameters by catheter manipulation. MATERIALS AND METHODS: A tracking method that is based on an active device localization was implemented on a 1.5-T MRI scanner. The system determines the current position and orientation of a catheter in 3D space in an endless feedback loop. Automatic scanning plane-adjustment procedures written in the software of the MRI system ensure image acquisition at the location of the catheter tip. The system calculates the device velocity to automatically adjust parameters such as field of view (FOV) and resolution. To evaluate the feasibility and performance in vivo and ex vivo, we performed experiments in two vessel phantoms and on six pigs. RESULTS: The system collected the tracking data within 40 msec; an additional 10-20 msec was then required to perform the localization and velocity calculations and to update the image parameters. The system could localize a motionless catheter in the aorta in 100% and a moving catheter in 98% of measured attempts. The system responded in real time to changes in device velocity by dynamically adjusting spatial resolution and FOV in both phantom and porcine trials. Using this technique, we successfully catheterized the renal artery in two pigs. CONCLUSION: Active tracking, combined with automatic scanning plane and imaging parameter adjustment, provides an intuitive MRI scanner interface for the guidance of the vascular procedure.

Active device tracking and high-resolution intravascular MRI using a novel catheter-based, opposed-solenoid phased array coil.

A novel two-element, catheter-based phased array coil was designed and built for both active MR device tracking and high-resolution vessel wall imaging. The device consists of two independent solenoid coils that are wound in opposite directions, connected to separate receive channels, and mounted collinearly on an angiographic catheter. The elements were used independently or together for tracking or imaging applications, respectively. The array's dual functionality was tested on a clinical 1.5 T MRI scanner in vitro, in vivo, and in situ. During real-time catheter tracking, each element gave rise to a high-amplitude peak in the respective projection data, which enabled reliable and robust device tracking as well as automated slice positioning. In vivo microimaging with 240 microm in-plane resolution was achieved in 9 s using the device and TrueFISP imaging. Therefore, a single device was successfully implemented that met the combined requirements of intravascular device tracking and imaging.

Real-time catheter tracking and adaptive imaging.

PURPOSE: To evaluate the performance of a real-time MR system for interventional procedures that adjusts specific image parameters in real time based on a catheter's speed of insertion. MATERIALS AND METHODS: The system was implemented using only the hardware provided with a standard short-bore 1.5 T scanner (Siemens Magnetom Sonata) (with the exception of small tracking markers affixed to the catheter). The system tracks the position of an MR microcoil-instrumented catheter and automatically updates the scan plane's position and orientation, as well as other features, including, but not limited to, field of view, resolution, tip angle, and TE. A real-time feedback loop continuously localizes the tracking markers, updates the scan plane position and orientation, calculates the catheter's speed, adjusts the value of specific image parameters, then collects new image data, reconstructs an image, and provides it for immediate display. The system was evaluated in phantom and in vivo porcine experiments. RESULTS: The system is able to accurately localize a moving catheter in the abdominal aorta, calculate the device speed, and respond by adjusting specified image parameters 98% of the time, with precision of approximately 2 mm and 1.5 degrees. CONCLUSION: Simply slowing the speed of the catheter allows the clinician to adjust predetermined image parameters. This work also has the potential to build a degree of intelligence into the scanner, enabling it to react to changes in the clinical environment and automatically optimize specific image parameters.