Factors Associated With Timeliness and Equity of Access to Outpatient MRI Examinations.

OBJECTIVE: The role of MRI in guiding patients' diagnosis and treatment is increasing. Therefore, timely MRI performance prevents delays that can impact patient care. We assessed the timeliness of performing outpatient MRIs using the socio-ecological model approach and evaluated multilevel factors associated with delays. METHODS: This institutional review board-approved study included outpatient MRI examinations ordered between October 1, 2021, and December 31, 2022, for performance at a large quaternary care health system. Mean order-to-performed (OtoP) interval (in days) and prolonged OtoP interval (defined as >10 days) for MRI orders with an expected date of 1 day to examination performance were measured. Logistic regression was used to assess patient-level (demographic and social determinants of health), radiology practice-level, and community-level factors associated with prolonged OtoP interval. RESULTS: There were 126,079 MRI examination orders with expected performance within 1 day placed during the study period (56% of all MRI orders placed). After excluding duplicates, there were 97,160 orders for unique patients. Of the MRI orders, 48% had a prolonged OtoP interval, and mean OtoP interval was 18.5 days. Factors significantly associated with delay in MRI performance included public insurance (odds ratio [OR] = 1.11, P < .001), female gender (OR = 1.11, P < .001), radiology subspecialty (ie, cardiac, OR = 1.71, P < .001), and patients from areas that are most deprived (ie, highest Area Deprivation Index quintile, OR = 1.70, P < .001). DISCUSSION: Nearly half of outpatient MRI orders were delayed, performed >10 days from the expected date selected by the ordering provider. Addressing multilevel factors associated with such delays may help enhance timeliness and equity of access to MRI examinations, potentially reducing diagnostic errors and treatment delays.

Evaluation of the Aggregated Time Savings in Adopting Fast Brain MRI Techniques for Outpatient Brain MRI.

INTRODUCTION: Clinical validation studies have demonstrated the ability of accelerated MRI sequences to decrease acquisition time and motion artifact while preserving image quality. The operational benefits, however, have been less explored. Here, we report our initial clinical experience in implementing fast MRI techniques for outpatient brain imaging during the COVID-19 pandemic. METHODS: Aggregate acquisition times were extracted from the medical record on consecutive imaging examinations performed during matched pre-implementation (7/1/2019-12/31/2019) and post-implementation periods (7/1/2020-12/31/2020). Expected acquisition time reduction for each MRI protocol was calculated through manual collection of acquisition times for the conventional and accelerated sequences performed during the pre- and post-implementation periods. Aggregate and expected acquisition times were compared for the five most frequently performed brain MRI protocols: brain without contrast (BR-), brain with and without contrast (BR+), multiple sclerosis (MS), memory loss (MML), and epilepsy (EPL). RESULTS: The expected time reductions for BR-, BR+, MS, MML, and EPL protocols were 6.6 min, 11.9 min, 14 min, 10.8 min, and 14.1 min, respectively. The overall median aggregate acquisition time was 31 [25, 36] min for the pre-implementation period and 18 [15, 22] min for the post-implementation period, with a difference of 13 min (42%). The median acquisition time was reduced by 4 min (25%) for BR-, 14.0 min (44%) for BR+, 14 min (38%) for MS, 11 min (52%) for MML, and 16 min (35%) for EPL. CONCLUSION: The implementation of fast brain MRI sequences significantly reduced the acquisition times for the most commonly performed outpatient brain MRI protocols.

Predicting health crises from early warning signs in patient medical records.

The COVID-19 global pandemic has caused unprecedented worldwide changes in healthcare delivery. While containment and mitigation approaches have been intensified, the progressive increase in the number of cases has overwhelmed health systems globally, highlighting the need for anticipation and prediction to be the basis of an efficient response system. This study demonstrates the role of population health metrics as early warning signs of future health crises. We retrospectively collected data from the emergency department of a large academic hospital in the northeastern United States from 01/01/2019 to 08/07/2021. A total of 377,694 patient records and 303 features were included for analysis. Departing from a multivariate artificial intelligence (AI) model initially developed to predict the risk of high-flow oxygen therapy or mechanical ventilation requirement during the COVID-19 pandemic, a total of 19 original variables and eight engineered features showing to be most predictive of the outcome were selected for further analysis. The temporal trends of the selected variables before and during the pandemic were characterized to determine their potential roles as early warning signs of future health crises. Temporal analysis of the individual variables included in the high-flow oxygen model showed that at a population level, the respiratory rate, temperature, low oxygen saturation, number of diagnoses during the first encounter, heart rate, BMI, age, sex, and neutrophil percentage demonstrated observable and traceable changes eight weeks before the first COVID-19 public health emergency declaration. Additionally, the engineered rule-based features built from the original variables also exhibited a pre-pandemic surge that preceded the first pandemic wave in spring 2020. Our findings suggest that the changes in routine population health metrics may serve as early warnings of future crises. This justifies the development of patient health surveillance systems, that can continuously monitor population health features, and alarm of new approaching public health crises before they become devastating.

Human knowledge models: Learning applied knowledge from the data.

Artificial intelligence and machine learning have demonstrated remarkable results in science and applied work. However, present AI models, developed to be run on computers but used in human-driven applications, create a visible disconnect between AI forms of processing and human ways of discovering and using knowledge. In this work, we introduce a new concept of "Human Knowledge Models" (HKMs), designed to reproduce human computational abilities. Departing from a vast body of cognitive research, we formalized the definition of HKMs into a new form of machine learning. Then, by training the models with human processing capabilities, we learned human-like knowledge, that humans can not only understand, but also compute, modify, and apply. We used several datasets from different applied fields to demonstrate the advantages of HKMs, including their high predictive power and resistance to noise and overfitting. Our results proved that HKMs can efficiently mine knowledge directly from the data and can compete with complex AI models in explaining the main data patterns. As a result, our study reveals the great potential of HKMs, particularly in the decision-making applications where "black box" models cannot be accepted. Moreover, this improves our understanding of how well human decision-making, modeled by HKMs, can approach the ideal solutions in real-life problems.

Temporal quality degradation in AI models.

As AI models continue to advance into many real-life applications, their ability to maintain reliable quality over time becomes increasingly important. The principal challenge in this task stems from the very nature of current machine learning models, dependent on the data as it was at the time of training. In this study, we present the first analysis of AI "aging": the complex, multifaceted phenomenon of AI model quality degradation as more time passes since the last model training cycle. Using datasets from four different industries (healthcare operations, transportation, finance, and weather) and four standard machine learning models, we identify and describe the main temporal degradation patterns. We also demonstrate the principal differences between temporal model degradation and related concepts that have been explored previously, such as data concept drift and continuous learning. Finally, we indicate potential causes of temporal degradation, and suggest approaches to detecting aging and reducing its impact.

Correction: An alternative to the black box: Strategy learning.

[This corrects the article DOI: 10.1371/journal.pone.0264485.].

An alternative to the black box: Strategy learning.

In virtually any practical field or application, discovering and implementing near-optimal decision strategies is essential for achieving desired outcomes. Workflow planning is one of the most common and important problems of this kind, as sub-optimal decision-making may create bottlenecks and delays that decrease efficiency and increase costs. Recently, machine learning has been used to attack this problem, but unfortunately, most proposed solutions are "black box" algorithms with underlying logic unclear to humans. This makes them hard to implement and impossible to trust, significantly limiting their practical use. In this work, we propose an alternative approach: using machine learning to generate optimal, comprehensible strategies which can be understood and used by humans directly. Through three common decision-making problems found in scheduling, we demonstrate the implementation and feasibility of this approach, as well as its great potential to attain near-optimal results.

Characterization of Pediatric Imaging Trends and Likelihood of Exam Cancellation in the COVID-19 Pandemic.

RATIONALE AND OBJECTIVE: The COVID-19 pandemic has caused unprecedented changes in radiology practice worldwide. There is a need for a framework of pediatric radiology resource allocation for future acute resource-limited settings.The aim of this study is to quantify and analyze changes in pediatric radiology practice during the COVID-19 pandemic considering demographic and clinical characteristics. MATERIALS AND METHODS: We retrospectively searched our institution's electronic health record for pediatric imaging exams from 09/15/19 to 05/01/20, with 03/15/20 as the dividing date between baseline and pandemic periods. Age, modality, exam indication, need for anesthesia/sedation, and exam completion or cancellation were recorded. All exams were compared between baseline and pandemic periods using a chi-square test and a logistic regression multivariate analysis. RESULTS: 15,424 exams were included for analysis [13,715 baseline period (mean age 10±5 years; 7440 males); 1047 COVID-19 period (mean age 9±5 years; 565 males)]. A statistically significantly lower proportion of adolescent exams (45.5% vs 53.3%), radiography modality (62.4% vs 70.4%) and non-traumatic pain indication (39.1% vs 46.3%) was observed during the COVID-19 period. Conversely, we found a higher proportion of neonatal (5.8% vs 3.8%), infant (5.6% vs 4.1%) and early childhood patients (12.9% vs 9.8%), CT (7.4% vs 5.9%) and ultrasound modalities (18.3% vs 13.5%), oncologic (8.8% vs 6.5%) and congenital/development disorder indications (6% vs 3.9%), and studies performed under anesthesia (2.7% vs 1.3%). Regarding exam completion rates, the neonatal age group (OR 1.960 [95% CI 0.353 - 0.591]; p <0.020) and MRI modality (OR 1.502 [95% CI: 0.214 - 0.318]; p <0.049) had higher odds of completion during the COVID-19 pandemic, while fluoroscopy modality was associated with lower odds of completion (OR 0.524 [95% CI: 0.328 - 0.839]; p = 0.011). CONCLUSION: The composition and completion of pediatric radiology exams changed substantially during the COVID-19 pandemic. A sub-set of exams resilient to cancellation was identified.

Optimization of Radiology Workflow with Artificial Intelligence.

The potential of artificial intelligence (AI) in radiology goes far beyond image analysis. AI can be used to optimize all steps of the radiology workflow by supporting a variety of nondiagnostic tasks, including order entry support, patient scheduling, resource allocation, and improving the radiologist's workflow. This article discusses several principal directions of using AI algorithms to improve radiological operations and workflow management, with the intention of providing a broader understanding of the value of applying AI in the radiology department.

Dynamic memory to alleviate catastrophic forgetting in continual learning with medical imaging.

Medical imaging is a central part of clinical diagnosis and treatment guidance. Machine learning has increasingly gained relevance because it captures features of disease and treatment response that are relevant for therapeutic decision-making. In clinical practice, the continuous progress of image acquisition technology or diagnostic procedures, the diversity of scanners, and evolving imaging protocols hamper the utility of machine learning, as prediction accuracy on new data deteriorates, or models become outdated due to these domain shifts. We propose a continual learning approach to deal with such domain shifts occurring at unknown time points. We adapt models to emerging variations in a continuous data stream while counteracting catastrophic forgetting. A dynamic memory enables rehearsal on a subset of diverse training data to mitigate forgetting while enabling models to expand to new domains. The technique balances memory by detecting pseudo-domains, representing different style clusters within the data stream. Evaluation of two different tasks, cardiac segmentation in magnetic resonance imaging and lung nodule detection in computed tomography, demonstrate a consistent advantage of the method.

Mobility-Guided Estimation of COVID-19 Transmission Rates.

It is of critical importance to estimate changing disease-transmission rates and their dependence on population mobility. A common approach to this problem involves fitting daily transmission rates using a susceptible-exposed-infected-recovered-(SEIR) model (regularizing to avoid overfitting) and then computing the relationship between the estimated transmission rate and mobility. Unfortunately, there are often several very different transmission-rate trajectories that can fit the reported cases well, meaning that the choice of regularization determines the final solution (and thus the mobility-transmission rate relationship) selected by the SEIR model. Moreover, the classical approaches to regularization-penalizing the derivative of the transmission rate trajectory-do not correspond to realistic properties of pandemic spread. Consequently, models fitted using derivative-based regularization are often biased toward underestimating the current transmission rate and future deaths. In this work, we propose mobility-driven regularization of the SEIR transmission rate trajectory. This method rectifies the artificial regularization problem, produces more accurate and unbiased forecasts of future deaths, and estimates a highly interpretable relationship between mobility and the transmission rate. For this analysis, mobility data related to the coronavirus disease 2019 pandemic was collected by Safegraph (San Francisco, California) from major US cities between March and August 2020.

COVID-19: Recovery Models for Radiology Departments.

The coronavirus disease 2019 (COVID-19) pandemic has greatly affected demand for imaging services, with marked reductions in demand for elective imaging and image-guided interventional procedures. To guide radiology planning and recovery from this unprecedented impact, three recovery models were developed to predict imaging volume over the course of the COVID-19 pandemic: (1) a long-term volume model with three scenarios based on prior disease outbreaks and other historical analogues, to aid in long-term planning when the pandemic was just beginning; (2) a short-term volume model based on the supply-demand approach, leveraging increasingly available COVID-19 data points to predict examination volume on a week-to-week basis; and (3) a next-wave model to estimate the impact from future COVID-19 surges. The authors present these models as techniques that can be used at any stage in an unpredictable pandemic timeline.

Continuous Learning AI in Radiology: Implementation Principles and Early Applications.

Artificial intelligence (AI) is becoming increasingly present in radiology and health care. This expansion is driven by the principal AI strengths: automation, accuracy, and objectivity. However, as radiology AI matures to become fully integrated into the daily radiology routine, it needs to go beyond replicating static models, toward discovering new knowledge from the data and environments around it. Continuous learning AI presents the next substantial step in this direction and brings a new set of opportunities and challenges. Herein, the authors discuss the main concepts and requirements for implementing continuous AI in radiology and illustrate them with examples from emerging applications.

Imaging Volume Trends and Recovery During the COVID-19 Pandemic: A Comparative Analysis Between a Large Urban Academic Hospital and Its Affiliated Imaging Centers.

RATIONALE AND OBJECTIVES: While affiliated imaging centers play an important role in healthcare systems, little is known of how their operations are impacted by the COVID-19 pandemic. Our goal was to investigate imaging volume trends during the pandemic at our large academic hospital compared to the affiliated imaging centers. MATERIALS AND METHODS: This was a descriptive retrospective study of imaging volume from an academic hospital (main hospital campus) and its affiliated imaging centers from January 1 through May 21, 2020. Imaging volume assessment was separated into prestate of emergency (SOE) period (before SOE in Massachusetts on March 10, 2020), "post-SOE" period (time after "nonessential" services closure on March 24, 2020), and "transition" period (between pre-SOE and post-SOE). RESULTS: Imaging volume began to decrease on March 11, 2020, after hospital policy to delay nonessential studies. The average weekly imaging volume during the post-SOE period declined by 54% at the main hospital campus and 64% at the affiliated imaging centers. The rate of imaging volume recovery was slower for affiliated imaging centers (slope = 6.95 for weekdays) compared to main hospital campus (slope = 7.18 for weekdays). CT, radiography, and ultrasound exhibited the lowest volume loss, with weekly volume decrease of 41%, 49%, and 53%, respectively, at the main hospital campus, and 43%, 61%, and 60%, respectively, at affiliated imaging centers. Mammography had the greatest volume loss of 92% at both the main hospital campus and affiliated imaging centers. CONCLUSION: Affiliated imaging center volume decreased to a greater degree than the main hospital campus and showed a slower rate of recovery. Furthermore, the trend in imaging volume and recovery were temporally related to public health announcements and COVID-19 cases.

Modeling workflows: Identifying the most predictive features in healthcare operational processes.

Limited resources and increased patient flow highlight the importance of optimizing healthcare operational systems to improve patient care. Accurate prediction of exam volumes, workflow surges and, most notably, patient delay and wait times are known to have significant impact on quality of care and patient satisfaction. The main objective of this work was to investigate the choice of different operational features to achieve (1) more accurate and concise process models and (2) more effective interventions. To exclude process modelling bias, data from four different workflows was considered, including a mix of walk-in, scheduled, and hybrid facilities. A total of 84 features were computed, based on previous literature and our independent work, all derivable from a typical Hospital Information System. The features were categorized by five subgroups: congestion, customer, resource, task and time features. Two models were used in the feature selection process: linear regression and random forest. Independent of workflow and the model used for selection, it was determined that congestion feature sets lead to models most predictive for operational processes, with a smaller number of predictors.

Modeling Human Perception of Image Quality.

Humans can determine image quality instantly and intuitively, but the mechanism of human perception of image quality is unknown. The purpose of this work was to identify the most important quantitative metrics responsible for the human perception of digital image quality. Digital images from two different datasets-CT tomography (MedSet) and scenic photographs of trees (TreeSet)-were presented in random pairs to unbiased human viewers. The observers were then asked to select the best-quality image from each image pair. The resulting human-perceived image quality (HPIQ) ranks were obtained from these pairwise comparisons with two different ranking approaches. Using various digital image quality metrics reported in the literature, we built two models to predict the observed HPIQ rankings, and to identify the most important HPIQ predictors. Evaluating the quality of our HPIQ models as the fraction of falsely predicted pairwise comparisons (inverted image pairs), we obtained 70-71% of correct HPIQ predictions for the first, and 73-76%for the second approach. Taking into account that 10-14% of inverted pairs were already present in the original rankings, limitations of the models, and only a few principal HPIQ predictors used, we find this result very satisfactory. We obtained a small set of most significant quantitative image metrics associated with the human perception of image quality. This can be used for automatic image quality ranking, machine learning, and quality-improvement algorithms.

Current Applications and Future Impact of Machine Learning in Radiology.

Recent advances and future perspectives of machine learning techniques offer promising applications in medical imaging. Machine learning has the potential to improve different steps of the radiology workflow including order scheduling and triage, clinical decision support systems, detection and interpretation of findings, postprocessing and dose estimation, examination quality control, and radiology reporting. In this article, the authors review examples of current applications of machine learning and artificial intelligence techniques in diagnostic radiology. In addition, the future impact and natural extension of these techniques in radiology practice are discussed.

Machine Learning for Predicting Patient Wait Times and Appointment Delays.

Being able to accurately predict waiting times and scheduled appointment delays can increase patient satisfaction and enable staff members to more accurately assess and respond to patient flow. In this work, the authors studied the applicability of machine learning models to predict waiting times at a walk-in radiology facility (radiography) and delay times at scheduled radiology facilities (CT, MRI, and ultrasound). In the proposed models, a variety of predictors derived from data available in the radiology information system were used to predict waiting or delay times. Several machine-learning algorithms, such as neural network, random forest, support vector machine, elastic net, multivariate adaptive regression splines, k-th nearest neighbor, gradient boosting machine, bagging, classification and regression tree, and linear regression, were evaluated to find the most accurate method. The elastic net model performed best among the 10 proposed models for predicting waiting times or delay times across all four modalities. The most important predictors were also identified.

Socioeconomic and Demographic Predictors of Missed Opportunities to Provide Advanced Imaging Services.

PURPOSE: The extent to which racial and socioeconomic disparities exist in accessing clinically appropriate, advanced diagnostic imaging has not been well studied. This study assesses the relationship between demographic and socioeconomic factors and the incidence of imaging missed care opportunities (IMCOs). METHODS: We performed a retrospective review of outpatient CT and MRI appointments at a quaternary academic medical center and affiliated outpatient facilities during a 12-month period. Missed appointments not rescheduled in advance were classified as IMCOs. Appropriateness criteria scores and demographics were also obtained. Univariate and multivariate analyses were performed to determine if demographic and socioeconomic factors were predictive of IMCOs. RESULTS: Overall, 57,847 patients met inclusion criteria, representing 89,943 scheduled unique imaging appointments of which 5,840 (6.1%) were IMCOs; 0.8% of IMCO appointments had low appropriateness scores compared with 1.2% of completed appointments (P < .01). Appointments covered by commercial insurance (5.2%) had a significantly lower rate of IMCOs than other payers: Medicare = 6.3%, Medicaid = 14.5%, self-pay = 12.0% (P < .05). The following factors were independent predictors of a patient having ≥ 1 IMCO: noncommercial insurance [odds ratio (OR) = 1.7-2.6], African American (OR = 1.8), Hispanic (OR = 1.2), other race (OR = 1.1), language other than English or Spanish (OR = 1.2), male gender (OR = 1.2), age ≥ 65 (OR = 0.71), and median household income of patient home zip code <$50,000 (OR = 1.4). CONCLUSIONS: Race and socioeconomic status are independent predictors of IMCOs. In efforts to enhance patient engagement, radiologists should be aware of the impact of race and socioeconomic status on access to clinically appropriate advanced diagnostic imaging.

Predicting No-Shows in Radiology Using Regression Modeling of Data Available in the Electronic Medical Record.

PURPOSE: To test whether data elements available in the electronic medical record (EMR) can be effectively leveraged to predict failure to attend a scheduled radiology examination. MATERIALS AND METHODS: Using data from a large academic medical center, we identified all patients with a diagnostic imaging examination scheduled from January 1, 2016, to April 1, 2016, and determined whether the patient successfully attended the examination. Demographic, clinical, and health services utilization variables available in the EMR potentially relevant to examination attendance were recorded for each patient. We used descriptive statistics and logistic regression models to test whether these data elements could predict failure to attend a scheduled radiology examination. The predictive accuracy of the regression models were determined by calculating the area under the receiver operator curve. RESULTS: Among the 54,652 patient appointments with radiology examinations scheduled during the study period, 6.5% were no-shows. No-show rates were highest for the modalities of mammography and CT and lowest for PET and MRI. Logistic regression indicated that 16 of the 27 demographic, clinical, and health services utilization factors were significantly associated with failure to attend a scheduled radiology examination (P ≤ .05). Stepwise logistic regression analysis demonstrated that previous no-shows, days between scheduling and appointments, modality type, and insurance type were most strongly predictive of no-show. A model considering all 16 data elements had good ability to predict radiology no-shows (area under the receiver operator curve = 0.753). The predictive ability was similar or improved when these models were analyzed by modality. CONCLUSION: Patient and examination information readily available in the EMR can be successfully used to predict radiology no-shows. Moving forward, this information can be proactively leveraged to identify patients who might benefit from additional patient engagement through appointment reminders or other targeted interventions to avoid no-shows.