Multi-centre benchmarking of deep learning models for COVID-19 detection in chest x-rays.

Introduction: This study is a retrospective evaluation of the performance of deep learning models that were developed for the detection of COVID-19 from chest x-rays, undertaken with the goal of assessing the suitability of such systems as clinical decision support tools. Methods: Models were trained on the National COVID-19 Chest Imaging Database (NCCID), a UK-wide multi-centre dataset from 26 different NHS hospitals and evaluated on independent multi-national clinical datasets. The evaluation considers clinical and technical contributors to model error and potential model bias. Model predictions are examined for spurious feature correlations using techniques for explainable prediction. Results: Models performed adequately on NHS populations, with performance comparable to radiologists, but generalised poorly to international populations. Models performed better in males than females, and performance varied across age groups. Alarmingly, models routinely failed when applied to complex clinical cases with confounding pathologies and when applied to radiologist defined "mild" cases. Discussion: This comprehensive benchmarking study examines the pitfalls in current practices that have led to impractical model development. Key findings highlight the need for clinician involvement at all stages of model development, from data curation and label definition, to model evaluation, to ensure that all clinical factors and disease features are appropriately considered during model design. This is imperative to ensure automated approaches developed for disease detection are fit-for-purpose in a clinical setting.

Artificial intelligence in ovarian cancer histopathology: a systematic review.

This study evaluates the quality of published research using artificial intelligence (AI) for ovarian cancer diagnosis or prognosis using histopathology data. A systematic search of PubMed, Scopus, Web of Science, Cochrane CENTRAL, and WHO-ICTRP was conducted up to May 19, 2023. Inclusion criteria required that AI was used for prognostic or diagnostic inferences in human ovarian cancer histopathology images. Risk of bias was assessed using PROBAST. Information about each model was tabulated and summary statistics were reported. The study was registered on PROSPERO (CRD42022334730) and PRISMA 2020 reporting guidelines were followed. Searches identified 1573 records, of which 45 were eligible for inclusion. These studies contained 80 models of interest, including 37 diagnostic models, 22 prognostic models, and 21 other diagnostically relevant models. Common tasks included treatment response prediction (11/80), malignancy status classification (10/80), stain quantification (9/80), and histological subtyping (7/80). Models were developed using 1-1375 histopathology slides from 1-776 ovarian cancer patients. A high or unclear risk of bias was found in all studies, most frequently due to limited analysis and incomplete reporting regarding participant recruitment. Limited research has been conducted on the application of AI to histopathology images for diagnostic or prognostic purposes in ovarian cancer, and none of the models have been demonstrated to be ready for real-world implementation. Key aspects to accelerate clinical translation include transparent and comprehensive reporting of data provenance and modelling approaches, and improved quantitative evaluation using cross-validation and external validations. This work was funded by the Engineering and Physical Sciences Research Council.

The Pitfalls of Using Open Data to Develop Deep Learning Solutions for COVID-19 Detection in Chest X-Rays.

Since the emergence of COVID-19, deep learning models have been developed to identify COVID-19 from chest X-rays. With little to no direct access to hospital data, the AI community relies heavily on public data comprising numerous data sources. Model performance results have been exceptional when training and testing on open-source data, surpassing the reported capabilities of AI in pneumonia-detection prior to the COVID-19 outbreak. In this study impactful models are trained on a widely used open-source data and tested on an external test set and a hospital dataset, for the task of classifying chest X-rays into one of three classes: COVID-19, non-COVID pneumonia and no-pneumonia. Classification performance of the models investigated is evaluated through ROC curves, confusion matrices and standard classification metrics. Explainability modules are implemented to explore the image features most important to classification. Data analysis and model evalutions show that the popular open-source dataset COVIDx is not representative of the real clinical problem and that results from testing on this are inflated. Dependence on open-source data can leave models vulnerable to bias and confounding variables, requiring careful analysis to develop clinically useful/viable AI tools for COVID-19 detection in chest X-rays.

AuguR: A Scalable Open-Source Interactive Web Application for Routinely Collected Data.

Most data collected by hospitals as a consequence of the delivery of routine care is not utilised for analytics or organisational intelligence. This project aims to develop tools to enhance the utilisation of routinely collected cancer data within hospitals across England. This was achieved by developing a web application using open source tools to provide health care professionals and hospital managers with easy to use, interactive analytics for cancer data. The application uses data items hospitals in England are mandated to collect as part of the Cancer Outcomes and Services Dataset (COSD), to provide clinical insight into survival outcomes, population distributions, service demands, waiting times, geographical case distributions and treatment information in real-time or near real-time. Development was guided by end user needs through the use of panels of clinical and non-clinical end users.

Collection of cancer Patient Reported Outcome Measures (PROMS) to link with primary and secondary electronic care records to understand and improve long term cancer outcomes: A protocol paper.

INTRODUCTION: More people are living with and beyond a cancer diagnosis. There is limited understanding of the long-term effects of cancer and cancer treatment on quality of life and personal and household finances when compared to people without cancer. In a separate protocol we have proposed to link de-identified data from electronic primary care and hospital records for a large population of cancer survivors and matched controls. In this current protocol, we propose the linkage of Patient Reported Outcomes Measures data to the above data for a subset of this population. The aim of this study is to investigate the full impact of living with and beyond a cancer diagnosis compared to age and gender matched controls. A secondary aim is to test the feasibility of the collection of Patient Reported Outcomes Measures (PROMS) data and the linkage procedures of the PROMs data to electronic health records data. MATERIALS AND METHODS: This is a cross-sectional study, aiming to recruit participants treated at the Leeds Teaching Hospitals National Health Service Trust. Eligible patients will be cancer survivors at around 5 years post-diagnosis (breast, colorectal and ovarian cancer) and non-cancer patient matched controls attending dermatology out-patient clinics. They will be identified by running a query on the Leeds Teaching Hospitals Trust patient records system. Approximately 6000 patients (2000 cases and 4000 controls) will be invited to participate via post. Participants will be invited to complete PROMs assessing factors such as quality of life and finances, which can be completed on paper or online (surveys includes established instruments, and bespoke instruments (demographics, financial costs). This PROMs data will then be linked to routinely collected de-identified data from patient's electronic primary care and hospital records. DISCUSSION: This innovative work aims to create a truly 'comprehensive patient record' to provide a broad picture of what happens to cancer patients across their cancer pathway, and the long-term impact of cancer treatment. Comparisons can be made between the cases and controls, to identify the aspects of life that has had the greatest impact following a cancer diagnosis. The feasibility of linking PROMs data to electronic health records can also be assessed. This work can inform future support offered to people living with and beyond a cancer diagnosis, clinical practice, and future research methodologies.

A data flow process for confidential data and its application in a health research project.

BACKGROUND: The use of linked healthcare data in research has the potential to make major contributions to knowledge generation and service improvement. However, using healthcare data for secondary purposes raises legal and ethical concerns relating to confidentiality, privacy and data protection rights. Using a linkage and anonymisation approach that processes data lawfully and in line with ethical best practice to create an anonymous (non-personal) dataset can address these concerns, yet there is no set approach for defining all of the steps involved in such data flow end-to-end. We aimed to define such an approach with clear steps for dataset creation, and to describe its utilisation in a case study linking healthcare data. METHODS: We developed a data flow protocol that generates pseudonymous datasets that can be reversibly linked, or irreversibly linked to form an anonymous research dataset. It was designed and implemented by the Comprehensive Patient Records (CPR) study in Leeds, UK. RESULTS: We defined a clear approach that received ethico-legal approval for use in creating an anonymous research dataset. Our approach used individual-level linkage through a mechanism that is not computer-intensive and was rendered irreversible to both data providers and processors. We successfully applied it in the CPR study to hospital and general practice and community electronic health record data from two providers, along with patient reported outcomes, for 365,193 patients. The resultant anonymous research dataset is available via DATA-CAN, the Health Data Research Hub for Cancer in the UK. CONCLUSIONS: Through ethical, legal and academic review, we believe that we contribute a defined approach that represents a framework that exceeds current minimum standards for effective pseudonymisation and anonymisation. This paper describes our methods and provides supporting information to facilitate the use of this approach in research.

Observational study investigating Tolerance Of Anticancer Systemic Therapy In the Elderly (TOASTIE): a protocol.

INTRODUCTION: The number of older adults diagnosed with cancer is increasing. Older adults are more likely to have pre-existing frailty, which is associated with greater chemotherapy-related toxicity. Early identification of those at risk of toxicity is important to reduce patient morbidity and mortality. Current chemotherapy toxicity prediction tools including the Cancer and Ageing Research Group (CARG) tool exist but are not in routine clinical use and have not been prospectively validated in a UK population. This study is the first prospective study to investigate the CARG tool in a UK population with cancer. METHODS AND ANALYSIS: Tolerance Of Anticancer Systemic Therapy In the Elderly is a prospective observational study of patients, aged ≥65 years, commencing first-line (any indication) chemotherapy for a solid-organ malignancy. Patients receiving other systemic anticancer agents or radiotherapy will be excluded. The primary objective will be to validate the ability of the CARG score to predict grade 3+ toxicity in this population. Secondary objectives include describing the feasibility of screening for frailty, as well as the prevalance of frailty in this population and assessing patient and clinician perception of chemotherapy toxicity risk. 500 patients will be recruited over a two year period. Baseline assessments will be recorded. At the end of the 6-month follow-up period, toxicity data will be retrospectively collected. A descriptive analysis of the recruited population will be performed. The validity of the CARG model will be analysed using receiver-operating characteristic curves and calculation of the area under the curve (c-statistic). ETHICS AND DISSEMINATION: The study has received ethical approval from the East of Scotland Research Ethics Service 20/ES/0114. Results will be reported in peer-reviewed scientific journals and disseminated to patient organisations and media.

Process Mining to Explore Variations in Endometrial Cancer Pathways from GP Referral to First Treatment.

The main challenge in the pathway analysis of cancer treatments is the complexity of the process. Process mining is one of the approaches that can be used to visualize and analyze these complex pathways. In this study, our purpose was to use process mining to explore variations in the treatment pathways of endometrial cancer. We extracted patient data from a hospital information system, created the process model, and analyzed the variations of the 62-day pathway from a General Practitioner referral to the first treatment in the hospital. We also analyzed the variations based on three different criteria: the type of the first treatment, the age at diagnosis, and the year of diagnosis. This approach should be of interest to others dealing with complex medical and healthcare processes.

Using a Multi-Level Process Comparison for Process Change Analysis in Cancer Pathways.

The area of process change over time is a particular concern in healthcare, where patterns of care emerge and evolve in response to individual patient needs. We propose a structured approach to analyse process change over time that is suitable for the complex domain of healthcare. Our approach applies a qualitative process comparison at three levels of abstraction: a holistic perspective (process model), a middle-level perspective (trace), and a fine-grained detail (activity). Our aim was to detect change points, localise and characterise the change, and unravel/understand the process evolution. We illustrate the approach using a case study of cancer pathways in Leeds where we found evidence of change points identified at multiple levels. In this paper, we extend our study by analysing the miners used in process discovery and providing a deeper analysis of the activity of investigation in trace and activity levels. In the experiment, we show that this qualitative approach provides a useful understanding of process change over time. Examining change at three levels provides confirmatory evidence of process change where perspectives agree, while contradictory evidence can lead to focused discussions with domain experts. This approach should be of interest to others dealing with processes that undergo complex change over time.

Current Treatment and Outcomes Benchmark for Locally Advanced or Metastatic Urothelial Cancer From a Large UK-Based Single Centre.

Objectives: To characterize treatment patterns and survival outcomes for patients with locally advanced or metastatic malignancy of the urothelial tract during a period immediately preceding the widespread use of immune checkpoint inhibitors in the UK. Patients and Methods: We retrospectively examined the electronic case notes of patients attending the Leeds Cancer Center, UK with locally advanced or metastatic urothelial carcinoma, receiving chemotherapy between January 2003 and March 2017. Patient characteristics, treatment patterns, and outcomes were collected. Summary and descriptive statistics were calculated for categorical and continuous variables as appropriate. The Kaplan-Meier method was used to estimate median survival and Cox regression proportional hazards model was used to explore relationships between clinical variables and outcome. Results: Two hundred and sixteen patients made up the study cohort, with a median age of 66 years (range: 35-83) and 72.7% being male. First-line treatment consisted of either a cisplatin- (44%) or carboplatin-based regimen (48%) in the majority of patients. Twenty seven percent of patients received a second-line of treatment (most commonly single-agent paclitaxel) following a first-line platinum containing regimen. Grade 4 neutropenia was observed in 19 and 27% of those treated with a first-line cisplatin- and carboplatin-based regimen, respectively. The median overall survival (mOS) of the study cohort was estimated to be 16.2 months (IQR: 10.6-28.3 months). Receipt by patients of cisplatin-based chemotherapy was associated with a longer mOS and this association persisted when survival analysis was adjusted for age, sex, performance status and presence of distant metastases. Conclusions: This study provides a useful benchmark for outcomes achieved in a real-world setting for patients with locally advanced or metastatic UC treated with chemotherapy in the immediate pre-immunotherapy era.