Fairness and generalizability of OCT normative databases: a comparative analysis.

PURPOSE: In supervised Machine Learning algorithms, labels and reports are important in model development. To provide a normality assessment, the OCT has an in-built normative database that provides a color base scale from the measurement database comparison. This article aims to evaluate and compare normative databases of different OCT machines, analyzing patient demographic, contrast inclusion and exclusion criteria, diversity index, and statistical approach to assess their fairness and generalizability. METHODS: Data were retrieved from Cirrus, Avanti, Spectralis, and Triton's FDA-approval and equipment manual. The following variables were compared: number of eyes and patients, inclusion and exclusion criteria, statistical approach, sex, race and ethnicity, age, participant country, and diversity index. RESULTS: Avanti OCT has the largest normative database (640 eyes). In every database, the inclusion and exclusion criteria were similar, including adult patients and excluding pathological eyes. Spectralis has the largest White (79.7%) proportionately representation, Cirrus has the largest Asian (24%), and Triton has the largest Black (22%) patient representation. In all databases, the statistical analysis applied was Regression models. The sex diversity index is similar in all datasets, and comparable to the ten most populous contries. Avanti dataset has the highest diversity index in terms of race, followed by Cirrus, Triton, and Spectralis. CONCLUSION: In all analyzed databases, the data framework is static, with limited upgrade options and lacking normative databases for new modules. As a result, caution in OCT normality interpretation is warranted. To address these limitations, there is a need for more diverse, representative, and open-access datasets that take into account patient demographics, especially considering the development of supervised Machine Learning algorithms in healthcare.

Fairness and generalisability in deep learning of retinopathy of prematurity screening algorithms: a literature review.

BACKGROUND: Retinopathy of prematurity (ROP) is a vasoproliferative disease responsible for more than 30 000 blind children worldwide. Its diagnosis and treatment are challenging due to the lack of specialists, divergent diagnostic concordance and variation in classification standards. While artificial intelligence (AI) can address the shortage of professionals and provide more cost-effective management, its development needs fairness, generalisability and bias controls prior to deployment to avoid producing harmful unpredictable results. This review aims to compare AI and ROP study's characteristics, fairness and generalisability efforts. METHODS: Our review yielded 220 articles, of which 18 were included after full-text assessment. The articles were classified into ROP severity grading, plus detection, detecting treatment requiring, ROP prediction and detection of retinal zones. RESULTS: All the article's authors and included patients are from middle-income and high-income countries, with no low-income countries, South America, Australia and Africa Continents representation.Code is available in two articles and in one on request, while data are not available in any article. 88.9% of the studies use the same retinal camera. In two articles, patients' sex was described, but none applied a bias control in their models. CONCLUSION: The reviewed articles included 180 228 images and reported good metrics, but fairness, generalisability and bias control remained limited. Reproducibility is also a critical limitation, with few articles sharing codes and none sharing data. Fair and generalisable ROP and AI studies are needed that include diverse datasets, data and code sharing, collaborative research, and bias control to avoid unpredictable and harmful deployments.

Sociodemographic disparities in ophthalmological clinical trials.

INTRODUCTION: In ophthalmology, clinical trials (CTs) guide the treatment of diseases such as diabetic retinopathy, myopia, age-related macular degeneration, glaucoma and keratoconus with distinct presentations, pathological characteristics and responses to treatment in minority populations.Reporting gender and race and ethnicity in healthcare studies is currently recommended by National Institutes of Health (NIH) and Food and Drug Administration (FDA) guidelines to ensure representativeness and generalisability; however, CT results that include this information have been limited in the past 30 years.The objective of this review is to analyse the sociodemographic disparities in ophthalmological phases III and IV CT based on publicly available data. METHODS: This study included phases III and IV complete ophthalmological CT available from clinicaltrials.org, and describes the country distribution, race and ethnicity description and gender, and funding characteristics. RESULTS: After a screening process, we included 654 CTs, with findings that corroborate the previous CT reviews' findings that most ophthalmological participants are white and from high-income countries. A description of race and ethnicity is reported in 37.1% of studies but less frequently included within the most studied ophthalmological specialty area (cornea, retina, glaucoma and cataracts). The incidence of race and ethnicity reporting has improved during the past 7 years. DISCUSSION: Although NIH and FDA promote guidelines to improve generalisability in healthcare studies, the inclusion of race and ethnicity in publications and diverse participants in ophthalmological CT is still limited. Actions from the research community and related stakeholders are necessary to increase representativeness and guarantee generalisability in ophthalmological research results to optimise care and reduce related healthcare disparities.

Sources of bias in artificial intelligence that perpetuate healthcare disparities-A global review.

BACKGROUND: While artificial intelligence (AI) offers possibilities of advanced clinical prediction and decision-making in healthcare, models trained on relatively homogeneous datasets, and populations poorly-representative of underlying diversity, limits generalisability and risks biased AI-based decisions. Here, we describe the landscape of AI in clinical medicine to delineate population and data-source disparities. METHODS: We performed a scoping review of clinical papers published in PubMed in 2019 using AI techniques. We assessed differences in dataset country source, clinical specialty, and author nationality, sex, and expertise. A manually tagged subsample of PubMed articles was used to train a model, leveraging transfer-learning techniques (building upon an existing BioBERT model) to predict eligibility for inclusion (original, human, clinical AI literature). Of all eligible articles, database country source and clinical specialty were manually labelled. A BioBERT-based model predicted first/last author expertise. Author nationality was determined using corresponding affiliated institution information using Entrez Direct. And first/last author sex was evaluated using the Gendarize.io API. RESULTS: Our search yielded 30,576 articles, of which 7,314 (23.9%) were eligible for further analysis. Most databases came from the US (40.8%) and China (13.7%). Radiology was the most represented clinical specialty (40.4%), followed by pathology (9.1%). Authors were primarily from either China (24.0%) or the US (18.4%). First and last authors were predominately data experts (i.e., statisticians) (59.6% and 53.9% respectively) rather than clinicians. And the majority of first/last authors were male (74.1%). INTERPRETATION: U.S. and Chinese datasets and authors were disproportionately overrepresented in clinical AI, and almost all of the top 10 databases and author nationalities were from high income countries (HICs). AI techniques were most commonly employed for image-rich specialties, and authors were predominantly male, with non-clinical backgrounds. Development of technological infrastructure in data-poor regions, and diligence in external validation and model re-calibration prior to clinical implementation in the short-term, are crucial in ensuring clinical AI is meaningful for broader populations, and to avoid perpetuating global health inequity.

Generalisability through local validation: overcoming barriers due to data disparity in healthcare.

Cho et al. report deep learning model accuracy for tilted myopic disc detection in a South Korean population. Here we explore the importance of generalisability of machine learning (ML) in healthcare, and we emphasise that recurrent underrepresentation of data-poor regions may inadvertently perpetuate global health inequity.Creating meaningful ML systems is contingent on understanding how, when, and why different ML models work in different settings. While we echo the need for the diversification of ML datasets, such a worthy effort would take time and does not obviate uses of presently available datasets if conclusions are validated and re-calibrated for different groups prior to implementation.The importance of external ML model validation on diverse populations should be highlighted where possible - especially for models built with single-centre data.

The weekend effect for stroke patients admitted to intensive care: A retrospective cohort analysis.

OBJECTIVES: To examine the effect of weekend admission on short and long-term morbidity and mortality, for patients admitted to intensive care after suffering a cerebrovascular accident (stroke). DESIGN, SETTING, AND PARTICIPANTS: A hospital-wide, retrospective cohort study of 3,729 adult stroke patients admitted to the Beth Israel Deaconess Medical Centre (BIDMC) intensive care unit (ICU) between 2001 and 2012, using the Medical Information Mart for Intensive Care III (MIMIC-III) database. PRIMARY OUTCOME MEASURES: Primary outcome measures were ICU length-of-stay and mortality, hospital length-of-stay and mortality, proportions of patients discharged home after admission, and 6-month mortality. RESULTS: Overall, 23% of BIDMC ICU stroke admissions occurred over the weekend. Those admitted over the weekend were likelier to have suffered haemorrhagic stroke than those admitted during the week (60.6% vs 47.9%). Those admitted on the weekend were younger, and likelier to be male and unmarried, with similar ethnic representation. The OASIS severity of illness (32.5 vs. 32) and lowest day-one GCS (12.6 vs. 12.9) were similar between groups. Unadjusted ICU-mortality was significantly higher for patients admitted over the weekend (OR 1.32, CI 1.08-1.61), but when adjusted for type of stroke, became non-significant (OR 1.17, CI 0.95-1.44). In-hospital mortality was significantly higher for patients admitted to ICU over the weekend in both unadjusted (OR 1.45, CI 1.22-1.73) and adjusted (OR 1.31, CI 1.09-1.58) analyses. There was no significant difference in ICU or hospital length of stay. While patients admitted on the weekend appeared less likely to be discharged back to home and more at risk of 6-month mortality compared to weekday admissions, results were non-significant. CONCLUSIONS: The effect of weekend ICU-admission for stroke patients appears to be significant for in-hospital mortality. There were no significant differences in adjusted ICU-mortality, ICU or hospital length-of-stay, or longer-term morbidity and mortality measures.