Automated reporting of cervical biopsies using artificial intelligence.

When detected at an early stage, the 5-year survival rate for people with invasive cervical cancer is 92%. Being aware of signs and symptoms of cervical cancer and early detection greatly improve the chances of successful treatment. We have developed an Artificial Intelligence (AI) algorithm, trained and evaluated on cervical biopsies for automated reporting of digital diagnostics. The aim is to increase overall efficiency of pathological diagnosis and to have the performance tuned to high sensitivity for malignant cases. Having a tool for triage/identifying cancer and high grade lesions may potentially reduce reporting time by identifying areas of interest in a slide for the pathologist and therefore improving efficiency. We trained and validated our algorithm on 1738 cervical WSIs with one WSI per patient. On the independent test set of 811 WSIs, we achieved 93.4% malignant sensitivity for classifying slides. Recognising a WSI, with our algorithm, takes approximately 1.5 minutes on the NVIDIA Tesla V100 GPU. Whole slide images of different formats (TIFF, iSyntax, and CZI) can be processed using this code, and it is easily extendable to other formats.

Magnifying Networks for Histopathological Images with Billions of Pixels.

Amongst the other benefits conferred by the shift from traditional to digital pathology is the potential to use machine learning for diagnosis, prognosis, and personalization. A major challenge in the realization of this potential emerges from the extremely large size of digitized images, which are often in excess of 100,000 × 100,000 pixels. In this paper, we tackle this challenge head-on by diverging from the existing approaches in the literature-which rely on the splitting of the original images into small patches-and introducing magnifying networks (MagNets). By using an attention mechanism, MagNets identify the regions of the gigapixel image that benefit from an analysis on a finer scale. This process is repeated, resulting in an attention-driven coarse-to-fine analysis of only a small portion of the information contained in the original whole-slide images. Importantly, this is achieved using minimal ground truth annotation, namely, using only global, slide-level labels. The results from our tests on the publicly available Camelyon16 and Camelyon17 datasets demonstrate the effectiveness of MagNets-as well as the proposed optimization framework-in the task of whole-slide image classification. Importantly, MagNets process at least five times fewer patches from each whole-slide image than any of the existing end-to-end approaches.

Correction: Guo et al. A Siamese Transformer Network for Zero-Shot Ancient Coin Classification. J. Imaging 2023, 9, 107.

Jochen Büttner was not included as an author in the original publication [...].

Disease: An ill-founded concept at odds with the principle of patient-centred medicine.

BACKGROUND: Despite the at least decades long record of philosophical recognition and interest, the intricacy of the deceptively familiar appearing concepts of 'disease', 'disorder', 'disability', and so forth, has only recently begun showing itself with clarity in the popular discourse wherein its newly emerging prominence stems from the liberties and restrictions contingent upon it. Whether a person is deemed to be afflicted by a disease or a disorder governs their ability to access health care, be it free at the point of use or provided by an insurer; it also influences the treatment of individuals by the judicial system and employers; it even affects one's own perception of self. AIMS: All existing philosophical definitions of disease struggle with coherency, causing much confusion and strife, and leading to inconsistencies in real-world practice. Hence, there is a real need for an alternative. MATERIALS AND METHODS: In the present article I analyse the variety of contemporary views of disease, showing them all to be inadequate and lacking in firm philosophical foundations, and failing to meet the desideratum of patient-driven care. RESULTS: Illuminated by the insights emanating from the said analysis, I introduce a novel approach with firm ethical foundations, which foundations are rooted in sentience, that is the subjective experience of sentient beings. DISCUSSION: I argue that the notion of disease is at best superfluous, and likely even harmful in the provision of compassionate and patient-centred care. CONCLUSION: Using a series of presently contentious cases illustrate the power of the proposed framework which is capable of providing actionable and humane solutions to problems that leave the current theories confounded.

Localization and phenotyping of tuberculosis bacteria using a combination of deep learning and SVMs.

Successful treatment of pulmonary tuberculosis (TB) depends on early diagnosis and careful monitoring of treatment response. Identification of acid-fast bacilli by fluorescence microscopy of sputum smears is a common tool for both tasks. Microscopy-based analysis of the intracellular lipid content and dimensions of individual Mycobacterium tuberculosis (Mtb) cells also describe phenotypic changes which may improve our biological understanding of antibiotic therapy for TB. However, fluorescence microscopy is a challenging, time-consuming and subjective procedure. In this work, we automate examination of fields of view (FOVs) from microscopy images to determine the lipid content and dimensions (length and width) of Mtb cells. We introduce an adapted variation of the UNet model to efficiently localising bacteria within FOVs stained by two fluorescence dyes; auramine O to identify Mtb and LipidTox Red to identify intracellular lipids. Thereafter, we propose a feature extractor in conjunction with feature descriptors to extract a representation into a support vector multi-regressor and estimate the length and width of each bacterium. Using a real-world data corpus from Tanzania, the proposed method i) outperformed previous methods for bacterial detection with a 8% improvement (Dice coefficient) and ii) estimated the cell length and width with a root mean square error of less than 0.01%. Our network can be used to examine phenotypic characteristics of Mtb cells visualised by fluorescence microscopy, improving consistency and time efficiency of this procedure compared to manual methods.

A Siamese Transformer Network for Zero-Shot Ancient Coin Classification.

Ancient numismatics, the study of ancient coins, has in recent years become an attractive domain for the application of computer vision and machine learning. Though rich in research problems, the predominant focus in this area to date has been on the task of attributing a coin from an image, that is of identifying its issue. This may be considered the cardinal problem in the field and it continues to challenge automatic methods. In the present paper, we address a number of limitations of previous work. Firstly, the existing methods approach the problem as a classification task. As such, they are unable to deal with classes with no or few exemplars (which would be most, given over 50,000 issues of Roman Imperial coins alone), and require retraining when exemplars of a new class become available. Hence, rather than seeking to learn a representation that distinguishes a particular class from all the others, herein we seek a representation that is overall best at distinguishing classes from one another, thus relinquishing the demand for exemplars of any specific class. This leads to our adoption of the paradigm of pairwise coin matching by issue, rather than the usual classification paradigm, and the specific solution we propose in the form of a Siamese neural network. Furthermore, while adopting deep learning, motivated by its successes in the field and its unchallenged superiority over classical computer vision approaches, we also seek to leverage the advantages that transformers have over the previously employed convolutional neural networks, and in particular their non-local attention mechanisms, which ought to be particularly useful in ancient coin analysis by associating semantically but not visually related distal elements of a coin's design. Evaluated on a large data corpus of 14,820 images and 7605 issues, using transfer learning and only a small training set of 542 images of 24 issues, our Double Siamese ViT model is shown to surpass the state of the art by a large margin, achieving an overall accuracy of 81%. Moreover, our further investigation of the results shows that the majority of the method's errors are unrelated to the intrinsic aspects of the algorithm itself, but are rather a consequence of unclean data, which is a problem that can be easily addressed in practice by simple pre-processing and quality checking.

Resolving the ethical quagmire of the persistent vegetative state.

BACKGROUND: A patient is diagnosed with the persistent vegetative state (PVS) when they show no evidence of the awareness of the self or the environment for an extended period of time. The chance of recovery of any mental function or the ability to interact in a meaningful way is low. Though rare, the condition, considering its nature as a state outwith the realm of the conscious, coupled with the trauma experienced by the patient's kin as well as health care staff confronted with painful decisions regarding the patient's care, has attracted a considerable amount of discussion within the bioethics community. AIMS: At present, there is a wealth of literature that discusses the relevant neurology, that elucidates the plethora of ethical challenges in understanding and dealing with the condition, and that analyses the real-world cases which have prominently featured in the mainstream media as a result of emotionally charged, divergent views concerning the provision of care to the patient. However, there is scarcely anything in the published scholarly literature that proposes concrete and practically actionable solutions to the now widely recognized moral conundrums. The present article describes a step in that direction. MATERIALS & METHODS: I start from the very foundations, laying out a sentientist approach which serves as the basis for the consequent moral decision-making, and then proceed to systematically identify and deconstruct the different cases of discord, using the aforementioned foundations as the basis for their resolution. RESULTS: A major intellectual contribution concerns the fluidity of the duty of care which I argue is demanded by the sentientist focus. DISCUSSION: The said duty is shown initially to have for its object the patient, which depending on the circumstances, can change to the patient's kin, or the health care staff themselves. CONCLUSION: In conclusion, the proposed framework represents the first comprehensive proposal regarding the decision-making processes involved in the deliberation on the provision of life sustaining treatment to a patient in a PVS.

Caveat Medicus: It's Time to Re-Think Stratification, You May Not Be Helping.

Background: The focus of the present Letter is on the large and seemingly fertile body of work captured under the umbrella of 'patient stratification'. Objectives: I identify and explain a fundamental methodological flaw underlying the manner in which the development of an increasingly large number of new stratification strategies is approached. Design: I show an inherent conflict between the assumptions made, and the very purpose of stratification and its application in practice. Methods: I analyse the methodological underpinnings of stratification as presently done and draw parallels with conceptually similarly flawed precedents which are now widely recognized. Results: The highlighted flaw is shown to undermine the overarching ultimate goal of improved patient outcomes by undue fixation on an ill-founded proxy. Conclusion: I issue a call for a re-think of the problem and the processes leading to the adoption of new stratification strategies in the clinic.

Detection of malignancy in whole slide images of endometrial cancer biopsies using artificial intelligence.

In this study we use artificial intelligence (AI) to categorise endometrial biopsy whole slide images (WSI) from digital pathology as either "malignant", "other or benign" or "insufficient". An endometrial biopsy is a key step in diagnosis of endometrial cancer, biopsies are viewed and diagnosed by pathologists. Pathology is increasingly digitised, with slides viewed as images on screens rather than through the lens of a microscope. The availability of these images is driving automation via the application of AI. A model that classifies slides in the manner proposed would allow prioritisation of these slides for pathologist review and hence reduce time to diagnosis for patients with cancer. Previous studies using AI on endometrial biopsies have examined slightly different tasks, for example using images alongside genomic data to differentiate between cancer subtypes. We took 2909 slides with "malignant" and "other or benign" areas annotated by pathologists. A fully supervised convolutional neural network (CNN) model was trained to calculate the probability of a patch from the slide being "malignant" or "other or benign". Heatmaps of all the patches on each slide were then produced to show malignant areas. These heatmaps were used to train a slide classification model to give the final slide categorisation as either "malignant", "other or benign" or "insufficient". The final model was able to accurately classify 90% of all slides correctly and 97% of slides in the malignant class; this accuracy is good enough to allow prioritisation of pathologists' workload.

Lymphocyte Classification from Hoechst Stained Slides with Deep Learning.

Multiplex immunofluorescence and immunohistochemistry benefit patients by allowing cancer pathologists to identify proteins expressed on the surface of cells. This enables cell classification, better understanding of the tumour microenvironment, and more accurate diagnoses, prognoses, and tailored immunotherapy based on the immune status of individual patients. However, these techniques are expensive. They are time consuming processes which require complex staining and imaging techniques by expert technicians. Hoechst staining is far cheaper and easier to perform, but is not typically used as it binds to DNA rather than to the proteins targeted by immunofluorescence techniques. In this work we show that through the use of deep learning it is possible to identify an immune cell subtype without immunofluorescence. We train a deep convolutional neural network to identify cells expressing the T lymphocyte marker CD3 from Hoechst 33342 stained tissue only. CD3 expressing cells are often used in key prognostic metrics such as assessment of immune cell infiltration, and by identifying them without the need for costly immunofluorescence, we present a promising new approach to cheaper prediction and improvement of patient outcomes. We also show that by using deep learning interpretability techniques, we can gain insight into the previously unknown morphological features which make this possible.

Use of High-Plex Data Reveals Novel Insights into the Tumour Microenvironment of Clear Cell Renal Cell Carcinoma.

Although immune checkpoint inhibitors (ICIs) have significantly improved the oncological outcomes, about one-third of patients affected by clear cell renal cell carcinoma (ccRCC) still experience recurrence. Current prognostic algorithms, such as the Leibovich score (LS), rely on morphological features manually assessed by pathologists and are therefore subject to bias. Moreover, these tools do not consider the heterogeneous molecular milieu present in the Tumour Microenvironment (TME), which may have prognostic value. We systematically developed a semi-automated method to investigate 62 markers and their combinations in 150 primary ccRCCs using Multiplex Immunofluorescence (mIF), NanoString GeoMx® Digital Spatial Profiling (DSP) and Artificial Intelligence (AI)-assisted image analysis in order to find novel prognostic signatures and investigate their spatial relationship. We found that coexpression of cancer stem cell (CSC) and epithelial-to-mesenchymal transition (EMT) markers such as OCT4 and ZEB1 are indicative of poor outcome. OCT4 and the immune markers CD8, CD34, and CD163 significantly stratified patients at intermediate LS. Furthermore, augmenting the LS with OCT4 and CD34 improved patient stratification by outcome. Our results support the hypothesis that combining molecular markers has prognostic value and can be integrated with morphological features to improve risk stratification and personalised therapy. To conclude, GeoMx® DSP and AI image analysis are complementary tools providing high multiplexing capability required to investigate the TME of ccRCC, while reducing observer bias.

A systemic challenge in dietetics: Methodological inadequacies, erroneous claims, and misleadinginterpretations, and transparency of post-publication scrutiny.

Background: Obesity is sweeping across the developed world. Yet, the public remains largely confused when it comes to the nature of dietary habits which would serve to counteract this trend. Aim: I highlight the responsibility that the scientific community bears when it comes to the confusion, and explain the kind of actions that are needed if the public trust in science is to be maintained. Methods: Starting from an example of a recently published and prominently featured article in a leading journal, I analyse various common methodological aspects of dietetics research and the consequent claims, contextualizing this within the broader environment which includes the scientific publishing process and the mainstream media. Results: Methodological inadequacies, erroneous claims, and misleading interpretations of findings are often found in dietetics research, highlighting the deficiencies of the system which fails to uphold the fundamental principles of scientific inquiry. Conclusion: It is imperative that individual scientists speak out and challenge poor science, unsatisfactory publishing processes, and bombastic and misleading communication of research.

Reproducibility of deep learning in digital pathology whole slide image analysis.

For a method to be widely adopted in medical research or clinical practice, it needs to be reproducible so that clinicians and regulators can have confidence in its use. Machine learning and deep learning have a particular set of challenges around reproducibility. Small differences in the settings or the data used for training a model can lead to large differences in the outcomes of experiments. In this work, three top-performing algorithms from the Camelyon grand challenges are reproduced using only information presented in the associated papers and the results are then compared to those reported. Seemingly minor details were found to be critical to performance and yet their importance is difficult to appreciate until the actual reproduction is attempted. We observed that authors generally describe the key technical aspects of their models well but fail to maintain the same reporting standards when it comes to data preprocessing which is essential to reproducibility. As an important contribution of the present study and its findings, we introduce a reproducibility checklist that tabulates information that needs to be reported in histopathology ML-based work in order to make it reproducible.

Generative Image Inpainting for Retinal Images using Generative Adversarial Networks.

The diagnosis and treatment of eye diseases is heavily reliant on the availability of retinal imagining equipment. To increase accessibility, lower-cost ophthalmoscopes, such as the Arclight, have been developed. However, a common drawback of these devices is a limited field of view. The narrow-field-of-view images of the eye can be concatenated to replicate a wide field of view. However, it is likely that not all angles of the eye are captured, which creates gaps. This limits the usefulness of the images in teaching, wherefore, artist's impressions of retinal pathologies are used. Recent research in the field of computer vision explores the automatic completion of holes in images by leveraging the structural understanding of similar images gained by neural networks. Specifically, generative adversarial networks are explored, which consist of two neural networks playing a game against each other to facilitate learning. We demonstrate a proof of concept for the generative image inpainting of retinal images using generative adversarial networks. Our work is motivated by the aim of devising more realistic images for medical teaching purposes. We propose the use of a Wasserstein generative adversarial network with a semantic image inpainting algorithm, as it produces the most realistic images.Clinical relevance- The research shows the use of generative adversarial networks in generating realistic training images.

How Good is the Science That Informs Government Policy? A Lesson From the U.K.'s Response to 2020 CoV-2 Outbreak.

In an era when public faith in politicians is dwindling, yet trust in scientists remains relatively high, governments are increasingly emphasizing the role of science based policy-making in response to challenges such as climate change and global pandemics. In this paper we question the quality of some scientific advice given to governments and the robustness and transparency of the entire framework which envelopes such advice, all of which raise serious ethical concerns. In particular we focus on the so-called Imperial Model which heavily influenced the government of the United Kingdom in devising its response to the COVID-19 crisis. We focus on and highlight several fundamental methodological flaws of the model, raise concerns as to the robustness of the system which permitted these to remain unchallenged, and discuss the relevant ethical consequences.

A Comparison of Methods for Studying the Tumor Microenvironment's Spatial Heterogeneity in Digital Pathology Specimens.

BACKGROUND: The tumor microenvironment is highly heterogeneous, and it is understood to affect tumor progression and patient outcome. A number of studies have reported the prognostic significance of tumor-infiltrating lymphocytes and tumor budding in colorectal cancer (CRC). However, the significance of the intratumoral heterogeneity present in the spatial distribution of these features within the tumor immune microenvironment (TIME) has not been previously reported. Evaluating this intratumoral heterogeneity may aid the understanding of the TIME's effect on patient prognosis as well as identify novel aggressive phenotypes which can be further investigated as potential targets for new treatment. METHODS: In this study, we propose and apply two spatial statistical methodologies for the evaluation of the intratumor heterogeneity present in the distribution of CD3 + and CD8 + lymphocytes and tumor buds (TB) in 232 Stage II CRC cases. Getis-Ord hotspot analysis was applied to quantify the cold and hotspots, defined as regions with a significantly low or high number of each feature of interest, respectively. A novel spatial heatmap methodology for the quantification of the cold and hotspots of each feature of interest, which took into account both the interpatient heterogeneity and the intratumor heterogeneity, was further developed. RESULTS: Resultant data from each analysis, characterizing the spatial intratumor heterogeneity of lymphocytes and TBs were used for the development of two new highly prognostic risk models. CONCLUSIONS: Our results highlight the value of applying spatial statistics for the assessment of the intratumor heterogeneity. Both Getis-Ord hotspot and our proposed spatial heatmap analysis are broadly applicable across other tissue types as well as other features of interest. AVAILABILITY: The code underpinning this publication can be accessed at https://doi.org/10.17630/c2306fe9-66e2-4442-ad89-f986220053e2.

Assessment of Immunological Features in Muscle-Invasive Bladder Cancer Prognosis Using Ensemble Learning.

The clinical staging and prognosis of muscle-invasive bladder cancer (MIBC) routinely includes the assessment of patient tissue samples by a pathologist. Recent studies corroborate the importance of image analysis in identifying and quantifying immunological markers from tissue samples that can provide further insight into patient prognosis. In this paper, we apply multiplex immunofluorescence to MIBC tissue sections to capture whole-slide images and quantify potential prognostic markers related to lymphocytes, macrophages, tumour buds, and PD-L1. We propose a machine-learning-based approach for the prediction of 5 year prognosis with different combinations of image, clinical, and spatial features. An ensemble model comprising several functionally different models successfully stratifies MIBC patients into two risk groups with high statistical significance (p value < 1×10-5). Critical to improving MIBC survival rates, our method correctly classifies 71.4% of the patients who succumb to MIBC, which is significantly more than the 28.6% of the current clinical gold standard, the TNM staging system.

Quantification of Advanced Dementia Patients' Engagement in Therapeutic Sessions: An Automatic Video Based Approach using Computer Vision and Machine Learning.

Most individuals with advanced dementia lose the ability to communicate with the outside world through speech. This limits their ability to participate in social activities crucial to their well-being and quality of life. However, there is mounting evidence that individuals with advanced dementia can still communicate non-verbally and benefit greatly from these interactions. A major problem in facilitating the advancement of this research is of a practical and methodical nature: assessing the success of treatment is currently done by humans, prone to subjective bias and inconsistency, and it involves laborious and time consuming effort. The present work is the first attempt at exploring if automatic (artificial intelligence based) quantification of the degree of patient engagement in Adaptive Interaction sessions, a highly promising intervention developed to improve the quality of life of nonverbal individuals with advanced dementia. Hence we describe a framework which uses computer vision and machine learning as a potential first step towards answering this question. Using a real-world data set of videos of therapeutic sessions, not acquired specifically for the purposes of the present work, we demonstrate highly promising results.

Corrigendum: Deep Learning for Whole Slide Image Analysis: An Overview.

[This corrects the article on p. 264 in vol. 6, PMID: 31824952.].

Big Data Driven Detection of Trees in Suburban Scenes Using Visual Spectrum Eye Level Photography.

The aim of the work described in this paper is to detect trees in eye level view images. Unlike previous work that universally considers highly constrained environments, such as natural parks and wooded areas, or simple scenes with little clutter and clear tree separation, our focus is on much more challenging suburban scenes, which are rich in clutter and highly variable in type and appearance (houses, falls, shrubs, cars, bicycles, pedestrians, hydrants, lamp posts, etc.). Thus, we motivate and introduce three different approaches: (i) a conventional computer vision based approach, employing manually engineered steps and making use of explicit human knowledge of the application domain, (ii) a more machine learning oriented approach, which learns from densely extracted local features in the form of scale invariant features (SIFT), and (iii) a machine learning based approach, which employs both colour and appearance models as a means of making the most of available discriminative information. We also make a significant contribution in regards to the collection of training and evaluation data. In contrast to the existing work, which relies on manual data collection (thus risking unintended bias) or corpora constrained in variability and limited in size (thus not allowing for reliable generalisation inferences to be made), we show how large amounts of representative data can be collected automatically using freely available tools, such as Google's Street View, and equally automatically processed to produce a large corpus of minimally biased imagery. Using a large data set collected in the manner and comprising tens of thousands of images, we confirm our theoretical arguments that motivated our machine learning based and colour-aware histograms of oriented gradients based method, which achieved a recall of 95% and precision of 97%.