Super resolution-based methodology for self-supervised segmentation of microscopy images.

Data-driven Artificial Intelligence (AI)/Machine learning (ML) image analysis approaches have gained a lot of momentum in analyzing microscopy images in bioengineering, biotechnology, and medicine. The success of these approaches crucially relies on the availability of high-quality microscopy images, which is often a challenge due to the diverse experimental conditions and modes under which these images are obtained. In this study, we propose the use of recent ML-based image super-resolution (SR) techniques for improving the image quality of microscopy images, incorporating them into multiple ML-based image analysis tasks, and describing a comprehensive study, investigating the impact of SR techniques on the segmentation of microscopy images. The impacts of four Generative Adversarial Network (GAN)- and transformer-based SR techniques on microscopy image quality are measured using three well-established quality metrics. These SR techniques are incorporated into multiple deep network pipelines using supervised, contrastive, and non-contrastive self-supervised methods to semantically segment microscopy images from multiple datasets. Our results show that the image quality of microscopy images has a direct influence on the ML model performance and that both supervised and self-supervised network pipelines using SR images perform better by 2%-6% in comparison to baselines, not using SR. Based on our experiments, we also establish that the image quality improvement threshold range [20-64] for the complemented Perception-based Image Quality Evaluator(PIQE) metric can be used as a pre-condition by domain experts to incorporate SR techniques to significantly improve segmentation performance. A plug-and-play software platform developed to integrate SR techniques with various deep networks using supervised and self-supervised learning methods is also presented.

Convergence research and training in computational bioengineering: a case study on AI/ML driven biofilm-material interaction discovery.

Initially, research disciplines operated independently, but the emergence of trans-disciplinary sciences led to convergence research, impacting graduate programs and research laboratories, especially in bioengineering and material engineering as presented here. Current graduate curriculum fails to efficiently prepare students for multidisciplinary and convergence research, thus creating a gap between the students and research laboratory expectations. We present a convergence training framework for graduate students, incorporating problem-based learning under the guidance of senior scientists and collaboration with postdoctoral researchers. This case study serves as a template for transdisciplinary convergent training projects - bridging the expertise gap and fostering successful convergence learning experiences in computational biointerface (material-biology interface). The 18-month Advanced Data Science Workshop, initiated in 2019, involves project-based learning, online training modules, and data collection. A pilot solution utilized Jupyter notebook on Google collaborator and culminated in a face-to-face workshop where project presentations and finalization occurred. The program started with 9 experts in the four diverse fields creating 14 curated projects in data science (Artificial Intelligence/Machine Learning), material science, biofilm engineering, and biointerface. These were integrated into convergence research through webinars by the experts. The experts chose 8 of the 14 projects to be part of an all-day in-person workshop, where over 20 learners formed eight teams that tackled complex problems at the interface of digital image processing, gene expression analysis, and material prediction. Each team was comprised of students and postdoctoral researchers or research scientists from diverse domains including computer science, materials science, and biofilm research. Some projects were selected for presentation at the international IEEE Bioinformatics conference in 2022, with three resulting Machine Learning (ML) models submitted as a journal paper. Students engaged in problem discussions, collaborated with experts from different disciplines, and received guidance in decomposing learning objectives. Based on learner feedback, this successful experience allows for consolidation and integration of convergence research via problem-based learning into the curriculum. Three bioengineering participants, who received training in data science and engineering, have received bioinformatics jobs in biotechnology industries.

Artificial Intelligence-Driven Image Analysis of Bacterial Cells and Biofilms.

The current study explores an artificial intelligence framework for measuring the structural features from microscopy images of the bacterial biofilms. Desulfovibrio alaskensis G20 (DA-G20) grown on mild steel surfaces is used as a model for sulfate reducing bacteria that are implicated in microbiologically influenced corrosion problems. Our goal is to automate the process of extracting the geometrical properties of the DA-G20 cells from the scanning electron microscopy (SEM) images, which is otherwise a laborious and costly process. These geometric properties are a biofilm phenotype that allow us to understand how the biofilm structurally adapts to the surface properties of the underlying metals, which can lead to better corrosion prevention solutions. We adapt two deep learning models: (a) a deep convolutional neural network (DCNN) model to achieve semantic segmentation of the cells, (d) a mask region-convolutional neural network (Mask R-CNN) model to achieve instance segmentation of the cells. These models are then integrated with moment invariants approach to measure the geometric characteristics of the segmented cells. Our numerical studies confirm that the Mask-RCNN and DCNN methods are 227x and 70x faster respectively, compared to the traditional method of manual identification and measurement of the cell geometric properties by the domain experts.

An AI-based approach for detecting cells and microbial byproducts in low volume scanning electron microscope images of biofilms.

Microbially induced corrosion (MIC) of metal surfaces caused by biofilms has wide-ranging consequences. Analysis of biofilm images to understand the distribution of morphological components in images such as microbial cells, MIC byproducts, and metal surfaces non-occluded by cells can provide insights into assessing the performance of coatings and developing new strategies for corrosion prevention. We present an automated approach based on self-supervised deep learning methods to analyze Scanning Electron Microscope (SEM) images and detect cells and MIC byproducts. The proposed approach develops models that can successfully detect cells, MIC byproducts, and non-occluded surface areas in SEM images with a high degree of accuracy using a low volume of data while requiring minimal expert manual effort for annotating images. We develop deep learning network pipelines involving both contrastive (Barlow Twins) and non-contrastive (MoCoV2) self-learning methods and generate models to classify image patches containing three labels-cells, MIC byproducts, and non-occluded surface areas. Our experimental results based on a dataset containing seven grayscale SEM images show that both Barlow Twin and MoCoV2 models outperform the state-of-the-art supervised learning models achieving prediction accuracy increases of approximately 8 and 6%, respectively. The self-supervised pipelines achieved this superior performance by requiring experts to annotate only ~10% of the input data. We also conducted a qualitative assessment of the proposed approach using experts and validated the classification outputs generated by the self-supervised models. This is perhaps the first attempt toward the application of self-supervised learning to classify biofilm image components and our results show that self-supervised learning methods are highly effective for this task while minimizing the expert annotation effort.

Correction of perceived visual distortions using a software application and correlation to age-related macular degeneration.

PURPOSE: To investigate the use of software-generated corrections in neutralizing perceived distortions in age-related macular degeneration. METHODS: A tablet-based application was utilized to elicit distortions. Five subjects (seven eyes: neovascular age-related macular degeneration and three eyes: non-neovascular age-related macular degeneration) traced the reference lines, and their distortion traces were recorded. To counter distortion, a software-generated trace was re-traced by subjects to produce a corrected trace. Final traces were superimposed on optical coherence tomography images and following distances calculated: (a) dDT: distance between distortion trace and reference line; (b) dGT: distance between software-generated trace and corrected trace; (c) dCT: distance between corrected trace and reference line. Mean percent improvement in distortion was reported. Mean effectiveness of correction was also reported by utilizing t test to compare dDT and dCT. The number of distortion traces with underlying lesions on optical coherence tomography was also analyzed. RESULTS: Mean age of the subjects was 76.6 (±9.5) years. Each patient traced six reference lines and each was considered a separate case. Out of 30 cases, 17 (56.6%) elicited distortion. Mean percent improvement in distortion was 71.3 ± 23% (p < 0.05). Twelve cases (70.6%) had an underlying lesion (eight cases: disrupted photoreceptor layer and four cases: normal photoreceptor layer). Mean percent improvement in cases with normal photoreceptor layer (90.8 ± 5.45%) was higher than with abnormal photoreceptor layer (58.5 ± 7.17%) (p < 0.05). Five cases with distortion had no associated underlying lesion. Mean percent improvement in these subjects was significantly higher than those with photoreceptor layer disruption. CONCLUSION: Software-generated corrections can potentially correct for perceived distortions in patients with age-related macular degeneration, especially in cases with preserved photoreceptor layer.

Analyzing OCT images of age-related macular degeneration patients to identify spatial health correlations.

An approach to automatically group age-related macular degeneration (AMD) patients having similar retinal health profiles by clustering Optical Coherence Tomography (OCT) images is described. Spatial health patterns within and across profiles are discovered by identifying segments of images that have similar levels of health in a given retina region. Segmentations of various sizes are considered and the segmentation where the segment similarity most closely matches the discovered health profiles is used to identify health patterns. Our experiments with OCT images of 10 AMD patients show that - i) health profiles generated by clustering closely correspond to those identified by a physician expert, ii) a rich set of spatial patterns can be discovered within and across profiles using regular image segmentation, and iii) new images can be successfully classified into existing profiles so that physicians can provide effective profile-based treatments.

Extracting Hot spots of Topics from Time Stamped Documents.

Identifying time periods with a burst of activities related to a topic has been an important problem in analyzing time-stamped documents. In this paper, we propose an approach to extract a hot spot of a given topic in a time-stamped document set. Topics can be basic, containing a simple list of keywords, or complex. Logical relationships such as and, or, and not are used to build complex topics from basic topics. A concept of presence measure of a topic based on fuzzy set theory is introduced to compute the amount of information related to the topic in the document set. Each interval in the time period of the document set is associated with a numeric value which we call the discrepancy score. A high discrepancy score indicates that the documents in the time interval are more focused on the topic than those outside of the time interval. A hot spot of a given topic is defined as a time interval with the highest discrepancy score. We first describe a naive implementation for extracting hot spots. We then construct an algorithm called EHE (Efficient Hot Spot Extraction) using several efficient strategies to improve performance. We also introduce the notion of a topic DAG to facilitate an efficient computation of presence measures of complex topics. The proposed approach is illustrated by several experiments on a subset of the TDT-Pilot Corpus and DBLP conference data set. The experiments show that the proposed EHE algorithm significantly outperforms the naive one, and the extracted hot spots of given topics are meaningful.