Piximi - An Images to Discovery web tool for bioimages and beyond.

Deep learning has greatly accelerated research in biological image analysis yet it often requires programming skills and specialized tool installation. Here we present Piximi, a modern, no-programming image analysis tool leveraging deep learning. Implemented as a web application at Piximi.app, Piximi requires no installation and can be accessed by any modern web browser. Its client-only architecture preserves the security of researcher data by running all computation locally. Piximi offers four core modules: a deep learning classifier, an image annotator, measurement modules, and pre-trained deep learning segmentation modules. Piximi is interoperable with existing tools and workflows by supporting import and export of common data and model formats. The intuitive researcher interface and easy access to Piximi allows biological researchers to obtain insights into images within just a few minutes. Piximi aims to bring deep learning-powered image analysis to a broader community by eliminating barriers to entry.

A mouse model of TB-associated lung fibrosis reveals persistent inflammatory macrophage populations during treatment.

Post-TB lung disease (PTLD) causes a significant burden of global disease. Fibrosis is a central component of many clinical features of PTLD. To date, we have a limited understanding of the mechanisms of TB-associated fibrosis and how these mechanisms are similar to or dissimilar from other fibrotic lung pathologies. We have adapted a mouse model of TB infection to facilitate the mechanistic study of TB-associated lung fibrosis. We find that the morphologies of fibrosis that develop in the mouse model are similar to the morphologies of fibrosis observed in human tissue samples. Using Second Harmonic Generation (SHG) microscopy, we are able to quantify a major component of fibrosis, fibrillar collagen, over time and with treatment. Inflammatory macrophage subpopulations persist during treatment; matrix remodeling enzymes and inflammatory gene signatures remain elevated. Our mouse model suggests that there is a therapeutic window during which adjunctive therapies could change matrix remodeling or inflammatory drivers of tissue pathology to improve functional outcomes after treatment for TB infection.

Cellpose as a reliable method for single-cell segmentation of autofluorescence microscopy images.

Autofluorescence microscopy uses intrinsic sources of molecular contrast to provide cellular-level information without extrinsic labels. However, traditional cell segmentation tools are often optimized for high signal-to-noise ratio (SNR) images, such as fluorescently labeled cells, and unsurprisingly perform poorly on low SNR autofluorescence images. Therefore, new cell segmentation tools are needed for autofluorescence microscopy. Cellpose is a deep learning network that is generalizable across diverse cell microscopy images and automatically segments single cells to improve throughput and reduce inter-human biases. This study aims to validate Cellpose for autofluorescence imaging, specifically from multiphoton intensity images of NAD(P)H. Manually segmented nuclear masks of NAD(P)H images were used to train new Cellpose models. These models were applied to PANC-1 cells treated with metabolic inhibitors and patient-derived cancer organoids (across 9 patients) treated with chemotherapies. These datasets include co-registered fluorescence lifetime imaging microscopy (FLIM) of NAD(P)H and FAD, so fluorescence decay parameters and the optical redox ratio (ORR) were compared between masks generated by the new Cellpose model and manual segmentation. The Dice score between repeated manually segmented masks was significantly lower than that of repeated Cellpose masks (p<0.0001) indicating greater reproducibility between Cellpose masks. There was also a high correlation (R 2 >0.9) between Cellpose and manually segmented masks for the ORR, mean NAD(P)H lifetime, and mean FAD lifetime across 2D and 3D cell culture treatment conditions. Masks generated from Cellpose and manual segmentation also maintain similar means, variances, and effect sizes between treatments for the ORR and FLIM parameters. Overall, Cellpose provides a fast, reliable, reproducible, and accurate method to segment single cells in autofluorescence microscopy images such that functional changes in cells are accurately captured in both 2D and 3D culture.

A Postdoctoral Training Program in Bioimage Analysis.

We herein describe a postdoctoral training program designed to train biologists with microscopy experience in bioimage analysis. We detail the rationale behind the program, the various components of the training program, and outcomes in terms of works produced and the career effects on past participants. We analyze the results of an anonymous survey distributed to past and present participants, indicating overall high value of all 12 rated aspects of the program, but significant heterogeneity in which aspects were most important to each participant. Finally, we propose this model as a template for other programs which may want to train experts in professional skill sets, and discuss the important considerations when running such a program. We believe that such programs can have extremely positive impact for both the trainees themselves and the broader scientific community.

Three million images and morphological profiles of cells treated with matched chemical and genetic perturbations.

The identification of genetic and chemical perturbations with similar impacts on cell morphology can elucidate compounds' mechanisms of action or novel regulators of genetic pathways. Research on methods for identifying such similarities has lagged due to a lack of carefully designed and well-annotated image sets of cells treated with chemical and genetic perturbations. Here we create such a Resource dataset, CPJUMP1, in which each perturbed gene's product is a known target of at least two chemical compounds in the dataset. We systematically explore the directionality of correlations among perturbations that target the same protein encoded by a given gene, and we find that identifying matches between chemical and genetic perturbations is a challenging task. Our dataset and baseline analyses provide a benchmark for evaluating methods that measure perturbation similarities and impact, and more generally, learn effective representations of cellular state from microscopy images. Such advancements would accelerate the applications of image-based profiling of cellular states, such as uncovering drug mode of action or probing functional genomics.

Creating and troubleshooting microscopy analysis workflows: common challenges and common solutions.

As microscopy diversifies and becomes ever-more complex, the problem of quantification of microscopy images has emerged as a major roadblock for many researchers. All researchers must face certainchallenges in turning microscopy images into answers, independent of their scientific question and the images they've generated. Challenges may arise at many stages throughout the analysis process, including handling of the image files, image pre-processing, object finding, or measurement, and statistical analysis. While the exact solution required for each obstacle will be problem-specific, by understanding tools and tradeoffs, optimizing data quality, breaking workflows and data sets into chunks, talking to experts, and thoroughly documenting what has been done, analysts at any experience level can learn to overcome these challenges and create better and easier image analyses.

Creating and troubleshooting microscopy analysis workflows: Common challenges and common solutions.

As microscopy diversifies and becomes ever more complex, the problem of quantification of microscopy images has emerged as a major roadblock for many researchers. All researchers must face certain challenges in turning microscopy images into answers, independent of their scientific question and the images they have generated. Challenges may arise at many stages throughout the analysis process, including handling of the image files, image pre-processing, object finding, or measurement, and statistical analysis. While the exact solution required for each obstacle will be problem-specific, by keeping analysis in mind, optimizing data quality, understanding tools and tradeoffs, breaking workflows and data sets into chunks, talking to experts, and thoroughly documenting what has been done, analysts at any experience level can learn to overcome these challenges and create better and easier image analyses.

The multimodality cell segmentation challenge: toward universal solutions.

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimental settings. Here, we present a multimodality cell segmentation benchmark, comprising more than 1,500 labeled images derived from more than 50 diverse biological experiments. The top participants developed a Transformer-based deep-learning algorithm that not only exceeds existing methods but can also be applied to diverse microscopy images across imaging platforms and tissue types without manual parameter adjustments. This benchmark and the improved algorithm offer promising avenues for more accurate and versatile cell analysis in microscopy imaging.

Understanding metric-related pitfalls in image analysis validation.

Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

Metrics reloaded: recommendations for image analysis validation.

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Harmonizing the Generation and Pre-publication Stewardship of FAIR Image data.

Together with the molecular knowledge of genes and proteins, biological images promise to significantly enhance the scientific understanding of complex cellular systems and to advance predictive and personalized therapeutic products for human health. For this potential to be realized, quality-assured image data must be shared among labs at a global scale to be compared, pooled, and reanalyzed, thus unleashing untold potential beyond the original purpose for which the data was generated. There are two broad sets of requirements to enable image data sharing in the life sciences. One set of requirements is articulated in the companion White Paper entitled "Enabling Global Image Data Sharing in the Life Sciences," which is published in parallel and addresses the need to build the cyberinfrastructure for sharing the digital array data (arXiv:2401.13023 [q-bio.OT], https://doi.org/10.48550/arXiv.2401.13023). In this White Paper, we detail a broad set of requirements, which involves collecting, managing, presenting, and propagating contextual information essential to assess the quality, understand the content, interpret the scientific implications, and reuse image data in the context of the experimental details. We start by providing an overview of the main lessons learned to date through international community activities, which have recently made considerable progress toward generating community standard practices for imaging Quality Control (QC) and metadata. We then provide a clear set of recommendations for amplifying this work. The driving goal is to address remaining challenges, and democratize access to common practices and tools for a spectrum of biomedical researchers, regardless of their expertise, access to resources, and geographical location.

Learning representations for image-based profiling of perturbations.

Measuring the phenotypic effect of treatments on cells through imaging assays is an efficient and powerful way of studying cell biology, and requires computational methods for transforming images into quantitative data. Here, we present an improved strategy for learning representations of treatment effects from high-throughput imaging, following a causal interpretation. We use weakly supervised learning for modeling associations between images and treatments, and show that it encodes both confounding factors and phenotypic features in the learned representation. To facilitate their separation, we constructed a large training dataset with images from five different studies to maximize experimental diversity, following insights from our causal analysis. Training a model with this dataset successfully improves downstream performance, and produces a reusable convolutional network for image-based profiling, which we call Cell Painting CNN. We evaluated our strategy on three publicly available Cell Painting datasets, and observed that the Cell Painting CNN improves performance in downstream analysis up to 30% with respect to classical features, while also being more computationally efficient.

High-dimensional phenotyping to define the genetic basis of cellular morphology.

The morphology of cells is dynamic and mediated by genetic and environmental factors. Characterizing how genetic variation impacts cell morphology can provide an important link between disease association and cellular function. Here, we combine genomic sequencing and high-content imaging approaches on iPSCs from 297 unique donors to investigate the relationship between genetic variants and cellular morphology to map what we term cell morphological quantitative trait loci (cmQTLs). We identify novel associations between rare protein altering variants in WASF2, TSPAN15, and PRLR with several morphological traits related to cell shape, nucleic granularity, and mitochondrial distribution. Knockdown of these genes by CRISPRi confirms their role in cell morphology. Analysis of common variants yields one significant association and nominate over 300 variants with suggestive evidence (P < 10-6) of association with one or more morphology traits. We then use these data to make predictions about sample size requirements for increasing discovery in cellular genetic studies. We conclude that, similar to molecular phenotypes, morphological profiling can yield insight about the function of genes and variants.

Understanding metric-related pitfalls in image analysis validation.

Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.

Community-developed checklists for publishing images and image analyses.

Images document scientific discoveries and are prevalent in modern biomedical research. Microscopy imaging in particular is currently undergoing rapid technological advancements. However, for scientists wishing to publish obtained images and image-analysis results, there are currently no unified guidelines for best practices. Consequently, microscopy images and image data in publications may be unclear or difficult to interpret. Here, we present community-developed checklists for preparing light microscopy images and describing image analyses for publications. These checklists offer authors, readers and publishers key recommendations for image formatting and annotation, color selection, data availability and reporting image-analysis workflows. The goal of our guidelines is to increase the clarity and reproducibility of image figures and thereby to heighten the quality and explanatory power of microscopy data.

Defining the boundaries: challenges and advances in identifying cells in microscopy images.

Segmentation, or the outlining of objects within images, is a critical step in the measurement and analysis of cells within microscopy images. While improvements continue to be made in tools that rely on classical methods for segmentation, deep learning-based tools increasingly dominate advances in the technology. Specialist models such as Cellpose continue to improve in accuracy and user-friendliness, and segmentation challenges such as the Multi-Modality Cell Segmentation Challenge continue to push innovation in accuracy across widely varying test data as well as efficiency and usability. Increased attention on documentation, sharing, and evaluation standards is leading to increased user-friendliness and acceleration toward the goal of a truly universal method.

Reproducible image-based profiling with Pycytominer.

Technological advances in high-throughput microscopy have facilitated the acquisition of cell images at a rapid pace, and data pipelines can now extract and process thousands of image-based features from microscopy images. These features represent valuable single-cell phenotypes that contain information about cell state and biological processes. The use of these features for biological discovery is known as image-based or morphological profiling. However, these raw features need processing before use and image-based profiling lacks scalable and reproducible open-source software. Inconsistent processing across studies makes it difficult to compare datasets and processing steps, further delaying the development of optimal pipelines, methods, and analyses. To address these issues, we present Pycytominer, an open-source software package with a vibrant community that establishes an image-based profiling standard. Pycytominer has a simple, user-friendly Application Programming Interface (API) that implements image-based profiling functions for processing high-dimensional morphological features extracted from microscopy images of cells. Establishing Pycytominer as a standard image-based profiling toolkit ensures consistent data processing pipelines with data provenance, therefore minimizing potential inconsistencies and enabling researchers to confidently derive accurate conclusions and discover novel insights from their data, thus driving progress in our field.

Assessing the performance of the Cell Painting assay across different imaging systems.

Quantitative microscopy is a powerful method for performing phenotypic screens from which image-based profiling can extract a wealth of information, termed profiles. These profiles can be used to elucidate the changes in cellular phenotypes across cell populations from different patient samples or following genetic or chemical perturbations. One such image-based profiling method is the Cell Painting assay, which provides morphological insight through the imaging of eight cellular compartments. Here, we examine the performance of the Cell Painting assay across multiple high-throughput microscope systems and find that all are compatible with this assay. Furthermore, we determine independently for each microscope system the best performing settings, providing those who wish to adopt this assay an ideal starting point for their own assays. We also explore the impact of microscopy setting changes in the Cell Painting assay and find that few dramatically reduce the quality of a Cell Painting profile, regardless of the microscope used.

CellProfiler plugins - An easy image analysis platform integration for containers and Python tools.

CellProfiler is a widely used software for creating reproducible, reusable image analysis workflows without needing to code. In addition to the >90 modules that make up the main CellProfiler program, CellProfiler has a plugins system that allows for the creation of new modules which integrate with other Python tools or tools that are packaged in software containers. The CellProfiler-plugins repository contains a number of these CellProfiler modules, especially modules that are experimental and/or dependency-heavy. Here, we present an upgraded CellProfiler-plugins repository, an example of accessing containerised tools, improved documentation and added citation/reference tools to facilitate the use and contribution of the community.