Visualizing temporal brain-state changes for fMRI using t-distributed stochastic neighbor embedding.

Purpose: Currently, functional magnetic resonance imaging (fMRI) is the most commonly used technique for obtaining dynamic information about the brain. However, because of the complexity of the data, it is often difficult to directly visualize the temporal aspect of the fMRI data. Approach: We outline a t -distributed stochastic neighbor embedding (t-SNE)-based postprocessing technique that can be used for visualization of temporal information from a 4D fMRI data. Apart from visualization, we also show its utility in detection of major changes in the brain meta-states during the scan duration. Results: The t-SNE approach is able to detect brain-state changes from task to rest and vice versa for single- and multitask fMRI data. A temporal visualization can also be obtained for task and resting state fMRI data for assessing the temporal dynamics during the scan duration. Additionally, hemodynamic delay can be quantified by comparison of the detected brain-state changes with the experiment paradigm for task fMRI data. Conclusion: The t-SNE visualization can visualize help identify major brain-state changes from fMRI data. Such visualization can provide information about the degree of involvement and attentiveness of the subject during the scan and can be potentially utilized as a quality control for subject's performance during the scan.

Automated Cervical Digitized Histology Whole-Slide Image Analysis Toolbox.

BACKGROUND: Cervical intraepithelial neoplasia (CIN) is regarded as a potential precancerous state of the uterine cervix. Timely and appropriate early treatment of CIN can help reduce cervical cancer mortality. Accurate estimation of CIN grade correlated with human papillomavirus type, which is the primary cause of the disease, helps determine the patient's risk for developing the disease. Colposcopy is used to select women for biopsy. Expert pathologists examine the biopsied cervical epithelial tissue under a microscope. The examination can take a long time and is prone to error and often results in high inter-and intra-observer variability in outcomes. METHODOLOGY: We propose a novel image analysis toolbox that can automate CIN diagnosis using whole slide image (digitized biopsies) of cervical tissue samples. The toolbox is built as a four-step deep learning model that detects the epithelium regions, segments the detected epithelial portions, analyzes local vertical segment regions, and finally classifies each epithelium block with localized attention. We propose an epithelium detection network in this study and make use of our earlier research on epithelium segmentation and CIN classification to complete the design of the end-to-end CIN diagnosis toolbox. RESULTS: The results show that automated epithelium detection and segmentation for CIN classification yields comparable results to manually segmented epithelium CIN classification. CONCLUSION: This highlights the potential as a tool for automated digitized histology slide image analysis to assist expert pathologists.

EpithNet: Deep Regression for Epithelium Segmentation in Cervical Histology Images.

BACKGROUND: Automated pathology techniques for detecting cervical cancer at the premalignant stage have advantages for women in areas with limited medical resources. METHODS: This article presents EpithNet, a deep learning approach for the critical step of automated epithelium segmentation in digitized cervical histology images. EpithNet employs three regression networks of varying dimensions of image input blocks (patches) surrounding a given pixel, with all blocks at a fixed resolution, using varying network depth. RESULTS: The proposed model was evaluated on 311 digitized histology epithelial images and the results indicate that the technique maximizes region-based information to improve pixel-wise probability estimates. EpithNet-mc model, formed by intermediate concatenation of the convolutional layers of the three models, was observed to achieve 94% Jaccard index (intersection over union) which is 26.4% higher than the benchmark model. CONCLUSIONS: EpithNet yields better epithelial segmentation results than state-of-the-art benchmark methods.

Spatiotemporal feature extraction and classification of Alzheimer's disease using deep learning 3D-CNN for fMRI data.

Purpose: Through the last three decades, functional magnetic resonance imaging (fMRI) has provided immense quantities of information about the dynamics of the brain, functional brain mapping, and resting-state brain networks. Despite providing such rich functional information, fMRI is still not a commonly used clinical technique due to inaccuracy involved in analysis of extremely noisy data. However, ongoing developments in deep learning techniques suggest potential improvements and better performance in many different domains. Our main purpose is to utilize the potentials of deep learning techniques for fMRI data for clinical use. Approach: We present one such synergy of fMRI and deep learning, where we apply a simplified yet accurate method using a modified 3D convolutional neural networks (CNN) to resting-state fMRI data for feature extraction and classification of Alzheimer's disease (AD). The CNN is designed in such a way that it uses the fMRI data with much less preprocessing, preserving both spatial and temporal information. Results: Once trained, the network is successfully able to classify between fMRI data from healthy controls and AD subjects, including subjects in the mild cognitive impairment (MCI) stage. We have also extracted spatiotemporal features useful for classification. Conclusion: This CNN can detect and differentiate between the earlier and later stages of MCI and AD and hence, it may have potential clinical applications in both early detection and better diagnosis of Alzheimer's disease.

DeepCIN: Attention-Based Cervical histology Image Classification with Sequential Feature Modeling for Pathologist-Level Accuracy.

BACKGROUND: Cervical cancer is one of the deadliest cancers affecting women globally. Cervical intraepithelial neoplasia (CIN) assessment using histopathological examination of cervical biopsy slides is subject to interobserver variability. Automated processing of digitized histopathology slides has the potential for more accurate classification for CIN grades from normal to increasing grades of pre-malignancy: CIN1, CIN2, and CIN3. METHODOLOGY: Cervix disease is generally understood to progress from the bottom (basement membrane) to the top of the epithelium. To model this relationship of disease severity to spatial distribution of abnormalities, we propose a network pipeline, DeepCIN, to analyze high-resolution epithelium images (manually extracted from whole-slide images) hierarchically by focusing on localized vertical regions and fusing this local information for determining Normal/CIN classification. The pipeline contains two classifier networks: (1) a cross-sectional, vertical segment-level sequence generator is trained using weak supervision to generate feature sequences from the vertical segments to preserve the bottom-to-top feature relationships in the epithelium image data and (2) an attention-based fusion network image-level classifier predicting the final CIN grade by merging vertical segment sequences. RESULTS: The model produces the CIN classification results and also determines the vertical segment contributions to CIN grade prediction. CONCLUSION: Experiments show that DeepCIN achieves pathologist-level CIN classification accuracy.

Assessment of Data Augmentation Strategies Toward Performance Improvement of Abnormality Classification in Chest Radiographs.

Image augmentation is a commonly performed technique to prevent class imbalance in datasets to compensate for insufficient training samples, or to prevent model overfitting. Traditional augmentation (TA) techniques include various image transformations, such as rotation, translation, channel splitting, etc. Alternatively, Generative Adversarial Network (GAN), due to its proven ability to synthesize convincingly-realistic images, has been used to perform image augmentation as well. However, it is unclear whether GAN augmentation (GA) strategy provides an advantage over TA for medical image classification tasks. In this paper, we study the usefulness of TA and GA for classifying abnormal chest X-ray (CXR) images. We first trained a progressive-growing GAN (PG-GAN) to synthesize high-resolution CXRs for performing GA. Then, we trained an abnormality classifier using three training sets individually - training set with TA, with GA and with no augmentation (NA). Finally, we analyzed the abnormality classifier's performance for the three training cases, which led to the following conclusions: (1) GAN strategy is not always superior to TA for improving the classifier's performance; (2) in comparison to NA, however, both TA and GA leads to a significant performance improvement; and, (3) increasing the quantity of images in TA and GA strategies also improves the classifier's performance.

Performance Evaluation of a Generative Adversarial Network for Deblurring Mobile-phone Cervical Images.

Visual examination forms an integral part of cervical cancer screening. With the recent rise in smartphone-based health technologies, capturing cervical images using a smartphone camera for telemedicine and automated screening is gaining popularity. However, such images are highly prone to image corruption, typically out-of-focus target or camera shake blur. In this paper, we applied a generative adversarial network (GAN) to deblur mobile-phone cervical (MC) images, and we evaluate the deblur quality using various measures. Our evaluation process is three-fold: first, we calculate the peak signal to noise ratio (PSNR) and the structural similarity (SSIM) of a test dataset with ground truth availability. Next, we calculate the perception based image quality evaluator (PIQE) score of a test dataset without ground truth availability. Finally, we classify a dataset of blurred and the corresponding deblurred images into normal/abnormal MC images. The resulting change in classification accuracy was our final assessment. Our evaluation experiments show that deblurring of MC images can potentially improve the accuracy of both manual and automated cancerous lesion screening.

Comparing Deep Learning Models for Multi-cell Classification in Liquid- based Cervical Cytology Image.

Liquid-based cytology (LBC) is a reliable automated technique for the screening of Papanicolaou (Pap) smear data. It is an effective technique for collecting a majority of the cervical cells and aiding cytopathologists in locating abnormal cells. Most methods published in the research literature rely on accurate cell segmentation as a prior, which remains challenging due to a variety of factors, e.g., stain consistency, presence of clustered cells, etc. We propose a method for automatic classification of cervical slide images through generation of labeled cervical patch data and extracting deep hierarchical features by fine-tuning convolution neural networks, as well as a novel graph-based cell detection approach for cellular level evaluation. The results show that the proposed pipeline can classify images of both single cell and overlapping cells. The VGG-19 model is found to be the best at classifying the cervical cytology patch data with 95 % accuracy under precision-recall curve.

Extraction of Aortic Knuckle Contour in Chest Radiographs Using Deep Learning.

In this paper, we aim to extract the aortic knuckle (AK) contour in chest radiographs, an anatomical structure rarely being addressed in the literature. Since the AK structure is small and thin, simply adopting the deep network methods that are successful for large organ segmentation is inadequate for achieving good pixel-level accuracy and resolving local ambiguities. To address this challenge, we propose a new coarse-to-fine segmentation approach which focuses on global and local information contexts, respectively. Two convolutional networks are used. For the coarse segmentation, we use FasterRCNN; for the fine segmentation, we use U-Net. Our evaluation uses the publicly available JSRT dataset; the results are promising. Besides presenting these results, we analyze issues such as the imprecision of manual contour marking, and automatic generation of the coarse segmentation ground-truth mask used for deep network training. Our approach is general and can be applied to extract other curve-like objects-of-interest.

Deep Learning Nuclei Detection in Digitized Histology Images by Superpixels.

BACKGROUND: Advances in image analysis and computational techniques have facilitated automatic detection of critical features in histopathology images. Detection of nuclei is critical for squamous epithelium cervical intraepithelial neoplasia (CIN) classification into normal, CIN1, CIN2, and CIN3 grades. METHODS: In this study, a deep learning (DL)-based nuclei segmentation approach is investigated based on gathering localized information through the generation of superpixels using a simple linear iterative clustering algorithm and training with a convolutional neural network. RESULTS: The proposed approach was evaluated on a dataset of 133 digitized histology images and achieved an overall nuclei detection (object-based) accuracy of 95.97%, with demonstrated improvement over imaging-based and clustering-based benchmark techniques. CONCLUSIONS: The proposed DL-based nuclei segmentation Method with superpixel analysis has shown improved segmentation results in comparison to state-of-the-art methods.

Enhancements in localized classification for uterine cervical cancer digital histology image assessment.

BACKGROUND: In previous research, we introduced an automated, localized, fusion-based approach for classifying uterine cervix squamous epithelium into Normal, CIN1, CIN2, and CIN3 grades of cervical intraepithelial neoplasia (CIN) based on digitized histology image analysis. As part of the CIN assessment process, acellular and atypical cell concentration features were computed from vertical segment partitions of the epithelium region to quantize the relative distribution of nuclei. METHODS: Feature data was extracted from 610 individual segments from 61 images for epithelium classification into categories of Normal, CIN1, CIN2, and CIN3. The classification results were compared against CIN labels obtained from two pathologists who visually assessed abnormality in the digitized histology images. In this study, individual vertical segment CIN classification accuracy improvement is reported using the logistic regression classifier for an expanded data set of 118 histology images. RESULTS: We analyzed the effects on classification using the same pathologist labels for training and testing versus using one pathologist labels for training and the other for testing. Based on a leave-one-out approach for classifier training and testing, exact grade CIN accuracies of 81.29% and 88.98% were achieved for individual vertical segment and epithelium whole-image classification, respectively. CONCLUSIONS: The Logistic and Random Tree classifiers outperformed the benchmark SVM and LDA classifiers from previous research. The Logistic Regression classifier yielded an improvement of 10.17% in CIN Exact grade classification results based on CIN labels for training-testing for the individual vertical segments and the whole image from the same single expert over the baseline approach using the reduced features. Overall, the CIN classification rates tended to be higher using the training-testing labels for the same expert than for training labels from one expert and testing labels from the other expert. The Exact class fusion- based CIN discrimination results obtained in this study are similar to the Exact class expert agreement rate.

Nuclei-Based Features for Uterine Cervical Cancer Histology Image Analysis With Fusion-Based Classification.

Cervical cancer, which has been affecting women worldwide as the second most common cancer, can be cured if detected early and treated well. Routinely, expert pathologists visually examine histology slides for cervix tissue abnormality assessment. In previous research, we investigated an automated, localized, fusion-based approach for classifying squamous epithelium into Normal, CIN1, CIN2, and CIN3 grades of cervical intraepithelial neoplasia (CIN) based on image analysis of 61 digitized histology images. This paper introduces novel acellular and atypical cell concentration features computed from vertical segment partitions of the epithelium region within digitized histology images to quantize the relative increase in nuclei numbers as the CIN grade increases. Based on the CIN grade assessments from two expert pathologists, image-based epithelium classification is investigated with voting fusion of vertical segments using support vector machine and linear discriminant analysis approaches. Leave-one-out is used for the training and testing for CIN classification, achieving an exact grade labeling accuracy as high as 88.5%.

A fusion-based approach for uterine cervical cancer histology image classification.

Expert pathologists commonly perform visual interpretation of histology slides for cervix tissue abnormality diagnosis. We investigated an automated, localized, fusion-based approach for cervix histology image analysis for squamous epithelium classification into Normal, CIN1, CIN2, and CIN3 grades of cervical intraepithelial neoplasia (CIN). The epithelium image analysis approach includes medial axis determination, vertical segment partitioning as medial axis orthogonal cuts, individual vertical segment feature extraction and classification, and image-based classification using a voting scheme fusing the vertical segment CIN grades. Results using 61 images showed at least 15.5% CIN exact grade classification improvement using the localized vertical segment fusion versus global image features.

Human papillomavirus load measured by Linear Array correlates with quantitative PCR in cervical cytology specimens.

Carcinogenic human papillomavirus (HPV) infections are necessary causes of most anogenital cancers. Viral load has been proposed as a marker for progression to cancer precursors but has been confirmed only for HPV16. Challenges in studying viral load are related to the lack of validated assays for a large number of genotypes. We compared viral load measured by Linear Array (LA) HPV genotyping with the gold standard, quantitative PCR (Q-PCR). LA genotyping and Q-PCR were performed in 143 cytology specimens from women referred to colposcopy. LA signal strength was measured by densitometry. Correlation coefficients and receiver operating characteristic (ROC) analyses were used to evaluate analytical and clinical performance. We observed a moderate to strong correlation between the two quantitative viral load measurements, ranging from an R value of 0.61 for HPV31 to an R value of 0.86 for HPV52. We also observed agreement between visual LA signal strength evaluation and Q-PCR. Both quantifications agreed on the disease stages with highest viral load, which varied by type (cervical intraepithelial neoplasia grade 2 [CIN2] for HPV52, CIN3 for HPV16 and HPV33, and cancer for HPV18 and HPV31). The area under the curve (AUC) for HPV16 Q-PCR at the CIN3 cutoff was 0.72 (P = 0.004), and the AUC for HPV18 LA at the CIN2 cutoff was 0.78 (P = 0.04). Quantification of LA signals correlates with the current gold standard for viral load, Q-PCR. Analyses of viral load need to address multiple infections and type attribution to evaluate whether viral load has clinical value beyond the established HPV16 finding. Our findings support conducting comprehensive studies of viral load and cervical cancer precursors using quantitative LA genotyping data.

Clinical and pathological heterogeneity of cervical intraepithelial neoplasia grade 3.

OBJECTIVE: Cervical intraepithelial neoplasia grade 3 (CIN3), the immediate cervical cancer precursor, is a target of cervical cancer prevention. However, less than half of CIN3s will progress to cancer. Routine treatment of all CIN3s and the majority of CIN2s may lead to overtreatment of many lesions that would not progress. To improve our understanding of CIN3 natural history, we performed a detailed characterization of CIN3 heterogeneity in a large referral population in the US. METHODS: We examined 309 CIN3 cases in the SUCCEED, a large population-based study of women with abnormal cervical cancer screening results. Histology information for 12 individual loop electrosurgical excision procedure (LEEP) segments was evaluated for each woman. We performed case-case comparisons of CIN3s to analyze determinants of heterogeneity and screening test performance. RESULTS: CIN3 cases varied substantially by size (1-10 LEEP segments) and by presentation with concomitant CIN2 and CIN1. All grades of CINs were equally distributed over the cervical surface. In half of the women, CIN3 lesions were found as multiple distinct lesions on the cervix. Women with large and solitary CIN3 lesions were more likely to be older, have longer sexual activity span, and have fewer multiple high risk HPV infections. Screening frequency, but not HPV16 positivity, was an important predictor of CIN3 size. Large CIN3 lesions were also characterized by high-grade clinical test results. CONCLUSIONS: We demonstrate substantial heterogeneity in clinical and pathological presentation of CIN3 in a US population. Time since sexual debut and participation in screening were predictors of CIN3 size. We did not observe a preferential site of CIN3 on the cervical surface that could serve as a target for cervical biopsy. Cervical cancer screening procedures were more likely to detect larger CIN3s, suggesting that CIN3s detected by multiple independent diagnostic tests may represent cases with increased risk of invasion.

On the creation of a segmentation library for digitized cervical and lumbar spine radiographs.

In this paper, we address the issue of computer-assisted indexing in one specific case, i.e., for the 17,000 digitized images of the spine acquired during the National Health and Nutrition Examination Survey (NHANES). The crucial step in this process is to accurately segment the cervical and lumbar spine in the radiographic images. To that end, we have implemented a unique segmentation system that consists of a suite of spine-customized automatic and semi-automatic statistical shape segmentation algorithms. Using the aforementioned system, we have developed experiments to optimally generate a library of spine segmentations, which currently include 2000 cervical and 2000 lumbar spines. This work is expected to contribute toward the creation of a biomedical Content-Based Image Retrieval system that will allow retrieval of vertebral shapes by using query by image example or query by shape example.

Optimal embedding for shape indexing in medical image databases.

This paper addresses the problem of indexing shapes in medical image databases. Shapes of organs are often indicative of disease, making shape similarity queries important in medical image databases. Mathematically, shapes with landmarks belong to shape spaces which are curved manifolds with a well defined metric. The challenge in shape indexing is to index data in such curved spaces. One natural indexing scheme is to use metric trees, but metric trees are prone to inefficiency. This paper proposes a more efficient alternative. We show that it is possible to optimally embed finite sets of shapes in shape space into a Euclidean space. After embedding, classical coordinate-based trees can be used for efficient shape retrieval. The embedding proposed in the paper is optimal in the sense that it least distorts the partial Procrustes shape distance. The proposed indexing technique is used to retrieve images by vertebral shape from the NHANES II database of cervical and lumbar spine X-ray images maintained at the National Library of Medicine. Vertebral shape strongly correlates with the presence of osteophytes, and shape similarity retrieval is proposed as a tool for retrieval by osteophyte presence and severity. Experimental results included in the paper evaluate (1) the usefulness of shape similarity as a proxy for osteophytes, (2) the computational and disk access efficiency of the new indexing scheme, (3) the relative performance of indexing with embedding to the performance of indexing without embedding, and (4) the computational cost of indexing using the proposed embedding versus the cost of an alternate embedding. The experimental results clearly show the relevance of shape indexing and the advantage of using the proposed embedding.

Double-edge detection of radiographic lumbar vertebrae images using pressurized open DGVF snakes.

The detection of double edges in X-ray images of lumbar vertebrae is of prime importance in the assessment of vertebral injury or collapse that may be caused by osteoporosis and other spine pathology. In addition, if the above double-edge detection process is conducted within an automatic framework, it would not only facilitate inexpensive and fast means of obtaining objective morphometric measurements on the spine, but also remove the human subjectivity involved in the morphometric analysis. This paper proposes a novel force-formulation scheme, termed as pressurized open directional gradient vector flow snakes, to discriminate and detect the superior and inferior double edges present in the radiographic images of the lumbar vertebrae. As part of the validation process, this algorithm is applied to a set of 100 lumbar images and the detection results are quantified using analyst-generated ground truth. The promising nature of the detection results bears testimony to the efficacy of the proposed approach.

An evaluation by midwives and gynecologists of treatability of cervical lesions by cryotherapy among human papillomavirus-positive women.

OBJECTIVES: To estimate efficacy of a visual triage of human papillomavirus (HPV)-positive women to either immediate cryotherapy or referral if not treatable (eg, invasive cancer, large precancers). METHODS: We evaluated visual triage in the HPV-positive women aged 25 to 55 years from the 10,000-woman Guanacaste Cohort Study (n = 552). Twelve Peruvian midwives and 5 international gynecologists assessed treatability by cryotherapy using digitized high-resolution cervical images taken at enrollment. The reference standard of treatability was determined by 2 lead gynecologists from the entire 7-year follow-up of the women. Women diagnosed with histologic cervical intraepithelial neoplasia grade 2 or worse or 5-year persistence of carcinogenic HPV infection were defined as needing treatment. RESULTS: Midwives and gynecologists judged 30.8% and 41.2% of women not treatable by cryotherapy, respectively (P < 0.01). Among 149 women needing treatment, midwives and gynecologists correctly identified 57.5% and 63.8% (P = 0.07 for difference) of 71 women judged not treatable by the lead gynecologists and 77.6% and 59.7% (P < 0.01 for difference) of 78 women judged treatable by cryotherapy. The proportion of women judged not treatable by a reviewer varied widely and ranged from 18.6% to 61.1%. Interrater agreement was poor with mean pairwise overall agreement of 71.4% and 66.3% and kappa's of 0.33 and 0.30 for midwives and gynecologists, respectively. CONCLUSIONS: In future "screen-and-treat" cervical cancer prevention programs using HPV testing and cryotherapy, practitioners will visually triage HPV-positive women. The suboptimal performance of visual triage suggests that screen-and-treat programs using cryotherapy might be insufficient for treating precancerous lesions. Improved, low-technology triage methods and/or improved safe and low-technology treatment options are needed.

Treatability by cryotherapy in a screen-and-treat strategy.

OBJECTIVES: We estimated the percentage of women infected with human papillomavirus (HPV+) who cannot be immediately treated with cryotherapy. MATERIALS AND METHODS: In a 10,000-woman Costa Rican cohort, we analyzed the 559 HPV+ women aged 25 to 55 years and estimated the proportion for whom immediate cryotherapy was not indicated (i.e., invasive cancer, large precancerous lesions, or benign abnormalities that risk failure such as large ectopy, squamocolumnar junction not visualized, polyps, ulcers, or distorted or atrophied cervix). To determine whether cryotherapy at time of baseline HPV screening would effectively treat HPV+ women, 2 expert gynecologists independently judged entire clinical histories (5-7 years of cytology, histology, and HPV tests) and a full longitudinal series of digitized cervical images. RESULTS: Reviewers judged 144 (25.8%) of 559 HPV+ women as not treatable by immediate cryotherapy. Among 72 women with cervical intraepithelial neoplasia grade 3 who would benefit most from a screening program, 35 (48.6%) were not treatable. In particular, 29 women (40.3%) were determined not treatable for reasons most likely associated with cryotherapy's inadequacy (lesion was large, suspected cancerous or in the endocervical canal or fornix). CONCLUSIONS: "Screen-and-treat" programs in low-resource settings will soon use a rapid HPV test to screen older women once or twice in their lifetime, identifying women at higher risk for precancer. Our findings suggest that cryotherapy might not effectively treat many precancers, and other safe, low-technology treatment options could be required, in a scenario where all HPV+ women in this targeted group would receive cryotherapy at the same visit.