Automated diagnosis of optical coherence tomography imaging on plaque vulnerability and its relation to clinical outcomes in coronary artery disease.

This study sought to develop a deep learning-based diagnostic algorithm for plaque vulnerability by analyzing intravascular optical coherence tomography (OCT) images and to investigate the relation between AI-plaque vulnerability and clinical outcomes in patients with coronary artery disease (CAD). A total of 1791 study patients who underwent OCT examinations were recruited from a multicenter clinical database, and the OCT images were first labeled as either normal, a stable plaque, or a vulnerable plaque by expert cardiologists. A DenseNet-121-based deep learning algorithm for plaque characterization was developed by training with 44,947 prelabeled OCT images, and demonstrated excellent differentiation among normal, stable plaques, and vulnerable plaques. Patients who were diagnosed with vulnerable plaques by the algorithm had a significantly higher rate of both events from the OCT-observed segments and clinical events than the patients with normal and stable plaque (log-rank p < 0.001). On the multivariate logistic regression analyses, the OCT diagnosis of a vulnerable plaque by the algorithm was independently associated with both types of events (p = 0.047 and p < 0.001, respectively). The AI analysis of intracoronary OCT imaging can assist cardiologists in diagnosing plaque vulnerability and identifying CAD patients with a high probability of occurrence of future clinical events.

Diagnostic performance for pulmonary adenocarcinoma on CT: comparison of radiologists with and without three-dimensional convolutional neural network.

OBJECTIVES: To compare diagnostic performance for pulmonary invasive adenocarcinoma among radiologists with and without three-dimensional convolutional neural network (3D-CNN). METHODS: Enrolled were 285 patients with adenocarcinoma in situ (AIS, n = 75), minimally invasive adenocarcinoma (MIA, n = 58), and invasive adenocarcinoma (IVA, n = 152). A 3D-CNN model was constructed with seven convolution-pooling and two max-pooling layers and fully connected layers, in which batch normalization, residual connection, and global average pooling were used. Only the flipping process was performed for augmentation. The output layer comprised two nodes for two conditions (AIS/MIA and IVA) according to prognosis. Diagnostic performance of the 3D-CNN model in 285 patients was calculated using nested 10-fold cross-validation. In 90 of 285 patients, results from each radiologist (R1, R2, and R3; with 9, 14, and 26 years of experience, respectively) with and without the 3D-CNN model were statistically compared. RESULTS: Without the 3D-CNN model, accuracy, sensitivity, and specificity of the radiologists were as follows: R1, 70.0%, 52.1%, and 90.5%; R2, 72.2%, 75%, and 69%; and R3, 74.4%, 89.6%, and 57.1%, respectively. With the 3D-CNN model, accuracy, sensitivity, and specificity of the radiologists were as follows: R1, 72.2%, 77.1%, and 66.7%; R2, 74.4%, 85.4%, and 61.9%; and R3, 74.4%, 93.8%, and 52.4%, respectively. Diagnostic performance of each radiologist with and without the 3D-CNN model had no significant difference (p > 0.88), but the accuracy of R1 and R2 was significantly higher with than without the 3D-CNN model (p < 0.01). CONCLUSIONS: The 3D-CNN model can support a less-experienced radiologist to improve diagnostic accuracy for pulmonary invasive adenocarcinoma without deteriorating any diagnostic performances. KEY POINTS: • The 3D-CNN model is a non-invasive method for predicting pulmonary invasive adenocarcinoma in CT images with high sensitivity. • Diagnostic accuracy by a less-experienced radiologist was better with the 3D-CNN model than without the model.

Improvement of nerve imaging speed with coherent anti-Stokes Raman scattering rigid endoscope using deep-learning noise reduction.

A coherent anti-Stokes Raman scattering (CARS) rigid endoscope was developed to visualize peripheral nerves without labeling for nerve-sparing endoscopic surgery. The developed CARS endoscope had a problem with low imaging speed, i.e. low imaging rate. In this study, we demonstrate that noise reduction with deep learning boosts the nerve imaging speed with CARS endoscopy. We employ fine-tuning and ensemble learning and compare deep learning models with three different architectures. In the fine-tuning strategy, deep learning models are pre-trained with CARS microscopy nerve images and retrained with CARS endoscopy nerve images to compensate for the small dataset of CARS endoscopy images. We propose using the equivalent imaging rate (EIR) as a new evaluation metric for quantitatively and directly assessing the imaging rate improvement by deep learning models. The highest EIR of the deep learning model was 7.0 images/min, which was 5 times higher than that of the raw endoscopic image of 1.4 images/min. We believe that the improvement of the nerve imaging speed will open up the possibility of reducing postoperative dysfunction by intraoperative nerve identification.

Nerve Segmentation with Deep Learning from Label-Free Endoscopic Images Obtained Using Coherent Anti-Stokes Raman Scattering.

Semantic segmentation with deep learning to extract nerves from label-free endoscopic images obtained using coherent anti-Stokes Raman scattering (CARS) for nerve-sparing surgery is described. We developed a CARS rigid endoscope in order to identify the exact location of peripheral nerves in surgery. Myelinated nerves are visualized with a CARS lipid signal in a label-free manner. Because the lipid distribution includes other tissues as well as nerves, nerve segmentation is required to achieve nerve-sparing surgery. We propose using U-Net with a VGG16 encoder as a deep learning model and pre-training with fluorescence images, which visualize the lipid distribution similar to CARS images, before fine-tuning with a small dataset of CARS endoscopy images. For nerve segmentation, we used 24 CARS and 1,818 fluorescence nerve images of three rabbit prostates. We achieved label-free nerve segmentation with a mean accuracy of 0.962 and an F 1 value of 0.860. Pre-training on fluorescence images significantly improved the performance of nerve segmentation in terms of the mean accuracy and F 1 value ( p < 0 . 05 ). Nerve segmentation of label-free endoscopic images will allow for safer endoscopic surgery, while reducing dysfunction and improving prognosis after surgery.

Convolutional Neural Network Can Recognize Drug Resistance of Single Cancer Cells.

It is known that single or isolated tumor cells enter cancer patients' circulatory systems. These circulating tumor cells (CTCs) are thought to be an effective tool for diagnosing cancer malignancy. However, handling CTC samples and evaluating CTC sequence analysis results are challenging. Recently, the convolutional neural network (CNN) model, a type of deep learning model, has been increasingly adopted for medical image analyses. However, it is controversial whether cell characteristics can be identified at the single-cell level by using machine learning methods. This study intends to verify whether an AI system could classify the sensitivity of anticancer drugs, based on cell morphology during culture. We constructed a CNN based on the VGG16 model that could predict the efficiency of antitumor drugs at the single-cell level. The machine learning revealed that our model could identify the effects of antitumor drugs with ~0.80 accuracies. Our results show that, in the future, realizing precision medicine to identify effective antitumor drugs for individual patients may be possible by extracting CTCs from blood and performing classification by using an AI system.

Activation of PKC induces leukocyte adhesion by the dephosphorylation of ERM.

Although circulating leukocytes are non-adherent cells, they also undergo adhesion in response to external stimuli. To elucidate this switch mechanism, we investigated PMA-induced cell adhesion in myelomonocytic KG-1 cells. PMA induced microvillius collapse, decrease of cell surface rigidity and exclusion of sialomucin from adhesion sites. All these adhesion-contributing events are linked to dephosphorylation of Ezrin/Radixin/Moesin (ERM) proteins. Indeed, PMA-treatment induced quick decrease of phosphorylated ERM proteins, while expression of Moesin-T558D, a phospho-mimetic mutant, inhibited PMA-induced cell adhesion. PMA-induced cell adhesion and ERM-dephophorylation were inhibited by PKC inhibitors or by a phosphatase inhibitor, indicating the involvement of PKC and protein phophatase in these processes. In peripheral T lymphocytes, ERM-dephosphorylation by adhesion-inducing stimuli was inhibited by a PKC inhibitor. Combined, these findings strongly suggest that external stimuli induce ERM-dephosphorylation via the activation of PKC in leukocytes and that ERM-dephosphorylation leads to leukocytes' adhesion.

Deep-UV excitation fluorescence microscopy for detection of lymph node metastasis using deep neural network.

Deep-UV (DUV) excitation fluorescence microscopy has potential to provide rapid diagnosis with simple technique comparing to conventional histopathology based on hematoxylin and eosin (H&E) staining. We established a fluorescent staining protocol for DUV excitation fluorescence imaging that has enabled clear discrimination of nucleoplasm, nucleolus, and cytoplasm. Fluorescence images of metastasis-positive/-negative lymph nodes of gastric cancer patients were used for patch-based training with a deep neural network (DNN) based on Inception-v3 architecture. The performance on small patches of the fluorescence images was comparable with that of H&E images. Gradient-weighted class activation mapping analysis revealed the areas where the trained model identified metastatic lesions in the images containing cancer cells. We extended the method to large-size image analysis enabling accurate detection of metastatic lesions. We discuss usefulness of DUV excitation fluorescence imaging with the aid of DNN analysis, which is promising for assisting pathologists in assessment of lymph node metastasis.

Sialomucin and phosphorylated-ERM are inhibitors for cadherin-mediated aggregate formation.

Cell adhesion is mediated by adhesion molecules, but also regulated by adhesion inhibitory molecules. Molecules such as leukocyte sialomucin and phosphorylated-Ezrin/Radixin/Moesin (ERM) inhibit cell-substratum adhesion. Here we show that these adhesion inhibitory molecules also inhibit aggregate formation of adherent cells in suspension culture. Expression of sialomucin, CD43 or CD34, inhibited formation of packed aggregates in HEK293T cells. Deletion mutant analysis and enzymatic cleavage indicated the significance of the extracellular sialomucin domain for this inhibition. Meanwhile, phosphorylated-ERM were decreased coincidently with aggregate formation. Combined with the inhibition of aggregate formation by the expression of phospho-mimetic Moesin mutant (Moesin-T558D), phosphorylated-ERM are inhibitors for aggregate formation. Increase of phosphorylated-ERM by CD43 and sialomucin-dependence of Moesin-T558D's inhibition indicate that sialomucin and phosphorylated-ERM collaborate to inhibit aggregate formation. Because aggregate formation of HEK293T cells is mediated by N-cadherin, sialomucin and phosphorylated-ERM inhibit cadherin-mediated cell-cell adhesion. Thus, sialomucin and phosphorylated-ERM are inhibitors for both cell-cell adhesion and cell-substratum adhesion, and regulation of these inhibitory molecules is essential for cell adhesion.

Application of deep learning (3-dimensional convolutional neural network) for the prediction of pathological invasiveness in lung adenocarcinoma: A preliminary study.

To compare results for radiological prediction of pathological invasiveness in lung adenocarcinoma between radiologists and a deep learning (DL) system.Ninety patients (50 men, 40 women; mean age, 66 years; range, 40-88 years) who underwent pre-operative chest computed tomography (CT) with 0.625-mm slice thickness were included in this retrospective study. Twenty-four cases of adenocarcinoma in situ (AIS), 20 cases of minimally invasive adenocarcinoma (MIA), and 46 cases of invasive adenocarcinoma (IVA) were pathologically diagnosed. Three radiologists of different levels of experience diagnosed each nodule by using previously documented CT findings to predict pathological invasiveness. DL was structured using a 3-dimensional (3D) convolutional neural network (3D-CNN) constructed with 2 successive pairs of convolution and max-pooling layers, and 2 fully connected layers. The output layer comprises 3 nodes to recognize the 3 conditions of adenocarcinoma (AIS, MIA, and IVA) or 2 nodes for 2 conditions (AIS and MIA/IVA). Results from DL and the 3 radiologists were statistically compared.No significant differences in pathological diagnostic accuracy rates were seen between DL and the 3 radiologists (P >.11). Receiver operating characteristic analysis demonstrated that area under the curve for DL (0.712) was almost the same as that for the radiologist with extensive experience (0.714; P = .98). Compared with the consensus results from radiologists, DL offered significantly inferior sensitivity (P = .0005), but significantly superior specificity (P = .02).Despite the small training data set, diagnostic performance of DL was almost the same as the radiologist with extensive experience. In particular, DL provided higher specificity than radiologists.

[Medical Application of Artificial Intelligence/Deep Learning].

Deep learning is a subset of the medical application of artificial intelligence. Its significant results are garnering attention, particularly in radiographic image interpretation, pathological diagnosis, gene analysis, and prediction of cancer recurrence. In this study, we summarize the concept of deep learning. The human body structure, from the molecule to physical functions, is a complex system. Deep learning is a new way to analyze its complex systems. An essential point of the analysis is the categorization of obstacles. To a certain extent, deep learning approximates a doctor's cognition.

Graphical classification of DNA sequences of HLA alleles by deep learning.

Alleles of human leukocyte antigen (HLA)-A DNAs are classified and expressed graphically by using artificial intelligence "Deep Learning (Stacked autoencoder)". Nucleotide sequence data corresponding to the length of 822 bp, collected from the Immuno Polymorphism Database, were compressed to 2-dimensional representation and were plotted. Profiles of the two-dimensional plots indicate that the alleles can be classified as clusters are formed. The two-dimensional plot of HLA-A DNAs gives a clear outlook for characterizing the various alleles.

Classification of C2C12 cells at differentiation by convolutional neural network of deep learning using phase contrast images.

In the field of regenerative medicine, tremendous numbers of cells are necessary for tissue/organ regeneration. Today automatic cell-culturing system has been developed. The next step is constructing a non-invasive method to monitor the conditions of cells automatically. As an image analysis method, convolutional neural network (CNN), one of the deep learning method, is approaching human recognition level. We constructed and applied the CNN algorithm for automatic cellular differentiation recognition of myogenic C2C12 cell line. Phase-contrast images of cultured C2C12 are prepared as input dataset. In differentiation process from myoblasts to myotubes, cellular morphology changes from round shape to elongated tubular shape due to fusion of the cells. CNN abstract the features of the shape of the cells and classify the cells depending on the culturing days from when differentiation is induced. Changes in cellular shape depending on the number of days of culture (Day 0, Day 3, Day 6) are classified with 91.3% accuracy. Image analysis with CNN has a potential to realize regenerative medicine industry.

Cysteine-rich protein 2 accelerates actin filament cluster formation.

Filamentous actin (F-actin) forms many types of structures and dynamically regulates cell morphology and movement, and plays a mechanosensory role for extracellular stimuli. In this study, we determined that the smooth muscle-related transcription factor, cysteine-rich protein 2 (CRP2), regulates the supramolecular networks of F-actin. The structures of CRP2 and F-actin in solution were analyzed by small-angle X-ray solution scattering (SAXS). The general shape of CRP2 was partially unfolded and relatively ellipsoidal in structure, and the apparent cross sectional radius of gyration (Rc) was about 15.8 Å. The predicted shape, derived by ab initio modeling, consisted of roughly four tandem clusters: LIM domains were likely at both ends with the middle clusters being an unfolded linker region. From the SAXS analysis, the Rc of F-actin was about 26.7 Å, and it was independent of CRP2 addition. On the other hand, in the low angle region of the CRP2-bound F-actin scattering, the intensities showed upward curvature with the addition of CRP2, which indicates increasing branching of F-actin following CRP2 binding. From biochemical analysis, the actin filaments were augmented and clustered by the addition of CRP2. This F-actin clustering activity of CRP2 was cooperative with α-actinin. Thus, binding of CRP2 to F-actin accelerates actin polymerization and F-actin cluster formation.

In silico characterization of cell-cell interactions using a cellular automata model of cell culture.

BACKGROUND: Cell proliferation is a key characteristic of eukaryotic cells. During cell proliferation, cells interact with each other. In this study, we developed a cellular automata model to estimate cell-cell interactions using experimentally obtained images of cultured cells. RESULTS: We used four types of cells; HeLa cells, human osteosarcoma (HOS) cells, rat mesenchymal stem cells (MSCs), and rat smooth muscle A7r5 cells. These cells were cultured and stained daily. The obtained cell images were binarized and clipped into squares containing about 104 cells. These cells showed characteristic cell proliferation patterns. The growth curves of these cells were generated from the cell proliferation images and we determined the doubling time of these cells from the growth curves. We developed a simple cellular automata system with an easily accessible graphical user interface. This system has five variable parameters, namely, initial cell number, doubling time, motility, cell-cell adhesion, and cell-cell contact inhibition (of proliferation). Within these parameters, we obtained initial cell numbers and doubling times experimentally. We set the motility at a constant value because the effect of the parameter for our simulation was restricted. Therefore, we simulated cell proliferation behavior with cell-cell adhesion and cell-cell contact inhibition as variables. By comparing growth curves and proliferation cell images, we succeeded in determining the cell-cell interaction properties of each cell. Simulated HeLa and HOS cells exhibited low cell-cell adhesion and weak cell-cell contact inhibition. Simulated MSCs exhibited high cell-cell adhesion and positive cell-cell contact inhibition. Simulated A7r5 cells exhibited low cell-cell adhesion and strong cell-cell contact inhibition. These simulated results correlated with the experimental growth curves and proliferation images. CONCLUSIONS: Our simulation approach is an easy method for evaluating the cell-cell interaction properties of cells.

Measurement of DNA Length Changes Upon CpG Hypermethylation by Microfluidic Molecular Stretching.

Abnormal DNA methylation in CpG-rich promoters is recognized as a distinct molecular feature of precursor lesions to cancer. Such unintended methylation can occur during in vitro differentiation of stem cells. It takes place in a subset of genes during the differentiation or expansion of stem cell derivatives under general culture conditions, which may need to be monitored in future cell transplantation studies. Here we demonstrate a microfluidic device for investigating morphological length changes in DNA methylation. Arrayed polymer chains of single DNA molecules were fluorescently observed by parallel trapping and stretching in the microfluidic channel. This observational study revealed that the shortened DNA length is due to the increased rigidity of the methylated DNA molecule. The trapping rate of the device for DNA molecules was substantially unaffected by changes in the CpG methylation.

Multispectral Emissions of Lanthanide-Doped Gadolinium Oxide Nanophosphors for Cathodoluminescence and Near-Infrared Upconversion/Downconversion Imaging.

Comprehensive imaging of a biological individual can be achieved by utilizing the variation in spatial resolution, the scale of cathodoluminescence (CL), and near-infrared (NIR), as favored by imaging probe Gd2O3 co-doped lanthanide nanophosphors (NPPs). A series of Gd2O3:Ln3+/Yb3+ (Ln3+: Tm3+, Ho3+, Er3+) NPPs with multispectral emission are prepared by the sol-gel method. The NPPs show a wide range of emissions spanning from the visible to the NIR region under 980 nm excitation. The dependence of the upconverting (UC)/downconverting (DC) emission intensity on the dopant ratio is investigated. The optimum ratios of dopants obtained for emissions in the NIR regions at 810 nm, 1200 nm, and 1530 nm are applied to produce nanoparticles by the homogeneous precipitation (HP) method. The nanoparticles produced from the HP method are used to investigate the dual NIR and CL imaging modalities. The results indicate the possibility of using Gd2O3 co-doped Ln3+/Yb3+ (Ln3+: Tm3+, Ho3+, Er3+) in correlation with NIR and CL imaging. The use of Gd2O3 promises an extension of the object dimension to the whole-body level by employing magnetic resonance imaging (MRI).

Inhibition of cell adhesion by phosphorylated Ezrin/Radixin/Moesin.

Altered phosphorylation status of the C-terminal Thr residues of Ezrin/Radixin/Moesin (ERM) is often linked to cell shape change. To determine the role of phophorylated ERM, we modified phosphorylation status of ERM and investigated changes in cell adhesion and morphology. Treatment with Calyculin-A (Cal-A), a protein phosphatase inhibitor, dramatically augmented phosphorylated ERM (phospho-ERM). Cal-A-treatment or expression of phospho-mimetic Moesin mutant (Moesin-TD) induced cell rounding in adherent cells. Moreover, reattachment of detached cells to substrate was inhibited by either treatment. Phospho-ERM, Moesin-TD and actin cytoskeleton were observed at the plasma membrane of such round cells. Augmented cell surface rigidity was also observed in both cases. Meanwhile, non-adherent KG-1 cells were rather rich in phospho-ERM. Treatment with Staurosporine, a protein kinase inhibitor that dephosphorylates phospho-ERM, up-regulated the integrin-dependent adhesion of KG-1 cells to substrate. These findings strongly suggest the followings: (1) Phospho-ERM inhibit cell adhesion, and therefore, dephosphorylation of ERM proteins is essential for cell adhesion. (2) Phospho-ERM induce formation and/or maintenance of spherical cell shape. (3) ERM are constitutively both phosphorylated and dephosphorylated in cultured adherent and non-adherent cells.

Distinct mechanical behavior of HEK293 cells in adherent and suspended states.

The mechanical features of individual animal cells have been regarded as indicators of cell type and state. Previously, we investigated the surface mechanics of cancer and normal stromal cells in adherent and suspended states using atomic force microscopy. Cancer cells possessed specific mechanical and actin cytoskeleton features that were distinct from normal stromal cells in adherent and suspended states. In this paper, we report the unique mechanical and actin cytoskeletal features of human embryonic kidney HEK293 cells. Unlike normal stromal and cancer cells, the surface stiffness of adherent HEK293 cells was very low, but increased after cell detachment from the culture surface. Induced actin filament depolymerization revealed that the actin cytoskeleton was the underlying source of the stiffness in suspended HEK293 cells. The exclusive mechanical response of HEK293 cells to perturbation of the actin cytoskeleton resembled that of adherent cancer cells and suspended normal stromal cells. Thus, with respect to their special cell-surface mechanical features, HEK293 cells could be categorized into a new class distinct from normal stromal and cancer cells.

Rare-earth-doped nanophosphors for multicolor cathodoluminescence nanobioimaging using scanning transmission electron microscopy.

We describe rare-earth-doped nanophosphors (RE-NPs) for biological imaging using cathodoluminescence(CL) microscopy based on scanning transmission electron microscopy (STEM). We report the first demonstration of multicolor CL nanobioimaging using STEM with nanophosphors. The CL spectra of the synthesized nanophosphors (Y2O3∶Eu, Y2O3∶Tb) were sufficiently narrow to be distinguished. From CL images of RE-NPs on an elastic carbon-coated copper grid, the spatial resolution was beyond the diffraction limit of light.Y2O3∶Tb and Y2O3∶Eu RE-NPs showed a remarkable resistance against electron beam exposure even at high acceleration voltage (80 kV) and retained a CL intensity of more than 97% compared with the initial intensity for 1 min. In biological CL imaging with STEM, heavy-metal-stained cell sections containing the RE-NPs were prepared,and both the CL images of RE-NPs and cellular structures, such as mitochondria, were clearly observed from STEM images with high contrast. The cellular CL imaging using RE-NPs also had high spatial resolution even though heavy-metal-stained cells are normally regarded as highly scattering media. Moreover, since theRE-NPs exhibit photoluminescence (PL) excited by UV light, they are useful for multimodal correlative imaging using CL and PL.