Deep Learning Is Singular, and That's Good.

In singular models, the optimal set of parameters forms an analytic set with singularities, and a classical statistical inference cannot be applied to such models. This is significant for deep learning as neural networks are singular, and thus, "dividing" by the determinant of the Hessian or employing the Laplace approximation is not appropriate. Despite its potential for addressing fundamental issues in deep learning, a singular learning theory appears to have made little inroads into the developing canon of a deep learning theory. Via a mix of theory and experiment, we present an invitation to the singular learning theory as a vehicle for understanding deep learning and suggest an important future work to make the singular learning theory directly applicable to how deep learning is performed in practice.

Trade-off between conservation of biological variation and batch effect removal in deep generative modeling for single-cell transcriptomics.

BACKGROUND: Single-cell RNA sequencing (scRNA-seq) technology has contributed significantly to diverse research areas in biology, from cancer to development. Since scRNA-seq data is high-dimensional, a common strategy is to learn low-dimensional latent representations better to understand overall structure in the data. In this work, we build upon scVI, a powerful deep generative model which can learn biologically meaningful latent representations, but which has limited explicit control of batch effects. Rather than prioritizing batch effect removal over conservation of biological variation, or vice versa, our goal is to provide a bird's eye view of the trade-offs between these two conflicting objectives. Specifically, using the well established concept of Pareto front from economics and engineering, we seek to learn the entire trade-off curve between conservation of biological variation and removal of batch effects. RESULTS: A multi-objective optimisation technique known as Pareto multi-task learning (Pareto MTL) is used to obtain the Pareto front between conservation of biological variation and batch effect removal. Our results indicate Pareto MTL can obtain a better Pareto front than the naive scalarization approach typically encountered in the literature. In addition, we propose to measure batch effect by applying a neural-network based estimator called Mutual Information Neural Estimation (MINE) and show benefits over the more standard maximum mean discrepancy measure. CONCLUSION: The Pareto front between conservation of biological variation and batch effect removal is a valuable tool for researchers in computational biology. Our results demonstrate the efficacy of applying Pareto MTL to estimate the Pareto front in conjunction with applying MINE to measure the batch effect.

Automating the assessment of biofouling in images using expert agreement as a gold standard.

Biofouling is the accumulation of organisms on surfaces immersed in water. It is of particular concern to the international shipping industry because it increases fuel costs and presents a biosecurity risk by providing a pathway for non-indigenous marine species to establish in new areas. There is growing interest within jurisdictions to strengthen biofouling risk-management regulations, but it is expensive to conduct in-water inspections and assess the collected data to determine the biofouling state of vessel hulls. Machine learning is well suited to tackle the latter challenge, and here we apply deep learning to automate the classification of images from in-water inspections to identify the presence and severity of fouling. We combined several datasets to obtain over 10,000 images collected from in-water surveys which were annotated by a group biofouling experts. We compared the annotations from three experts on a 120-sample subset of these images, and found that they showed 89% agreement (95% CI: 87-92%). Subsequent labelling of the whole dataset by one of these experts achieved similar levels of agreement with this group of experts, which we defined as performing at most 5% worse (p [Formula: see text] 0.009-0.054). Using these expert labels, we were able to train a deep learning model that also agreed similarly with the group of experts (p [Formula: see text] 0.001-0.014), demonstrating that automated analysis of biofouling in images is feasible and effective using this method.

The Change-Plane Cox Model.

We propose a projection pursuit technique in survival analysis for finding lower-dimensional projections that exhibit differentiated survival outcome. This idea is formally introduced as the change-plane Cox model, a non-regular Cox model with a change-plane in the covariate space dividing the population into two subgroups whose hazards are proportional. The proposed technique offers a potential framework for principled subgroup discovery. Estimation of the change-plane is accomplished via likelihood maximization over a data-driven sieve constructed using sliced inverse regression. Consistency of the sieve procedure for the change-plane parameters is established. In simulations the sieve estimator demonstrates better classification performance for subgroup identification than alternatives.

A Low-Cost, Grab-and-Go Breakfast Intervention for Rural High School Students: Changes in School Breakfast Program Participation Among At-Risk Students in Minnesota.

OBJECTIVE: Evaluate the impact of a grab-and-go component embedded within a larger intervention designed to promote School Breakfast Program (SBP) participation. DESIGN: Secondary data analysis. SETTING: Rural Minnesota high schools. PARTICIPANTS: Eight schools were enrolled in the grab-and-go only intervention component. An at-risk sample of students (n = 364) who reported eating breakfast ≤3 d/wk at baseline was enrolled at these schools. INTERVENTIONS: Grab-and-go style breakfast carts and policies were introduced to allow all students to eat outside the cafeteria. MAIN OUTCOME MEASURES: Administrative records were used to determine percent SBP participation (proportion of non-absent days on which fully reimbursable meals were received) for each student and school-level averages. ANALYSIS: Linear mixed models. RESULTS: School-level increases in SBP participation from baseline to the school year of intervention implementation were observed for schools enrolled in the grab-and-go only component (13.0% to 22.6%). Student-level increases in SBP participation were observed among the at-risk sample (7.6% to 21.9%) and among subgroups defined by free- or reduced-price meal eligibility and ethnic or racial background. Participation in SBP increased among students eligible for free or reduced-price meals from 13.9% to 30.7% and among ineligible students from 4.3% to 17.2%. CONCLUSIONS AND IMPLICATIONS: Increasing access to the SBP and social support for eating breakfast are effective promotion strategies.

Control of Blood Glucose for Type-1 Diabetes by Using Reinforcement Learning with Feedforward Algorithm.

BACKGROUND: Type-1 diabetes is a condition caused by the lack of insulin hormone, which leads to an excessive increase in blood glucose level. The glucose kinetics process is difficult to control due to its complex and nonlinear nature and with state variables that are difficult to measure. METHODS: This paper proposes a method for automatically calculating the basal and bolus insulin doses for patients with type-1 diabetes using reinforcement learning with feedforward controller. The algorithm is designed to keep the blood glucose stable and directly compensate for the external events such as food intake. Its performance was assessed using simulation on a blood glucose model. The usage of the Kalman filter with the controller was demonstrated to estimate unmeasurable state variables. RESULTS: Comparison simulations between the proposed controller with the optimal reinforcement learning and the proportional-integral-derivative controller show that the proposed methodology has the best performance in regulating the fluctuation of the blood glucose. The proposed controller also improved the blood glucose responses and prevented hypoglycemia condition. Simulation of the control system in different uncertain conditions provided insights on how the inaccuracies of carbohydrate counting and meal-time reporting affect the performance of the control system. CONCLUSION: The proposed controller is an effective tool for reducing postmeal blood glucose rise and for countering the effects of external known events such as meal intake and maintaining blood glucose at a healthy level under uncertainties.

School Breakfast Program Participation and Rural Adolescents' Purchasing Behaviors in Food Stores and Restaurants.

BACKGROUND: Little is known about adolescents' food purchasing behaviors in rural areas. This study examined whether purchasing food at stores/restaurants around schools was related to adolescents' participation in school breakfast programs and overall diet in rural Minnesota. METHODS: Breakfast-skippers enrolled in a group-randomized intervention in 2014 to 2015 (N = 404 from 8 schools) completed 24-hour dietary recalls and pre/post surveys assessing food establishment purchase frequency. Healthy Eating Index Scores (HEI-2010) were calculated for each student. Student-level school breakfast participation (SBP) was obtained from school food service records. Mixed-effects regression models estimated: (1) whether SBP was associated with store/restaurant use at baseline, (2) whether an increase in SBP was associated with a decrease in store/restaurant use, and (3) whether stores/restaurant use was associated with HEI-2010 scores at baseline. RESULTS: Students with increased SBP were more likely to decrease fast-food restaurant purchases on the way home from school (OR 1.017, 95% CI 1.005, 1.029), but were less likely to decrease purchases at food stores for breakfast (OR 0.979, 95% CI 0.959, 0.999). Food establishment use was associated with lower HEI-2010 dairy component scores (p = .017). CONCLUSIONS: Increasing participation in school breakfast may result in modest changes in purchases at food establishments.

Latent Supervised Learning.

A new machine learning task is introduced, called latent supervised learning, where the goal is to learn a binary classifier from continuous training labels which serve as surrogates for the unobserved class labels. A specific model is investigated where the surrogate variable arises from a two-component Gaussian mixture with unknown means and variances, and the component membership is determined by a hyperplane in the covariate space. The estimation of the separating hyperplane and the Gaussian mixture parameters forms what shall be referred to as the change-line classification problem. A data-driven sieve maximum likelihood estimator for the hyperplane is proposed, which in turn can be used to estimate the parameters of the Gaussian mixture. The estimator is shown to be consistent. Simulations as well as empirical data show the estimator has high classification accuracy.

Image and statistical analysis of melanocytic histology.

AIMS: We applied digital image analysis techniques to study selected types of melanocytic lesions. METHODS AND RESULTS: We used advanced digital image analysis to compare melanocytic lesions as follows: (i) melanoma to nevi, (ii) melanoma subtypes to nevi, (iii) severely dysplastic nevi to other nevi and (iv) melanoma to severely dysplastic nevi. We were successful in differentiating melanoma from nevi [receiver operating characteristic area (ROC) 0.95] using image-derived features, among which those related to nuclear size and shape and distance between nuclei were most important. Dividing melanoma into subtypes, even greater separation was obtained (ROC area 0.98 for superficial spreading melanoma; 0.95 for lentigo maligna melanoma; and 0.99 for unclassified). Severely dysplastic nevi were best differentiated from conventional and mildly dysplastic nevi by differences in cellular staining qualities (ROC area 0.84). We found that melanomas were separated from severely dysplastic nevi by features related to shape and staining qualities (ROC area 0.95). All comparisons were statistically significant (P < 0.0001). CONCLUSIONS: We offer a unique perspective into the evaluation of melanocytic lesions and demonstrate a technological application with increasing prevalence, and with potential use as an adjunct to traditional diagnosis in the future.

Prognostic DNA methylation biomarkers in ovarian cancer.

PURPOSE: Aberrant DNA methylation, now recognized as a contributing factor to neoplasia, often shows definitive gene/sequence preferences unique to specific cancer types. Correspondingly, distinct combinations of methylated loci can function as biomarkers for numerous clinical correlates of ovarian and other cancers. EXPERIMENTAL DESIGN: We used a microarray approach to identify methylated loci prognostic for reduced progression-free survival (PFS) in advanced ovarian cancer patients. Two data set classification algorithms, Significance Analysis of Microarray and Prediction Analysis of Microarray, successfully identified 220 candidate PFS-discriminatory methylated loci. Of those, 112 were found capable of predicting PFS with 95% accuracy, by Prediction Analysis of Microarray, using an independent set of 40 advanced ovarian tumors (from 20 short-PFS and 20 long-PFS patients, respectively). Additionally, we showed the use of these predictive loci using two bioinformatics machine-learning algorithms, Support Vector Machine and Multilayer Perceptron. CONCLUSION: In this report, we show that highly prognostic DNA methylation biomarkers can be successfully identified and characterized, using previously unused, rigorous classifying algorithms. Such ovarian cancer biomarkers represent a promising approach for the assessment and management of this devastating disease.

Hypermethylation of 18S and 28S ribosomal DNAs predicts progression-free survival in patients with ovarian cancer.

PURPOSE: Repetitive ribosomal DNA (rDNA) genes are GC-rich clusters in the human genome. The aim of the study was to determine the methylation status of two rDNA subunits, the 18S and 28S genes, in ovarian tumors and to correlate methylation levels with clinicopathologic features in a cohort of ovarian cancer patients. EXPERIMENTAL DESIGN: 18S and 28S rDNA methylation was examined by quantitative methylation-specific PCR in 74 late-stage ovarian cancers, 9 histologically uninvolved, and 11 normal ovarian surface epithelial samples. In addition, methylation and gene expression levels of 18S and 28S rDNAs in two ovarian cancer cell lines were examined by reverse transcription-PCR before and after treatment with the demethylating drug 5'-aza-2'-deoxycytidine. RESULTS: The methylation level (amount of methylated rDNA/beta-actin) of 18S and 28S rDNAs was significantly higher (P < 0.05) in tumors than in normal ovarian surface epithelial samples. Methylation of 18S and 28S rDNA was highly correlated (R2= 0.842). Multivariate analysis by Cox regression found that rDNA hypermethylation [hazard ratio (HR), 0.25; P < 0.01], but not age (HR, 1.29; P = 0.291) and stage (HR, 1.09; P = 0.709), was independently associated with longer progression-free survival. In ovarian cancer cell lines, methylation levels of rDNA correlated with gene down-regulation and 5'-aza-2'-deoxycytidine treatment resulted in a moderate increase in 18S and 28S rDNA gene expressions. CONCLUSION: This is the first report of rDNA hypermethylation in ovarian tumors. Furthermore, rDNA methylation levels were higher in patients with long progression-free survival versus patients with short survival. Thus, rDNA methylation as a prognostic marker in ovarian cancer warrants further investigation.

Loss of estrogen receptor signaling triggers epigenetic silencing of downstream targets in breast cancer.

Alterations in histones, chromatin-related proteins, and DNA methylation contribute to transcriptional silencing in cancer, but the sequence of these molecular events is not well understood. Here we demonstrate that on disruption of estrogen receptor (ER) alpha signaling by small interfering RNA, polycomb repressors and histone deacetylases are recruited to initiate stable repression of the progesterone receptor (PR) gene, a known ERalpha target, in breast cancer cells. The event is accompanied by acquired DNA methylation of the PR promoter, leaving a stable mark that can be inherited by cancer cell progeny. Reestablishing ERalpha signaling alone was not sufficient to reactivate the PR gene; reactivation of the PR gene also requires DNA demethylation. Methylation microarray analysis further showed that progressive DNA methylation occurs in multiple ERalpha targets in breast cancer genomes. The results imply, for the first time, the significance of epigenetic regulation on ERalpha target genes, providing new direction for research in this classical signaling pathway.

Methylation-specific oligonucleotide microarray.

The methylation-specific oligonucleotide (MSO) microarray is a high-throughput approach capable of detecting DNA methylation in genes across several CpG sites. Based on the bisulfite modification of DNA that converts unmethylated cytosines to uracil but leaves the 5'methylcytosine intact, the method utilizes short oligonucleotides corresponding to the methylated and unmethylated alleles as probes affixed on solid support and products amplified from bisulfite-treated DNA as targets for hybridization. MSO is suitable for examining a panel of genes across multiple clinical samples. This approach can generate a robust dataset for discovering profiles of gene methylation in cancer with aberrant DNA methylation in the neoplastic genome and widespread hypermethylation in tumor suppressor genes. MSO and other oligonucleotide-based arrays have been applied successfully for analyses of single genes and have been useful in delineating and predicting tumor subgroups using clustering methods. Here we focus on design criteria important to the interrogation of multiple CpG sites across several genes.

Methyl-CpG binding proteins identify novel sites of epigenetic inactivation in human cancer.

Methyl-CpG binding proteins (MBDs) mediate histone deacetylase-dependent transcriptional silencing at methylated CpG islands. Using chromatin immunoprecitation (ChIP) we have found that gene-specific profiles of MBDs exist for hypermethylated promoters of breast cancer cells, whilst a common pattern of histone modifications is shared. This unique distribution of MBDs is also characterized in chromosomes by comparative genomic hybridization of immunoprecipitated DNA and immunolocalization. Most importantly, we demonstrate that MBD association to methylated DNA serves to identify novel targets of epigenetic inactivation in human cancer. We combined the ChIP assay of MBDs with a CpG island microarray (ChIP on chip). The scenario revealed shows that, while many genes are regulated by multiple MBDs, others are associated with a single MBD. These target genes displayed methylation- associated transcriptional silencing in breast cancer cells and primary tumours. The candidates include the homeobox gene PAX6, the prolactin hormone receptor, and dipeptidylpeptidase IV among others. Our results support an essential role for MBDs in gene silencing and, when combined with genomic strategies, their potential to 'catch' new hypermethylated genes in cancer.

Double RNA interference of DNMT3b and DNMT1 enhances DNA demethylation and gene reactivation.

Small interfering RNAs (siRNAs) are newly identified molecules shown to silence genes via targeted mRNA degradation. In this study, we used specific siRNAs as a tool to probe the relationship between two DNA methyltransferase genes, DNMT3b and DNMT1, in the maintenance of DNA methylation patterns in the genome. Levels of DNMT3b or DNMT1 mRNAs and proteins were markedly decreased (up to 80%) on transfecting these siRNAs into the ovarian cancer cell line CP70. The resulting RNA interference showed differential effects on DNA demethylation and gene reactivation in the treated cells. The DNMT1 siRNA treatment led to a partial removal of DNA methylation from three inactive promoter CpG islands, TWIST, RASSF1A, and HIN-1, and restored the expression of these genes. This epigenetic alteration appeared less effective in cells transfected with DNMT3b siRNA. However, the combined treatment of DNMT3b and DNMT1 siRNAs greatly enhanced this demethylation effect, producing 7-15-fold increases in their expression. We also used a microarray approach to examine this RNA interference on 8640 CpG island loci in CP70 cells. The combined siRNA treatment had a greater demethylation effect on 241 methylated loci and selected repetitive sequences than that of the single treatment. Our data thus suggest that whereas DNMT1 plays a key role in methylation maintenance, DNMT3b may act as an accessory to support the function in CP70 cells. This study also shows that siRNA is a powerful tool for interrogating the mechanisms of DNA methylation in normal and pathological genomes.

Genetic unmasking of epigenetically silenced tumor suppressor genes in colon cancer cells deficient in DNA methyltransferases.

Hypermethylation associated silencing of the CpG islands of tumor suppressor genes is a common hallmark of human cancer. Here we report a functional search for hypermethylated CpG islands using the colorectal cancer cell line HCT-116, in which two major DNA methyltransferases, DNMT1 and DNMT3b, have been genetically disrupted (DKO cells). Using two molecular screenings for differentially methylated loci [differential methylation hybridization (DMH) and amplification of inter-methylated sites (AIMS)], we found that DKO cells, but not the single DNMT1 or DNMT3b knockouts, have a massive loss of hypermethylated CpG islands that induces the re-activation of the contiguous genes. We have characterized a substantial number of these CpG island associated genes with potentially important roles in tumorigenesis, such as the cadherin member FAT, or the homeobox genes LMX-1 and DUX-4. For other genes whose role in transformation has not been characterized, such as the calcium channel alpha1I or the thromboxane A2 receptor, their re-introduction in DKO cells inhibited colony formation. Thus, our results demonstrate the role of DNMT1 and DNMT3b in CpG island methylation associated silencing and the usefulness of genetic disruption strategies in searching for new hypermethylated loci.

Methylation target array for rapid analysis of CpG island hypermethylation in multiple tissue genomes.

Hypermethylation of multiple CpG islands is a common event in cancer. To assess the prognostic values of this epigenetic alteration, we developed Methylation Target Array (MTA), derived from the concept of tissue microarray, for simultaneous analysis of DNA hypermethylation in hundreds of tissue genomes. In MTA, linker-ligated CpG island fragments were digested with methylation-sensitive endonucleases and amplified with flanking primers. A panel of 468 MTA amplicons, which represented the whole repertoire of methylated CpG islands in 93 breast tumors, 20 normal breast tissues, and 4 breast cancer cell lines, were arrayed on nylon membrane for probe hybridization. Positive hybridization signals detected in tumor amplicons, but not in normal amplicons, were indicative of aberrant hypermethylation in tumor samples. This is attributed to aberrant sites that were protected from methylation-sensitive restriction and were amplified by PCR in tumor samples, while the same sites were restricted and could not be amplified in normal samples. Hypermethylation frequencies of the 10 genes tested in breast tumors and cancer cell lines were 60% for GPC3, 58% for RASSF1A, 32% for 3OST3B, 30% for HOXA5, 28% for uPA, 25% for WT1, 23% for BRCA1, 9% for DAPK1, and 0% for KL. Furthermore, hypermethylation of 5 to 7 loci of these genes was significantly correlated with hormone receptor status, clinical stages, and ages at diagnosis of the patients analyzed. This novel approach thus provides an additional avenue for assessing clinicopathological consequences of DNA hypermethylation in breast cancer.

Aberrant DNA methylation in ovarian cancer: is there an epigenetic predisposition to drug response?

Epigenetic regulation of gene expression has been observed in a variety of tumor types. We have used microarray technology to evaluate the predisposition of drug response by aberrant methylation in ovarian cancer. Results indicate that loss of gene activity due to hypermethylation potentially confers a predisposition in certain cancer types and is an early event in disease progression. Methylation profiles of ovarian cancer might be useful for early cancer detection and prediction of chemotherapy outcome in a clinical context.

Triple analysis of the cancer epigenome: an integrated microarray system for assessing gene expression, DNA methylation, and histone acetylation.

We developed a novel microarray system to assess gene expression, DNA methylation, and histone acetylation in parallel, and to dissect the complex hierarchy of epigenetic changes in cancer. An integrated microarray panel consisting of 1507 short CpG island tags located at the 5'-end regions (including the first exons) was used to assess effects of epigenetic treatments on a human epithelial ovarian cancer cell line. Treatment with methylation (5-aza-2'-deoxycytidine) or deacetylation (trichostatin A) inhibitors alone resulted in up-regulation of 1.9 or 1.1% of the genes analyzed; however, the combined treatment resulted in synergistic reactivation of more genes (10.4%; P < 0.001 versus either treatment alone). On the basis of either primary or secondary responses to the treatments, genes were identified as methylation-dependent or -independent. Synergistic reactivation of the methylation-dependent genes by 5-aza-2'-deoxycytidine plus trichostatin A revealed a functional interaction between methylated promoters and deacetylated histones. Increased expression of some methylation-independent genes was associated with enhanced histone acetylation, but up-regulation of most of the genes identified using this technology was because of events downstream of the epigenetic cascade. We demonstrate proof of principle for using the triple microarray system in analyzing the dynamic relationship between transcription factors and promoter targets in cancer genomes.