Deep two-way matrix reordering for relational data analysis.

Matrix reordering is a task to permute the rows and columns of a given observed matrix such that the resulting reordered matrix shows meaningful or interpretable structural patterns. Most existing matrix reordering techniques share the common processes of extracting some feature representations from an observed matrix in a predefined manner, and applying matrix reordering based on it. However, in some practical cases, we do not always have prior knowledge about the structural pattern of an observed matrix. To address this problem, we propose a new matrix reordering method, called deep two-way matrix reordering (DeepTMR), using a neural network model. The trained network can automatically extract nonlinear row/column features from an observed matrix, which can then be used for matrix reordering. Moreover, the proposed DeepTMR provides the denoised mean matrix of a given observed matrix as an output of the trained network. This denoised mean matrix can be used to visualize the global structure of the reordered observed matrix. We demonstrate the effectiveness of the proposed DeepTMR by applying it to both synthetic and practical datasets.

Bayesian optimization design for finding a maximum tolerated dose combination in phase I clinical trials.

The development of combination therapies has become commonplace because potential synergistic benefits are expected for resistant patients of single-agent treatment. In phase I clinical trials, the underlying premise is toxicity increases monotonically with increasing dose levels. This assumption cannot be applied in drug combination trials, however, as there are complex drug-drug interactions. Although many parametric model-based designs have been developed, strong assumptions may be inappropriate owing to little information available about dose-toxicity relationships. No standard solution for finding a maximum tolerated dose combination has been established. With these considerations, we propose a Bayesian optimization design for identifying a single maximum tolerated dose combination. Our proposed design utilizing Bayesian optimization guides the next dose by a balance of information between exploration and exploitation on the nonparametrically estimated dose-toxicity function, thereby allowing us to reach a global optimum with fewer evaluations. We evaluate the proposed design by comparing it with a Bayesian optimal interval design and with the partial-ordering continual reassessment method. The simulation results suggest that the proposed design works well in terms of correct selection probabilities and dose allocations. The proposed design has high potential as a powerful tool for use in finding a maximum tolerated dose combination.

Bayesian optimization for estimating the maximum tolerated dose in Phase I clinical trials.

We introduce a Bayesian optimization method for estimating the maximum tolerated dose in this article. A number of parametric model-based methods have been proposed to estimate the maximum tolerated dose; however, parametric model-based methods need an assumption that dose-toxicity relationships follow specific theoretical models. This assumption potentially leads to suboptimal dose selections if the dose-toxicity curve is misspecified. Our proposed method is based on a Bayesian optimization framework for finding a global optimizer of unknown functions that are expensive to evaluate while using very few function evaluations. It models dose-toxicity relationships with a nonparametric model; therefore, a more flexible estimation can be realized compared with existing parametric model-based methods. Also, most existing methods rely on point estimates of dose-toxicity curves in their dose selections. In contrast, our proposed method exploits a probabilistic model for an unknown function to determine the next dose candidate without ignoring the uncertainty of posterior while imposing some dose-escalation limitations. We investigate the operating characteristics of our proposed method by comparing them with those of the Bayesian-based continual reassessment method and two different nonparametric methods. Simulation results suggest that our proposed method works successfully in terms of selections of the maximum tolerated dose correctly and safe dose allocations.

Bayesian optimization design for dose-finding based on toxicity and efficacy outcomes in phase I/II clinical trials.

In phase I trials, the main goal is to identify a maximum tolerated dose under an assumption of monotonicity in dose-response relationships. On the other hand, such monotonicity is no longer applied to biologic agents because a different mode of action from that of cytotoxic agents potentially draws unimodal or flat dose-efficacy curves. Therefore, biologic agents require an optimal dose that provides a sufficient efficacy rate under an acceptable toxicity rate instead of a maximum tolerated dose. Many trials incorporate both toxicity and efficacy data, and drugs with a variety of modes of actions are increasingly being developed; thus, optimal dose estimation designs have been receiving increased attention. Although numerous authors have introduced parametric model-based designs, it is not always appropriate to apply strong assumptions in dose-response relationships. We propose a new design based on a Bayesian optimization framework for identifying optimal doses for biologic agents in phase I/II trials. Our proposed design models dose-response relationships via nonparametric models utilizing a Gaussian process prior, and the uncertainty of estimates is considered in the dose selection process. We compared the operating characteristics of our proposed design against those of three other designs through simulation studies. These include an expansion of Bayesian optimal interval design, the parametric model-based EffTox design, and the isotonic design. In simulations, our proposed design performed well and provided results that were more stable than those from the other designs, in terms of the accuracy of optimal dose estimations and the percentage of correct recommendations.

Independently Interpretable Lasso for Generalized Linear Models.

Sparse regularization such as ℓ1 regularization is a quite powerful and widely used strategy for high-dimensional learning problems. The effectiveness of sparse regularization has been supported practically and theoretically by several studies. However, one of the biggest issues in sparse regularization is that its performance is quite sensitive to correlations between features. Ordinary ℓ1 regularization selects variables correlated with each other under weak regularizations, which results in deterioration of not only its estimation error but also interpretability. In this letter, we propose a new regularization method, independently interpretable lasso (IILasso), for generalized linear models. Our proposed regularizer suppresses selecting correlated variables, so that each active variable affects the response independently in the model. Hence, we can interpret regression coefficients intuitively, and the performance is also improved by avoiding overfitting. We analyze the theoretical property of the IILasso and show that the proposed method is advantageous for its sign recovery and achieves almost minimax optimal convergence rate. Synthetic and real data analyses also indicate the effectiveness of the IILasso.

On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces.

Deep learning has been applied to various tasks in the field of machine learning and has shown superiority to other common procedures such as kernel methods. To provide a better theoretical understanding of the reasons for its success, we discuss the performance of deep learning and other methods on a nonparametric regression problem with a Gaussian noise. Whereas existing theoretical studies of deep learning have been based mainly on mathematical theories of well-known function classes such as Hölder and Besov classes, we focus on function classes with discontinuity and sparsity, which are those naturally assumed in practice. To highlight the effectiveness of deep learning, we compare deep learning with a class of linear estimators representative of a class of shallow estimators. It is shown that the minimax risk of a linear estimator on the convex hull of a target function class does not differ from that of the original target function class. This results in the suboptimality of linear methods over a simple but non-convex function class, on which deep learning can attain nearly the minimax-optimal rate. In addition to this extreme case, we consider function classes with sparse wavelet coefficients. On these function classes, deep learning also attains the minimax rate up to log factors of the sample size, and linear methods are still suboptimal if the assumed sparsity is strong. We also point out that the parameter sharing of deep neural networks can remarkably reduce the complexity of the model in our setting.

Platelet-rich fibrin prepared from stored whole-blood samples.

BACKGROUND: In regenerative therapy, self-clotted platelet concentrates, such as platelet-rich fibrin (PRF), are generally prepared on-site and are immediately used for treatment. If blood samples or prepared clots can be preserved for several days, their clinical applicability will expand. Here, we prepared PRF from stored whole-blood samples and examined their characteristics. METHODS: Blood samples were collected from non-smoking, healthy male donors (aged 27-67 years, N = 6), and PRF clots were prepared immediately or after storage for 1-2 days. Fibrin fiber was examined by scanning electron microscopy. Bioactivity was evaluated by means of a bioassay system involving human periosteal cells, whereas PDGF-BB concentrations were determined by an enzyme-linked immunosorbent assay. RESULTS: Addition of optimal amounts of a 10% CaCl2 solution restored the coagulative ability of whole-blood samples that contained an anticoagulant (acid citrate dextrose) and were stored for up to 2 days at ambient temperature. In PRF clots prepared from the stored whole-blood samples, the thickness and cross-links of fibrin fibers were almost identical to those of freshly prepared PRF clots. PDGF-BB concentrations in the PRF extract were significantly lower in stored whole-blood samples than in fresh samples; however, both extracts had similar stimulatory effects on periosteal-cell proliferation. CONCLUSIONS: Quality of PRF clots prepared from stored whole-blood samples is not reduced significantly and can be ensured for use in regenerative therapy. Therefore, the proposed method enables a more flexible treatment schedule and choice of a more suitable platelet concentrate immediately before treatment, not after blood collection.

An Evaluation of the Accuracy of the Subtraction Method Used for Determining Platelet Counts in Advanced Platelet-Rich Fibrin and Concentrated Growth Factor Preparations.

Platelet concentrates should be quality-assured of purity and identity prior to clinical use. Unlike for the liquid form of platelet-rich plasma, platelet counts cannot be directly determined in solid fibrin clots and are instead calculated by subtracting the counts in other liquid or semi-clotted fractions from those in whole blood samples. Having long suspected the validity of this method, we herein examined the possible loss of platelets in the preparation process. Blood samples collected from healthy male donors were immediately centrifuged for advanced platelet-rich fibrin (A-PRF) and concentrated growth factors (CGF) according to recommended centrifugal protocols. Blood cells in liquid and semi-clotted fractions were directly counted. Platelets aggregated on clot surfaces were observed by scanning electron microscopy. A higher centrifugal force increased the numbers of platelets and platelet aggregates in the liquid red blood cell fraction and the semi-clotted red thrombus in the presence and absence of the anticoagulant, respectively. Nevertheless, the calculated platelet counts in A-PRF/CGF preparations were much higher than expected, rendering the currently accepted subtraction method inaccurate for determining platelet counts in fibrin clots. To ensure the quality of solid types of platelet concentrates chairside in a timely manner, a simple and accurate platelet-counting method should be developed immediately.

Direct learning of sparse changes in Markov networks by density ratio estimation.

We propose a new method for detecting changes in Markov network structure between two sets of samples. Instead of naively fitting two Markov network models separately to the two data sets and figuring out their difference, we directly learn the network structure change by estimating the ratio of Markov network models. This density-ratio formulation naturally allows us to introduce sparsity in the network structure change, which highly contributes to enhancing interpretability. Furthermore, computation of the normalization term, a critical bottleneck of the naive approach, can be remarkably mitigated. We also give the dual formulation of the optimization problem, which further reduces the computation cost for large-scale Markov networks. Through experiments, we demonstrate the usefulness of our method.

Density-difference estimation.

We address the problem of estimating the difference between two probability densities. A naive approach is a two-step procedure of first estimating two densities separately and then computing their difference. However, this procedure does not necessarily work well because the first step is performed without regard to the second step, and thus a small estimation error incurred in the first stage can cause a big error in the second stage. In this letter, we propose a single-shot procedure for directly estimating the density difference without separately estimating two densities. We derive a nonparametric finite-sample error bound for the proposed single-shot density-difference estimator and show that it achieves the optimal convergence rate. We then show how the proposed density-difference estimator can be used in L²-distance approximation. Finally, we experimentally demonstrate the usefulness of the proposed method in robust distribution comparison such as class-prior estimation and change-point detection.

Nonlinear system identification for prostate cancer and optimality of intermittent androgen suppression therapy.

These days prostate cancer is one of the most common types of malignant neoplasm in men. Androgen ablation therapy (hormone therapy) has been shown to be effective for advanced prostate cancer. However, continuous hormone therapy often causes recurrence. This results from the progression of androgen-dependent cancer cells to androgen-independent cancer cells during the continuous hormone therapy. One possible method to prevent the progression to the androgen-independent state is intermittent androgen suppression (IAS) therapy, which ceases dosing intermittently. In this paper, we propose two methods to estimate the dynamics of prostate cancer, and investigate the IAS therapy from the viewpoint of optimality. The two methods that we propose for dynamics estimation are a variational Bayesian method for a piecewise affine (PWA) system and a Gaussian process regression method. We apply the proposed methods to real clinical data and compare their predictive performances. Then, using the estimated dynamics of prostate cancer, we observe how prostate cancer behaves for various dosing schedules. It can be seen that the conventional IAS therapy is a way of imposing high cost for dosing while keeping the prostate cancer in a safe state. We would like to dedicate this paper to the memory of Professor Luigi M. Ricciardi.

Relative density-ratio estimation for robust distribution comparison.

Divergence estimators based on direct approximation of density ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity test. However, since density-ratio functions often possess high fluctuation, divergence estimation is a challenging task in practice. In this letter, we use relative divergences for distribution comparison, which involves approximation of relative density ratios. Since relative density ratios are always smoother than corresponding ordinary density ratios, our proposed method is favorable in terms of nonparametric convergence speed. Furthermore, we show that the proposed divergence estimator has asymptotic variance independent of the model complexity under a parametric setup, implying that the proposed estimator hardly overfits even with complex models. Through experiments, we demonstrate the usefulness of the proposed approach.

Sufficient dimension reduction via squared-loss mutual information estimation.

The goal of sufficient dimension reduction in supervised learning is to find the low-dimensional subspace of input features that contains all of the information about the output values that the input features possess. In this letter, we propose a novel sufficient dimension-reduction method using a squared-loss variant of mutual information as a dependency measure. We apply a density-ratio estimator for approximating squared-loss mutual information that is formulated as a minimum contrast estimator on parametric or nonparametric models. Since cross-validation is available for choosing an appropriate model, our method does not require any prespecified structure on the underlying distributions. We elucidate the asymptotic bias of our estimator on parametric models and the asymptotic convergence rate on nonparametric models. The convergence analysis utilizes the uniform tail-bound of a U-process, and the convergence rate is characterized by the bracketing entropy of the model. We then develop a natural gradient algorithm on the Grassmann manifold for sufficient subspace search. The analytic formula of our estimator allows us to compute the gradient efficiently. Numerical experiments show that the proposed method compares favorably with existing dimension-reduction approaches on artificial and benchmark data sets.

Least-squares two-sample test.

The goal of the two-sample test (a.k.a. the homogeneity test) is, given two sets of samples, to judge whether the probability distributions behind the samples are the same or not. In this paper, we propose a novel non-parametric method of two-sample test based on a least-squares density ratio estimator. Through various experiments, we show that the proposed method overall produces smaller type-II error (i.e., the probability of judging the two distributions to be the same when they are actually different) than a state-of-the-art method, with slightly larger type-I error (i.e., the probability of judging the two distributions to be different when they are actually the same).

Least-squares independent component analysis.

Accurately evaluating statistical independence among random variables is a key element of independent component analysis (ICA). In this letter, we employ a squared-loss variant of mutual information as an independence measure and give its estimation method. Our basic idea is to estimate the ratio of probability densities directly without going through density estimation, thereby avoiding the difficult task of density estimation. In this density ratio approach, a natural cross-validation procedure is available for hyperparameter selection. Thus, all tuning parameters such as the kernel width or the regularization parameter can be objectively optimized. This is an advantage over recently developed kernel-based independence measures and is a highly useful property in unsupervised learning problems such as ICA. Based on this novel independence measure, we develop an ICA algorithm, named least-squares independent component analysis.

Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search.

Methods for directly estimating the ratio of two probability density functions have been actively explored recently since they can be used for various data processing tasks such as non-stationarity adaptation, outlier detection, and feature selection. In this paper, we develop a new method which incorporates dimensionality reduction into a direct density-ratio estimation procedure. Our key idea is to find a low-dimensional subspace in which densities are significantly different and perform density-ratio estimation only in this subspace. The proposed method, D(3)-LHSS (Direct Density-ratio estimation with Dimensionality reduction via Least-squares Hetero-distributional Subspace Search), is shown to overcome the limitation of baseline methods.

Piecewise affine systems modelling for optimizing hormone therapy of prostate cancer.

Prostate cancer is one of the most common malignant neoplasms in men with an overall incidence of approximately 15 per cent during the normal life span. Androgen-deprivation therapy (hormone therapy) is an effective treatment of this disease when progressed to an advanced stage. Despite impressive responses, such treatment when applied on a continuous basis is not curative and eventually culminates in androgen-independent disease. On the other hand, intermittent androgen suppression (IAS) was first conceived as a potential way of delaying progression to androgen-independence, in addition offering the possibility of reducing adverse effects and improving the quality of life. Although the validity of this approach has been confirmed in several clinical studies, the optimal scheduling of the cycles of on- and off-treatment remains to be explored. In the present article, we show that IAS lends itself to mathematical modelling with hybrid dynamical systems and that the model we have developed can be used to select the best strategy for keeping prostate cancer in an androgen-dependent state as long as possible. Our results also suggest that the current way of using IAS exceeds what is necessary for optimal control; in fact, we have found that to achieve optimal control, the amount of therapy (dose and duration of drugs) can be reduced by a factor of one half.

Mutual information estimation reveals global associations between stimuli and biological processes.

BACKGROUND: Although microarray gene expression analysis has become popular, it remains difficult to interpret the biological changes caused by stimuli or variation of conditions. Clustering of genes and associating each group with biological functions are often used methods. However, such methods only detect partial changes within cell processes. Herein, we propose a method for discovering global changes within a cell by associating observed conditions of gene expression with gene functions. RESULTS: To elucidate the association, we introduce a novel feature selection method called Least-Squares Mutual Information (LSMI), which computes mutual information without density estimaion, and therefore LSMI can detect nonlinear associations within a cell. We demonstrate the effectiveness of LSMI through comparison with existing methods. The results of the application to yeast microarray datasets reveal that non-natural stimuli affect various biological processes, whereas others are no significant relation to specific cell processes. Furthermore, we discover that biological processes can be categorized into four types according to the responses of various stimuli: DNA/RNA metabolism, gene expression, protein metabolism, and protein localization. CONCLUSION: We proposed a novel feature selection method called LSMI, and applied LSMI to mining the association between conditions of yeast and biological processes through microarray datasets. In fact, LSMI allows us to elucidate the global organization of cellular process control.