Mixture Complexity and Its Application to Gradual Clustering Change Detection.

We consider measuring the number of clusters (cluster size) in the finite mixture models for interpreting their structures. Many existing information criteria have been applied for this issue by regarding it as the same as the number of mixture components (mixture size); however, this may not be valid in the presence of overlaps or weight biases. In this study, we argue that the cluster size should be measured as a continuous value and propose a new criterion called mixture complexity (MC) to formulate it. It is formally defined from the viewpoint of information theory and can be seen as a natural extension of the cluster size considering overlap and weight bias. Subsequently, we apply MC to the issue of gradual clustering change detection. Conventionally, clustering changes have been regarded as abrupt, induced by the changes in the mixture size or cluster size. Meanwhile, we consider the clustering changes to be gradual in terms of MC; it has the benefits of finding the changes earlier and discerning the significant and insignificant changes. We further demonstrate that the MC can be decomposed according to the hierarchical structures of the mixture models; it helps us to analyze the detail of substructures.

Summarizing Finite Mixture Model with Overlapping Quantification.

Finite mixture models are widely used for modeling and clustering data. When they are used for clustering, they are often interpreted by regarding each component as one cluster. However, this assumption may be invalid when the components overlap. It leads to the issue of analyzing such overlaps to correctly understand the models. The primary purpose of this paper is to establish a theoretical framework for interpreting the overlapping mixture models by estimating how they overlap, using measures of information such as entropy and mutual information. This is achieved by merging components to regard multiple components as one cluster and summarizing the merging results. First, we propose three conditions that any merging criterion should satisfy. Then, we investigate whether several existing merging criteria satisfy the conditions and modify them to fulfill more conditions. Second, we propose a novel concept named clustering summarization to evaluate the merging results. In it, we can quantify how overlapped and biased the clusters are, using mutual information-based criteria. Using artificial and real datasets, we empirically demonstrate that our methods of modifying criteria and summarizing results are effective for understanding the cluster structures. We therefore give a new view of interpretability/explainability for model-based clustering.

Predicting 10-2 Visual Field From Optical Coherence Tomography in Glaucoma Using Deep Learning Corrected With 24-2/30-2 Visual Field.

Purpose: To investigate whether a correction based on a Humphrey field analyzer (HFA) 24-2/30-2 visual field (VF) can improve the prediction performance of a deep learning model to predict the HFA 10-2 VF test from macular optical coherence tomography (OCT) measurements. Methods: This is a multicenter, cross-sectional study. The training dataset comprised 493 eyes of 285 subjects (407, open-angle glaucoma [OAG]; 86, normative) who underwent HFA 10-2 testing and macular OCT. The independent testing dataset comprised 104 OAG eyes of 82 subjects who had undergone HFA 10-2 test, HFA 24-2/30-2 test, and macular OCT. A convolutional neural network (CNN) DL model was trained to predict threshold sensitivity (TH) values in HFA 10-2 from retinal thickness measured by macular OCT. The predicted TH values was modified by pattern-based regularization (PBR) and corrected with HFA 24-2/30-2. Absolute error (AE) of mean TH values and mean absolute error (MAE) of TH values were compared between the CNN-PBR alone model and the CNN-PBR corrected with HFA 24-2/30-2. Results: AE of mean TH values was lower in the CNN-PBR with HFA 24-2/30-2 correction than in the CNN-PBR alone (1.9dB vs. 2.6dB; P = 0.006). MAE of TH values was lower in the CNN-PBR with correction compared to the CNN-PBR alone (4.2dB vs. 5.3 dB; P < 0.001). The inferior temporal quadrant showed lower prediction errors compared with other quadrants. Conclusions: The performance of a DL model to predict 10-2 VF from macular OCT was improved by the correction with HFA 24-2/30-2. Translational Relevance: This model can reduce the burden of additional HFA 10-2 by making the best use of routinely performed HFA 24-2/30-2 and macular OCT.

Change sign detection with differential MDL change statistics and its applications to COVID-19 pandemic analysis.

We are concerned with the issue of detecting changes and their signs from a data stream. For example, when given time series of COVID-19 cases in a region, we may raise early warning signals of an epidemic by detecting signs of changes in the data. We propose a novel methodology to address this issue. The key idea is to employ a new information-theoretic notion, which we call the differential minimum description length change statistics (D-MDL), for measuring the scores of change sign. We first give a fundamental theory for D-MDL. We then demonstrate its effectiveness using synthetic datasets. We apply it to detecting early warning signals of the COVID-19 epidemic using time series of the cases for individual countries. We empirically demonstrate that D-MDL is able to raise early warning signals of events such as significant increase/decrease of cases. Remarkably, for about [Formula: see text] of the events of significant increase of cases in studied countries, our method can detect warning signals as early as nearly six days on average before the events, buying considerably long time for making responses. We further relate the warning signals to the dynamics of the basic reproduction number R0 and the timing of social distancing. The results show that our method is a promising approach to the epidemic analysis from a data science viewpoint.

Word2vec Skip-Gram Dimensionality Selection via Sequential Normalized Maximum Likelihood.

In this paper, we propose a novel information criteria-based approach to select the dimensionality of the word2vec Skip-gram (SG). From the perspective of the probability theory, SG is considered as an implicit probability distribution estimation under the assumption that there exists a true contextual distribution among words. Therefore, we apply information criteria with the aim of selecting the best dimensionality so that the corresponding model can be as close as possible to the true distribution. We examine the following information criteria for the dimensionality selection problem: the Akaike's Information Criterion (AIC), Bayesian Information Criterion (BIC), and Sequential Normalized Maximum Likelihood (SNML) criterion. SNML is the total codelength required for the sequential encoding of a data sequence on the basis of the minimum description length. The proposed approach is applied to both the original SG model and the SG Negative Sampling model to clarify the idea of using information criteria. Additionally, as the original SNML suffers from computational disadvantages, we introduce novel heuristics for its efficient computation. Moreover, we empirically demonstrate that SNML outperforms both BIC and AIC. In comparison with other evaluation methods for word embedding, the dimensionality selected by SNML is significantly closer to the optimal dimensionality obtained by word analogy or word similarity tasks.

A Joint Multitask Learning Model for Cross-sectional and Longitudinal Predictions of Visual Field Using OCT.

Purpose: We constructed a multitask learning model (latent space linear regression and deep learning [LSLR-DL]) in which the 2 tasks of cross-sectional predictions (using OCT) of visual field (VF; central 10°) and longitudinal progression predictions of VF (30°) were performed jointly via sharing the deep learning (DL) component such that information from both tasks was used in an auxiliary manner (The Association for Computing Machinery's Special Interest Group on Knowledge Discovery and Data Mining [SIGKDD] 2021). The purpose of the current study was to investigate the prediction accuracy preparing an independent validation dataset. Design: Cohort study. Participants: Cross-sectional training and testing data sets included the VF (Humphrey Field Analyzer [HFA] 10-2 test) and an OCT measurement (obtained within 6 months) from 591 eyes of 351 healthy people or patients with open-angle glaucoma (OAG) and from 155 eyes of 131 patients with OAG, respectively. Longitudinal training and testing data sets included 7984 VF results (HFA 24-2 test) from 998 eyes of 592 patients with OAG and 1184 VF results (HFA 24-2 test) from 148 eyes of 84 patients with OAG, respectively. Each eye had 8 VF test results (HFA 24-2 test). The OCT sequences within the observation period were used. Methods: Root mean square error (RMSE) was used to evaluate the accuracy of LSLR-DL for the cross-sectional prediction of VF (HFA 10-2 test). For the longitudinal prediction, the final (eighth) VF test (HFA 24-2 test) was predicted using a shorter VF series and relevant OCT images, and the RMSE was calculated. For comparison, RMSE values were calculated by applying the DL component (cross-sectional prediction) and the ordinary pointwise linear regression (longitudinal prediction). Main Outcome Measures: Root mean square error in the cross-sectional and longitudinal predictions. Results: Using LSLR-DL, the mean RMSE in the cross-sectional prediction was 6.4 dB and was between 4.4 dB (VF tests 1 and 2) and 3.7 dB (VF tests 1-7) in the longitudinal prediction, indicating that LSLR-DL significantly outperformed other methods. Conclusions: The results of this study indicate that LSLR-DL is useful for both the cross-sectional prediction of VF (HFA 10-2 test) and the longitudinal progression prediction of VF (HFA 24-2 test).

Deep learning model to predict visual field in central 10° from optical coherence tomography measurement in glaucoma.

BACKGROUND/AIM: To train and validate the prediction performance of the deep learning (DL) model to predict visual field (VF) in central 10° from spectral domain optical coherence tomography (SD-OCT). METHODS: This multicentre, cross-sectional study included paired Humphrey field analyser (HFA) 10-2 VF and SD-OCT measurements from 591 eyes of 347 patients with open-angle glaucoma (OAG) or normal subjects for the training data set. We trained a convolutional neural network (CNN) for predicting VF threshold (TH) sensitivity values from the thickness of the three macular layers: retinal nerve fibre layer, ganglion cell layer+inner plexiform layer and outer segment+retinal pigment epithelium. We implemented pattern-based regularisation on top of CNN to avoid overfitting. Using an external testing data set of 160 eyes of 131 patients with OAG, the prediction performance (absolute error (AE) and R2 between predicted and actual TH values) was calculated for (1) mean TH in whole VF and (2) each TH of 68 points. For comparison, we trained support vector machine (SVM) and multiple linear regression (MLR). RESULTS: AE of whole VF with CNN was 2.84±2.98 (mean±SD) dB, significantly smaller than those with SVM (5.65±5.12 dB) and MLR (6.96±5.38 dB) (all, p<0.001). Mean of point-wise mean AE with CNN was 5.47±3.05 dB, significantly smaller than those with SVM (7.96±4.63 dB) and MLR (11.71±4.15 dB) (all, p<0.001). R2 with CNN was 0.74 for the mean TH of whole VF, and 0.44±0.24 for the overall 68 points. CONCLUSION: DL model showed considerably accurate prediction of HFA 10-2 VF from SD-OCT.

Improving Visual Field Trend Analysis with OCT and Deeply Regularized Latent-Space Linear Regression.

PURPOSE: To investigate whether OCT measurements can improve visual field (VF) trend analyses in glaucoma patients using the deeply regularized latent-space linear regression (DLLR) model. DESIGN: Retrospective cohort study. PARTICIPANTS: Training and testing datasets included 7984 VF results from 998 eyes of 592 patients and 1184 VF results from 148 eyes of 84 patients with open-angle glaucoma, respectively. Each eye underwent a series of 8 VF tests with the Humphrey Field Analyzer OCT series obtained within the same observation period. METHODS: Using pointwise linear regression (PLR), the threshold values of a patient's eighth VF results were predicted using values from shorter VF series (first to second VF tests [VF1-2], first to third VF tests, . . . , to first to seventh VF tests [VF1-7]), and the root mean square error (RMSE) was calculated. With DLLR, OCT measurements (macular retinal nerve fiber layer thickness, the thickness of macular ganglion cell layer and inner plexiform layer, and the thickness of the outer segment and retinal pigment epithelium) that were obtained within the period of shorter VF series were incorporated into the model to predict the eighth VF. MAIN OUTCOME MEASURES: Prediction accuracy of VF trend analyses. RESULTS: The mean ± standard deviation RMSE resulting from PLR averaged 27.48 ± 16.14 dB for VF1-2 and 3.98 ± 2.25 dB for VF1-7. Significantly (P < 0.001) smaller RMSEs were obtained from DLLR: 4.57 ± 2.71 dB (VF1-2) and 3.65 ± 2.27 dB (VF1-7). CONCLUSIONS: It is useful to include OCT measurements when predicting future VF progression in glaucoma patients, especially with short VF series.

Predicting the Glaucomatous Central 10-Degree Visual Field From Optical Coherence Tomography Using Deep Learning and Tensor Regression.

PURPOSE: To predict the visual field (VF) of glaucoma patients within the central 10° from optical coherence tomography (OCT) measurements using deep learning and tensor regression. DESIGN: Cross-sectional study. METHODS: Humphrey 10-2 VFs and OCT measurements were carried out in 505 eyes of 304 glaucoma patients and 86 eyes of 43 normal subjects. VF sensitivity at each test point was predicted from OCT-measured thicknesses of macular ganglion cell layer + inner plexiform layer, retinal nerve fiber layer, and outer segment + retinal pigment epithelium. Two convolutional neural network (CNN) models were generated: (1) CNN-PR, which simply connects the output of the CNN to each VF test point; and (2) CNN-TR, which connects the output of the CNN to each VF test point using tensor regression. Prediction performance was assessed using 5-fold cross-validation through the root mean squared error (RMSE). For comparison, RMSE values were also calculated using multiple linear regression (MLR) and support vector regression (SVR). In addition, the absolute prediction error for predicting mean sensitivity in the whole VF was analyzed. RESULTS: RMSE with the CNN-TR model averaged 6.32 ± 3.76 (mean ± standard deviation) dB. Significantly (P < .05) larger RMSEs were obtained with other models: CNN-PR (6.76 ± 3.86 dB), SVR (7.18 ± 3.87 dB), and MLR (8.56 ± 3.69 dB). The absolute mean prediction error for the whole VF was 2.72 ± 2.60 dB with the CNN-TR model. CONCLUSION: The Humphrey 10-2 VF can be predicted from OCT-measured retinal layer thicknesses using deep learning and tensor regression.

Long-tailed distributions of inter-event times as mixtures of exponential distributions.

Inter-event times of various human behaviour are apparently non-Poissonian and obey long-tailed distributions as opposed to exponential distributions, which correspond to Poisson processes. It has been suggested that human individuals may switch between different states, in each of which they are regarded to generate events obeying a Poisson process. If this is the case, inter-event times should approximately obey a mixture of exponential distributions with different parameter values. In the present study, we introduce the minimum description length principle to compare mixtures of exponential distributions with different numbers of components (i.e. constituent exponential distributions). Because these distributions violate the identifiability property, one is mathematically not allowed to apply the Akaike or Bayes information criteria to their maximum-likelihood estimator to carry out model selection. We overcome this theoretical barrier by applying a minimum description principle to joint likelihoods of the data and latent variables. We show that mixtures of exponential distributions with a few components are selected, as opposed to more complex mixtures in various datasets, and that the fitting accuracy is comparable to that of state-of-the-art algorithms to fit power-law distributions to data. Our results lend support to Poissonian explanations of apparently non-Poissonian human behaviour.

Model Selection for Non-Negative Tensor Factorization with Minimum Description Length.

Non-negative tensor factorization (NTF) is a widely used multi-way analysis approach that factorizes a high-order non-negative data tensor into several non-negative factor matrices. In NTF, the non-negative rank has to be predetermined to specify the model and it greatly influences the factorized matrices. However, its value is conventionally determined by specialists' insights or trial and error. This paper proposes a novel rank selection criterion for NTF on the basis of the minimum description length (MDL) principle. Our methodology is unique in that (1) we apply the MDL principle on tensor slices to overcome a problem caused by the imbalance between the number of elements in a data tensor and that in factor matrices, and (2) we employ the normalized maximum likelihood (NML) code-length for histogram densities. We employ synthetic and real data to empirically demonstrate that our method outperforms other criteria in terms of accuracies for estimating true ranks and for completing missing values. We further show that our method can produce ranks suitable for knowledge discovery.

Detection of Longitudinal Visual Field Progression in Glaucoma Using Machine Learning.

PURPOSE: Global indices of standard automated perimerty are insensitive to localized losses, while point-wise indices are sensitive but highly variable. Region-wise indices sit in between. This study introduces a machine learning-based index for glaucoma progression detection that outperforms global, region-wise, and point-wise indices. DESIGN: Development and comparison of a prognostic index. METHOD: Visual fields from 2085 eyes of 1214 subjects were used to identify glaucoma progression patterns using machine learning. Visual fields from 133 eyes of 71 glaucoma patients were collected 10 times over 10 weeks to provide a no-change, test-retest dataset. The parameters of all methods were identified using visual field sequences in the test-retest dataset to meet fixed 95% specificity. An independent dataset of 270 eyes of 136 glaucoma patients and survival analysis were used to compare methods. RESULTS: The time to detect progression in 25% of the eyes in the longitudinal dataset using global mean deviation (MD) was 5.2 (95% confidence interval, 4.1-6.5) years; 4.5 (4.0-5.5) years using region-wise, 3.9 (3.5-4.6) years using point-wise, and 3.5 (3.1-4.0) years using machine learning analysis. The time until 25% of eyes showed subsequently confirmed progression after 2 additional visits were included were 6.6 (5.6-7.4) years, 5.7 (4.8-6.7) years, 5.6 (4.7-6.5) years, and 5.1 (4.5-6.0) years for global, region-wise, point-wise, and machine learning analyses, respectively. CONCLUSIONS: Machine learning analysis detects progressing eyes earlier than other methods consistently, with or without confirmation visits. In particular, machine learning detects more slowly progressing eyes than other methods.

A basket two-part model to analyze medical expenditure on interdependent multiple sectors.

This study proposes a novel statistical methodology to analyze expenditure on multiple medical sectors using consumer data. Conventionally, medical expenditure has been analyzed by two-part models, which separately consider purchase decision and amount of expenditure. We extend the traditional two-part models by adding the step of basket analysis for dimension reduction. This new step enables us to analyze complicated interdependence between multiple sectors without an identification problem. As an empirical application for the proposed method, we analyze data of 13 medical sectors from the Medical Expenditure Panel Survey. In comparison with the results of previous studies that analyzed the multiple sector independently, our method provides more detailed implications of the impacts of individual socioeconomic status on the composition of joint purchases from multiple medical sectors; our method has a better prediction performance.

Predicting disease progression from short biomarker series using expert advice algorithm.

Well-trained clinicians may be able to provide diagnosis and prognosis from very short biomarker series using information and experience gained from previous patients. Although mathematical methods can potentially help clinicians to predict the progression of diseases, there is no method so far that estimates the patient state from very short time-series of a biomarker for making diagnosis and/or prognosis by employing the information of previous patients. Here, we propose a mathematical framework for integrating other patients' datasets to infer and predict the state of the disease in the current patient based on their short history. We extend a machine-learning framework of "prediction with expert advice" to deal with unstable dynamics. We construct this mathematical framework by combining expert advice with a mathematical model of prostate cancer. Our model predicted well the individual biomarker series of patients with prostate cancer that are used as clinical samples.