A New Method Based on Locally Optimal Step Length in Accelerated Gradient Descent for Quantum State Tomography.

Quantum state tomography (QST) is one of the key steps in determining the state of the quantum system, which is essential for understanding and controlling it. With statistical data from measurements and Positive Operator-Valued Measures (POVMs), the goal of QST is to find a density operator that best fits the measurement data. Several optimization-based methods have been proposed for QST, and one of the most successful approaches is based on Accelerated Gradient Descent (AGD) with fixed step length. While AGD with fixed step size is easy to implement, it is computationally inefficient when the computational time required to calculate the gradient is high. In this paper, we propose a new optimal method for step-length adaptation, which results in a much faster version of AGD for QST. Numerical results confirm that the proposed method is much more time-efficient than other similar methods due to the optimized step size.

Application of Wiener Filter Based on Improved BB Gradient Descent in Iris Image Restoration.

Iris recognition, renowned for its exceptional precision, has been extensively utilized across diverse industries. However, the presence of noise and blur frequently compromises the quality of iris images, thereby adversely affecting recognition accuracy. In this research, we have refined the traditional Wiener filter image restoration technique by integrating it with a gradient descent strategy, specifically employing the Barzilai-Borwein (BB) step size selection. This innovative approach is designed to enhance both the precision and resilience of iris recognition systems. The BB gradient method is adept at optimizing the parameters of the Wiener filter by introducing simulated blurring and noise conditions to the iris images. Through this process, it is capable of restoring images that have been degraded by blur and noise, leading to a significant improvement in the clarity of the restored images and, consequently, a notable elevation in recognition performance. The results of our experiments have demonstrated that this advanced method surpasses conventional filtering techniques in terms of both subjective visual quality assessments and objective peak signal-to-noise ratio (PSNR) evaluations.

A Network-Constrain Weibull AFT Model for Biomarkers Discovery.

We propose AFTNet, a novel network-constraint survival analysis method based on the Weibull accelerated failure time (AFT) model solved by a penalized likelihood approach for variable selection and estimation. When using the log-linear representation, the inference problem becomes a structured sparse regression problem for which we explicitly incorporate the correlation patterns among predictors using a double penalty that promotes both sparsity and grouping effect. Moreover, we establish the theoretical consistency for the AFTNet estimator and present an efficient iterative computational algorithm based on the proximal gradient descent method. Finally, we evaluate AFTNet performance both on synthetic and real data examples.

ANS-SCMC: A matrix completion method based on adaptive neighbourhood similarity and sparse constraints for predicting microbe-disease associations.

The use of matrix completion methods to predict the association between microbes and diseases can effectively improve treatment efficiency. However, the similarity measures used in the existing methods are often influenced by various factors such as neighbourhood size, choice of similarity metric, or multiple parameters for similarity fusion, making it challenging. Additionally, matrix completion is currently limited by the sparsity of the initial association matrix, which restricts its predictive performance. To address these problems, we propose a matrix completion method based on adaptive neighbourhood similarity and sparse constraints (ANS-SCMC) for predict microbe-disease potential associations. Adaptive neighbourhood similarity learning dynamically uses the decomposition results as effective information for the next learning iteration by simultaneously performing local manifold structure learning and decomposition. This approach effectively preserves fine local structure information and avoids the influence of weight parameters directly involved in similarity measurement. Additionally, the sparse constraint-based matrix completion approach can better handle the sparsity challenge in the association matrix. Finally, the algorithm we proposed has achieved significantly higher predictive performance in the validation compared to several commonly used prediction methods proposed to date. Furthermore, in the case study, the prediction algorithm achieved an accuracy of up to 80% for the top 10 microbes associated with type 1 diabetes and 100% for Crohn's disease respectively.

Human lung cancer classification and comprehensive analysis using different machine learning techniques.

Lung cancer is the most common causes of death among all cancer-related diseases. A lung scan examination of the patient is the primary diagnostic technique. This scan analysis pertains to an MRI, CT, or X-ray. The automated classification of lung cancer is difficult due to the involvement of multiple steps in imaging patients' lungs. In this manuscript, human lung cancer classification and comprehensive analysis using different machine learning techniques is proposed. Initially, the input images are gathered using lung cancer dataset. The proposed method processes these images using image-processing techniques, and further machine learning techniques are utilized for categorization. Seven different classifiers including the k-nearest neighbors (KNN), support vector machine (SVM), decision tree (DT), multinomial naive Bayes (MNB), stochastic gradient descent (SGD), random forest (RF), and multi-layer perceptron (MLP) classifier are used, which classifies the lung cancer as malignant and benign. The performance of the proposed approach is examined using performances metrics, like positive predictive value, accuracy, sensitivity, and f-score are evaluated. Among them, the performance of the MLP classifier provides 25.34%, 45.39%, 15.39%, 41.28%, 22.17%, and 12.12% higher accuracy than other KNN, SVM, DT, MNB, SGD, and RF respectively. RESEARCH HIGHLIGHTS: Lung cancer is a leading cause of cancer-related death. Imaging (MRI, CT, and X-ray) aids diagnosis. Automated classification of lung cancer faces challenges due to complex imaging steps. This study proposes human lung cancer classification using diverse machine learning techniques. Input images from lung cancer dataset undergo image processing and machine learning. Classifiers like k-nearest neighbors, support vector machine, decision tree, multinomial naive Bayes, stochastic gradient descent, random forest, and multi-layer perceptron (MLP) classify cancer types; MLP excels in accuracy.

A novel block-coordinate gradient descent algorithm for simultaneous grouped selection of fixed and random effects in joint modeling.

Joint models for longitudinal and time-to-event data are receiving increasing attention owing to its capability of capturing the possible association between these two types of data. Typically, a joint model consists of a longitudinal submodel for longitudinal processes and a survival submodel for the time-to-event response, and links two submodels by common covariates that may carry both fixed and random effects. However, research gaps still remain on how to simultaneously select fixed and random effects from the two submodels under the joint modeling framework efficiently and effectively. In this article, we propose a novel block-coordinate gradient descent (BCGD) algorithm to simultaneously select multiple longitudinal covariates that may carry fixed and random effects in the joint model. Specifically, for the multiple longitudinal processes, a linear mixed effect model is adopted where random intercepts and slopes serve as essential covariates of the trajectories, and for the survival submodel, the popular proportional hazard model is employed. A penalized likelihood estimation is used to control the dimensionality of covariates in the joint model and estimate the unknown parameters, especially when estimating the covariance matrix of random effects. The proposed BCGD method can successfully capture the useful covariates of both fixed and random effects with excellent selection power, and efficiently provide a relatively accurate estimate of fixed and random effects empirically. The simulation results show excellent performance of the proposed method and support its effectiveness. The proposed BCGD method is further applied on two real data sets, and we examine the risk factors for the effects of different heart valves, differing on type of tissue, implanted in the aortic position and the risk factors for the diagnosis of primary biliary cholangitis.

CNVDeep: deep association of copy number variants with neurocognitive disorders.

BACKGROUND: Copy number variants (CNVs) have become increasingly instrumental in understanding the etiology of all diseases and phenotypes, including Neurocognitive Disorders (NDs). Among the well-established regions associated with ND are small parts of chromosome 16 deletions (16p11.2) and chromosome 15 duplications (15q3). Various methods have been developed to identify associations between CNVs and diseases of interest. The majority of methods are based on statistical inference techniques. However, due to the multi-dimensional nature of the features of the CNVs, these methods are still immature. The other aspect is that regions discovered by different methods are large, while the causative regions may be much smaller. RESULTS: In this study, we propose a regularized deep learning model to select causal regions for the target disease. With the help of the proximal [20] gradient descent algorithm, the model utilizes the group LASSO concept and embraces a deep learning model in a sparsity framework. We perform the CNV analysis for 74,811 individuals with three types of brain disorders, autism spectrum disorder (ASD), schizophrenia (SCZ), and developmental delay (DD), and also perform cumulative analysis to discover the regions that are common among the NDs. The brain expression of genes associated with diseases has increased by an average of 20 percent, and genes with homologs in mice that cause nervous system phenotypes have increased by 18 percent (on average). The DECIPHER data source also seeks other phenotypes connected to the detected regions alongside gene ontology analysis. The target diseases are correlated with some unexplored regions, such as deletions on 1q21.1 and 1q21.2 (for ASD), deletions on 20q12 (for SCZ), and duplications on 8p23.3 (for DD). Furthermore, our method is compared with other machine learning algorithms. CONCLUSIONS: Our model effectively identifies regions associated with phenotypic traits using regularized deep learning. Rather than attempting to analyze the whole genome, CNVDeep allows us to focus only on the causative regions of disease.

Additional fractional gradient descent identification algorithm based on multi-innovation principle for autoregressive exogenous models.

This paper proposed the additional fractional gradient descent identification algorithm based on the multi-innovation principle for autoregressive exogenous models. This algorithm incorporates an additional fractional order gradient to the integer order gradient. The two gradients are synchronously used to identify model parameters, thereby accelerating the convergence of the algorithm. Furthermore, to address the limitation of conventional gradient descent algorithms, which only use the information of the current moment to estimate the parameters of the next moment, resulting in low information utilisation, the multi-innovation principle is applied. Specifically, the integer-order gradient and additional fractional-order gradient are expanded into multi-innovation forms, and the parameters of the next moment are simultaneously estimated using multi-innovation matrices, thereby further enhancing the parameter estimation speed and identification accuracy. The convergence of the algorithm is demonstrated, and its effectiveness is verified through simulations and experiments involving the identification of a 3-DOF gyroscope system.

Variational Supertrees for Bayesian Phylogenetics.

Bayesian phylogenetic inference is powerful but computationally intensive. Researchers may find themselves with two phylogenetic posteriors on overlapping data sets and may wish to approximate a combined result without having to re-run potentially expensive Markov chains on the combined data set. This raises the question: given overlapping subsets of a set of taxa (e.g. species or virus samples), and given posterior distributions on phylogenetic tree topologies for each of these taxon sets, how can we optimize a probability distribution on phylogenetic tree topologies for the entire taxon set? In this paper we develop a variational approach to this problem and demonstrate its effectiveness. Specifically, we develop an algorithm to find a suitable support of the variational tree topology distribution on the entire taxon set, as well as a gradient-descent algorithm to minimize the divergence from the restrictions of the variational distribution to each of the given per-subset probability distributions, in an effort to approximate the posterior distribution on the entire taxon set.

Numerical algorithm for nonlinearity compensation of hardly constrained actuation for trajectory tracking control of deadzone-included dynamic systems.

This paper addresses the control of a nonlinear system affected by deadzone effects, using a constrained actuator. The system itself incorporates a second-order oscillatory dynamic actuator, with an unknown nonlinear input-output relationship. The proposed algorithm not only accommodates the deadzone constraints on control inputs but also considers the actuator's saturation limits in control input calculations. It introduces a trajectory tracking mechanism that, instead of directly following the primary trajectory, adheres to an alternative trajectory capable of stable tracking, gradually converging to the main trajectory while accounting for operational constraints. In practical control systems, the actuator's input-output relationship is often nonlinear and unknown, requiring inversion for model-based control. This paper employs an offline-trained neural network trained on synthetic data to identify and approximate the actuator's behavior. To optimize the control system's performance and ensure stability during sudden error changes, the control input operates in two modes: position and velocity control. This dual-mode control allows for continuous switching between the two, facilitated by an innovative optimization technique based on the gradient descent method with a variable step size. Simulation results validate the effectiveness of the proposed algorithm in controlling systems constrained by hard limits and featuring nonlinear oscillatory actuators, providing a valuable contribution to the field of control systems.

Bayesian Networks for Prescreening in Depression: Algorithm Development and Validation.

BACKGROUND: Identifying individuals with depressive symptomatology (DS) promptly and effectively is of paramount importance for providing timely treatment. Machine learning models have shown promise in this area; however, studies often fall short in demonstrating the practical benefits of using these models and fail to provide tangible real-world applications. OBJECTIVE: This study aims to establish a novel methodology for identifying individuals likely to exhibit DS, identify the most influential features in a more explainable way via probabilistic measures, and propose tools that can be used in real-world applications. METHODS: The study used 3 data sets: PROACTIVE, the Brazilian National Health Survey (Pesquisa Nacional de Saúde [PNS]) 2013, and PNS 2019, comprising sociodemographic and health-related features. A Bayesian network was used for feature selection. Selected features were then used to train machine learning models to predict DS, operationalized as a score of ≥10 on the 9-item Patient Health Questionnaire. The study also analyzed the impact of varying sensitivity rates on the reduction of screening interviews compared to a random approach. RESULTS: The methodology allows the users to make an informed trade-off among sensitivity, specificity, and a reduction in the number of interviews. At the thresholds of 0.444, 0.412, and 0.472, determined by maximizing the Youden index, the models achieved sensitivities of 0.717, 0.741, and 0.718, and specificities of 0.644, 0.737, and 0.766 for PROACTIVE, PNS 2013, and PNS 2019, respectively. The area under the receiver operating characteristic curve was 0.736, 0.801, and 0.809 for these 3 data sets, respectively. For the PROACTIVE data set, the most influential features identified were postural balance, shortness of breath, and how old people feel they are. In the PNS 2013 data set, the features were the ability to do usual activities, chest pain, sleep problems, and chronic back problems. The PNS 2019 data set shared 3 of the most influential features with the PNS 2013 data set. However, the difference was the replacement of chronic back problems with verbal abuse. It is important to note that the features contained in the PNS data sets differ from those found in the PROACTIVE data set. An empirical analysis demonstrated that using the proposed model led to a potential reduction in screening interviews of up to 52% while maintaining a sensitivity of 0.80. CONCLUSIONS: This study developed a novel methodology for identifying individuals with DS, demonstrating the utility of using Bayesian networks to identify the most significant features. Moreover, this approach has the potential to substantially reduce the number of screening interviews while maintaining high sensitivity, thereby facilitating improved early identification and intervention strategies for individuals experiencing DS.

A Hybrid Method for Density Power Divergence Minimization with Application to Robust Univariate Location and Scale Estimation.

We develop a new globally convergent optimization method for solving a constrained minimization problem underlying the minimum density power divergence estimator for univariate Gaussian data in the presence of outliers. Our hybrid procedure combines classical Newton's method with a gradient descent iteration equipped with a step control mechanism based on Armijo's rule to ensure global convergence. Extensive simulations comparing the resulting estimation procedure with the more prominent robust competitor, Minimum Covariance Determinant (MCD) estimator, across a wide range of breakdown point values suggest improved efficiency of our method. Application to estimation and inference for a real-world dataset is also given.

A SAR Ship Detection Method Based on Adversarial Training.

SAR (synthetic aperture radar) ship detection is a hot topic due to the breadth of its application. However, limited by the volume of the SAR image, the generalization ability of the detector is low, which makes it difficult to adapt to new scenes. Although many data augmentation methods-for example, clipping, pasting, and mixing-are used, the accuracy is improved little. In order to solve this problem, the adversarial training is used for data generation in this paper. Perturbation is added to the SAR image to generate new samples for training, and it can make the detector learn more abundant features and promote the robustness of the detector. By separating batch normalization between clean samples and disturbed images, the performance degradation on clean samples is avoided. By simultaneously perturbing and selecting large losses of classification and location, it can keep the detector adaptable to more confrontational samples. The optimization efficiency and results are improved through K-step average perturbation and one-step gradient descent. The experiments on different detectors show that the proposed method achieves 8%, 10%, and 17% AP (Average Precision) improvement on the SSDD, SAR-Ship-Dataset, and AIR-SARShip, compared to the traditional data augmentation methods.

Efficient standardization of clinical T2-weighted images: Phase-conjugacy e-CAMP with projected gradient descent.

PURPOSE: To standardize T 2 $$ {}_2 $$ -weighted images from clinical Turbo Spin Echo (TSE) scans by generating corresponding T 2 $$ {}_2 $$ maps with the goal of removing scanner- and/or protocol-specific heterogeneity. METHODS: The T 2 $$ {}_2 $$ map is estimated by minimizing an objective function containing a data fidelity term in a Virtual Conjugate Coils (VCC) framework, where the signal evolution model is expressed as a linear constraint. The objective function is minimized by Projected Gradient Descent (PGD). RESULTS: The algorithm achieves accuracy comparable to methods with customized sampling schemes for accelerated T 2 $$ {}_2 $$ mapping. The results are insensitive to the tunable parameters, and the relaxed background phase prior produces better T 2 $$ {}_2 $$ maps compared to the strict real-value enforcement. It is worth noting that the algorithm works well with challenging T 2 $$ {}_2 $$ w-TSE data using typical clinical parameters. The observed normalized root mean square error ranges from 6.8% to 12.3% over grey and white matter, a clinically common level of quantitative map error. CONCLUSION: The novel methodological development creates an efficient algorithm that allows for T 2 $$ {}_2 $$ map generated from TSE data with typical clinical parameters, such as high resolution, long echo train length, and low echo spacing. Reconstruction of T 2 $$ {}_2 $$ maps from TSE data with typical clinical parameters has not been previously reported.

Gradient-based autonomous obstacle avoidance trajectory planning for B-spline UAVs.

Unmanned aerial vehicles (UAVs) have become the focus of current research because of their practicability in various scenarios. However, current local path planning methods often result in trajectories with numerous sharp or inflection points, which are not ideal for smooth UAV flight. This paper introduces a UAV path planning approach based on distance gradients. The key improvements include generating collision-free paths using collision information from initial trajectories and obstacles. Then, collision-free paths are subsequently optimized using distance gradient information. Additionally, a trajectory time adjustment method is proposed to ensure the feasibility and safety of the trajectory while prioritizing smoothness. The Limited-memory BFGS algorithm is employed to efficiently solve optimal local paths, with the ability to quickly restart the trajectory optimization program. The effectiveness of the proposed method is validated in the Robot Operating System simulation environment, demonstrating its ability to meet trajectory planning requirements for UAVs in complex unknown environments with high dynamics. Moreover, it surpasses traditional UAV trajectory planning methods in terms of solution speed, trajectory length, and data volume.

An optimization model for monthly time-step drilling schedule under planned field production.

The field production profile over the yearly horizon is planned for a balance between economy, security, and sustainability of energy. An optimal drilling schedule is required to achieve the planned production profile with minimized drilling frequency and summation. In this study, we treat each possible production process of each well as a dependent time series and the basic unit. Then we ensemble all of them into a tensor. Based on formulated tensor calculation and Lasso regularization, a linear mathematical optimization model for well drilling schedule was developed. The model is aimed at minimizing production profile error while optimizing drilling frequency and summation. Although the model proposed in this work requires more memory consumption to be solved using a computer, it is assured as a linear model and could be numerically globally solved in a stable and efficient way using gradient descent, avoiding complex nonlinear programming problems. Main input data and parameters involved in the model are analyzed in detail to understand the effects of different production parameters on the drilling schedule and production profile. The proposed model in this work can evaluate the manual drilling schedule and automatically generate an optimized drilling schedule for the gas field, significantly reducing development plan formulation time.

Pairwise stochastic approximation for confirmatory factor analysis of categorical data.

Pairwise likelihood is a limited-information method widely used to estimate latent variable models, including factor analysis of categorical data. It can often avoid evaluating high-dimensional integrals and, thus, is computationally more efficient than relying on the full likelihood. Despite its computational advantage, the pairwise likelihood approach can still be demanding for large-scale problems that involve many observed variables. We tackle this challenge by employing an approximation of the pairwise likelihood estimator, which is derived from an optimization procedure relying on stochastic gradients. The stochastic gradients are constructed by subsampling the pairwise log-likelihood contributions, for which the subsampling scheme controls the per-iteration computational complexity. The stochastic estimator is shown to be asymptotically equivalent to the pairwise likelihood one. However, finite-sample performance can be improved by compounding the sampling variability of the data with the uncertainty introduced by the subsampling scheme. We demonstrate the performance of the proposed method using simulation studies and two real data applications.

Multi-Step Internet Traffic Forecasting Models with Variable Forecast Horizons for Proactive Network Management.

The ISP (Internet Service Provider) industry relies heavily on internet traffic forecasting (ITF) for long-term business strategy planning and proactive network management. Effective ITF frameworks are necessary to manage these networks and prevent network congestion and over-provisioning. This study introduces an ITF model designed for proactive network management. It innovatively combines outlier detection and mitigation techniques with advanced gradient descent and boosting algorithms, including Gradient Boosting Regressor (GBR), Extreme Gradient Boosting (XGB), Light Gradient Boosting Machine (LGB), CatBoost Regressor (CBR), and Stochastic Gradient Descent (SGD). In contrast to traditional methods that rely on synthetic datasets, our model addresses the problems caused by real aberrant ISP traffic data. We evaluated our model across varying forecast horizons-six, nine, and twelve steps-demonstrating its adaptability and superior predictive accuracy compared to traditional forecasting models. The integration of the outlier detection and mitigation module significantly enhances the model's performance, ensuring robust and accurate predictions even in the presence of data volatility and anomalies. To guarantee that our suggested model works in real-world situations, our research is based on an extensive experimental setup that uses real internet traffic monitoring from high-speed ISP networks.

On the different regimes of stochastic gradient descent.

Modern deep networks are trained with stochastic gradient descent (SGD) whose key hyperparameters are the number of data considered at each step or batch size [Formula: see text], and the step size or learning rate [Formula: see text]. For small [Formula: see text] and large [Formula: see text], SGD corresponds to a stochastic evolution of the parameters, whose noise amplitude is governed by the "temperature" [Formula: see text]. Yet this description is observed to break down for sufficiently large batches [Formula: see text], or simplifies to gradient descent (GD) when the temperature is sufficiently small. Understanding where these cross-overs take place remains a central challenge. Here, we resolve these questions for a teacher-student perceptron classification model and show empirically that our key predictions still apply to deep networks. Specifically, we obtain a phase diagram in the [Formula: see text]-[Formula: see text] plane that separates three dynamical phases: i) a noise-dominated SGD governed by temperature, ii) a large-first-step-dominated SGD and iii) GD. These different phases also correspond to different regimes of generalization error. Remarkably, our analysis reveals that the batch size [Formula: see text] separating regimes (i) and (ii) scale with the size [Formula: see text] of the training set, with an exponent that characterizes the hardness of the classification problem.

Detection of Chylous Plasma Based on Machine Learning and Hyperspectral Techniques.

Chylous blood is the main cause of unqualified and scrapped blood among volunteer blood donors. Therefore, a diagnostic method that can quickly and accurately identify chylous blood before donation is needed. In this study, the GaiaSorter "Gaia" hyperspectral sorter was used to extract 254 bands of plasma images, ranging from 900 nm to 1700 nm. Four different machine learning algorithms were used, including decision tree, Gaussian Naive Bayes (GaussianNB), perceptron, and stochastic gradient descent models. First, the preliminary classification accuracies were compared with the original data, which showed that the effects of the decision tree and GaussianNB models were better; their average accuracies could reach over 90%. Then, the feature dimension reduction was performed on the original data. The results showed that the effects of the decision tree were better with a classification accuracy of 93.33%. the classification of chylous plasma using different chylous indices suggested that the accuracies of the decision trees model both before and after the feature dimension reductions were the best with over 80% accuracy. The results of feature dimension reduction showed that the characteristic bands corresponded to all kinds of plasma, thereby showing their classification and identification potential. By applying the spectral characteristics of plasma to medical technology, this study suggested a rapid and effective method for the identification of chylous plasma and provided a reference for the blood detection technology to achieve the goal of reducing wasting blood resources and improving the work efficiency of the medical staff.