An Efficient Fisher Matrix Approximation Method for Large-Scale Neural Network Optimization.

Although the shapes of the parameters are not crucial for designing first-order optimization methods in large scale empirical risk minimization problems, they have important impact on the size of the matrix to be inverted when developing second-order type methods. In this article, we propose an efficient and novel second-order method based on the parameters in the real matrix space [Formula: see text] and a matrix-product approximate Fisher matrix (MatFisher) by using the products of gradients. The size of the matrix to be inverted is much smaller than that of the Fisher information matrix in the real vector space [Formula: see text]. Moreover, by utilizing the matrix delayed update and the block diagonal approximation techniques, the computational cost can be controlled and is comparable with first-order methods. A global convergence and a superlinear local convergence analysis are established under mild conditions. Numerical results on image classification with ResNet50, quantum chemistry modeling with SchNet, and data-driven partial differential equations solution with PINN illustrate that our method is quite competitive to the state-of-the-art methods.

Clinical decision support model for tooth extraction therapy derived from electronic dental records.

STATEMENT OF PROBLEM: Tooth extraction therapy serves as a key initial step in many prosthodontic treatment plans. Dentists must make an appropriate decision on the tooth extraction therapy considering multiple determinants and whether a clinical decision support (CDS) model might help. PURPOSE: The purpose of this retrospective records study was to construct a CDS model to predict tooth extraction therapy in clinical situations by using electronic dental records (EDRs). MATERIAL AND METHODS: The cohort involved 4135 deidentified EDRs of 3559 patients from the database of a prosthodontics department. Knowledge-based algorithms were first proposed to convert raw data from EDRs into structured data for feature extraction. Redundant features were filtered by a recursive feature-elimination method. The tooth extraction problem was then modeled alternatively as a binary or triple classification problem to be solved by 5 machine learning algorithms. Five machine learning algorithms within each model were compared, as well as the efficiency between 2 models. In addition, the proposed CDS was verified by 2 prosthodontists. RESULTS: The triple classification model outperformed the binary model with the F1 score of the Extreme Gradient Boost (XGBoost) algorithm as 0.856 and 0.847, respectively. The XGBoost outperformed the other 4 algorithms. The accuracy, precision, and recall of the XGBoost algorithm were 0.962, 0.865, and 0.830 in the binary classification and 0.924, 0.879, and 0.836 in the triple classification, respectively. The performance of the 2 prosthodontists was inferior to the models. CONCLUSIONS: The CDS model for tooth extraction therapy achieved high performance in terms of decision-making derived from EDRs.