MENet: A Mitscherlich function based ensemble of CNN models to classify lung cancer using CT scans.

Lung cancer is one of the leading causes of cancer-related deaths worldwide. To reduce the mortality rate, early detection and proper treatment should be ensured. Computer-aided diagnosis methods analyze different modalities of medical images to increase diagnostic precision. In this paper, we propose an ensemble model, called the Mitscherlich function-based Ensemble Network (MENet), which combines the prediction probabilities obtained from three deep learning models, namely Xception, InceptionResNetV2, and MobileNetV2, to improve the accuracy of a lung cancer prediction model. The ensemble approach is based on the Mitscherlich function, which produces a fuzzy rank to combine the outputs of the said base classifiers. The proposed method is trained and tested on the two publicly available lung cancer datasets, namely Iraq-Oncology Teaching Hospital/National Center for Cancer Diseases (IQ-OTH/NCCD) and LIDC-IDRI, both of these are computed tomography (CT) scan datasets. The obtained results in terms of some standard metrics show that the proposed method performs better than state-of-the-art methods. The codes for the proposed work are available at https://github.com/SuryaMajumder/MENet.

Interpretable Predictive Modelling of Basalt Fiber Reinforced Concrete Splitting Tensile Strength Using Ensemble Machine Learning Methods and SHAP Approach.

Basalt fibers are a type of reinforcing fiber that can be added to concrete to improve its strength, durability, resistance to cracking, and overall performance. The addition of basalt fibers with high tensile strength has a particularly favorable impact on the splitting tensile strength of concrete. The current study presents a data set of experimental results of splitting tests curated from the literature. Some of the best-performing ensemble learning techniques such as Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), Random Forest, and Categorical Boosting (CatBoost) have been applied to the prediction of the splitting tensile strength of concrete reinforced with basalt fibers. State-of-the-art performance metrics such as the root mean squared error, mean absolute error and the coefficient of determination have been used for measuring the accuracy of the prediction. The impact of each input feature on the model prediction has been visualized using the Shapley Additive Explanations (SHAP) algorithm and individual conditional expectation (ICE) plots. A coefficient of determination greater than 0.9 could be achieved by the XGBoost algorithm in the prediction of the splitting tensile strength.

Optimization and Predictive Modeling of Reinforced Concrete Circular Columns.

Metaheuristic optimization techniques are widely applied in the optimal design of structural members. This paper presents the application of the harmony search algorithm to the optimal dimensioning of reinforced concrete circular columns. For the objective of optimization, the total cost of steel and concrete associated with the construction process were selected. The selected variables of optimization include the diameter of the column, the total cross-sectional area of steel, the unit costs of steel and concrete used in the construction, the total length of the column, and applied axial force and the bending moment acting on the column. By using the minimum allowable dimensions as the constraints of optimization, 3125 different data samples were generated where each data sample is an optimal design configuration. Based on the generated dataset, the SHapley Additive exPlanations (SHAP) algorithm was applied in combination with ensemble learning predictive models to determine the impact of each design variable on the model predictions. The relationships between the design variables and the objective function were visualized using the design of experiments methodology. Applying state-of-the-art statistical accuracy measures such as the coefficient of determination, the predictive models were demonstrated to be highly accurate. The current study demonstrates a novel technique for generating large datasets for the development of data-driven machine learning models. This new methodology can enhance the availability of large datasets, thereby facilitating the application of high-performance machine learning predictive models for optimal structural design.

Optimal Dimensioning of Retaining Walls Using Explainable Ensemble Learning Algorithms.

This paper develops predictive models for optimal dimensions that minimize the construction cost associated with reinforced concrete retaining walls. Random Forest, Extreme Gradient Boosting (XGBoost), Categorical Gradient Boosting (CatBoost), and Light Gradient Boosting Machine (LightGBM) algorithms were applied to obtain the predictive models. Predictive models were trained using a comprehensive dataset, which was generated using the Harmony Search (HS) algorithm. Each data sample in this database consists of a unique combination of the soil density, friction angle, ultimate bearing pressure, surcharge, the unit cost of concrete, and six different dimensions that describe an optimal retaining wall geometry. The influence of these design features on the optimal dimensioning and their interdependence are explained and visualized using the SHapley Additive exPlanations (SHAP) algorithm. The prediction accuracy of the used ensemble learning methods is evaluated with different metrics of accuracy such as the coefficient of determination, root mean square error, and mean absolute error. Comparing predicted and actual optimal dimensions on a test set showed that an R2 score of 0.99 could be achieved. In terms of computational speed, the LightGBM algorithm was found to be the fastest, with an average execution speed of 6.17 s for the training and testing of the model. On the other hand, the highest accuracy could be achieved by the CatBoost algorithm. The availability of open-source machine learning algorithms and high-quality datasets makes it possible for designers to supplement traditional design procedures with newly developed machine learning techniques. The novel methodology proposed in this paper aims at producing larger datasets, thereby increasing the applicability and accuracy of machine learning algorithms in relation to optimal dimensioning of structures.

Interpretable Machine Learning Algorithms to Predict the Axial Capacity of FRP-Reinforced Concrete Columns.

Fiber-reinforced polymer (FRP) rebars are increasingly being used as an alternative to steel rebars in reinforced concrete (RC) members due to their excellent corrosion resistance capability and enhanced mechanical properties. Extensive research works have been performed in the last two decades to develop predictive models, codes, and guidelines to estimate the axial load-carrying capacity of FRP-RC columns. This study utilizes the power of artificial intelligence and develops an alternative approach to predict the axial capacity of FRP-RC columns more accurately using data-driven machine learning (ML) algorithms. A database of 117 tests of axially loaded FRP-RC columns is collected from the literature. The geometric and material properties, column shape and slenderness ratio, reinforcement details, and FRP types are used as the input variables, while the load-carrying capacity is used as the output response to develop the ML models. Furthermore, the input-output relationship of the ML model is explained through feature importance analysis and the SHapely Additive exPlanations (SHAP) approach. Eight ML models, namely, Kernel Ridge Regression, Lasso Regression, Support Vector Machine, Gradient Boosting Machine, Adaptive Boosting, Random Forest, Categorical Gradient Boosting, and Extreme Gradient Boosting, are used in this study for capacity prediction, and their relative performances are compared to identify the best-performing ML model. Finally, predictive equations are proposed using the harmony search optimization and the model interpretations obtained through the SHAP algorithm.

The Development of a Suicidal Ideation Predictive Model for Community-Dwelling Elderly Aged >55 Years.

PURPOSE: Suicide is an important health and social concern worldwide. Both suicidal ideation and suicide rates are higher in the elderly population than in other age groups; thus, more careful attention and targeted interventions are required. Therefore, we have developed a model to predict suicidal ideation in the community-dwelling elderly aged of >55 years. PATIENTS AND METHODS: A random forest algorithm was applied to those who participated in the Korea Welfare Panel. We used a total of 26 variables as potential predictors. To resolve the imbalance in the dataset resulting from the low frequency of suicidal ideation, training was performed by applying the synthetic minority oversampling technique. The performance index was calculated by applying the predictive model to the test set, which was not included in the training process. RESULTS: A total of 6410 elderly Korean aged of >55 (mean, 71.48; standard deviation, 9.56) years were included in the analysis, of which 2.7% had suicidal ideation. The results for predicting suicidal ideation using the 26 chosen variables showed an AUC of 0.879, accuracy of 0.871, sensitivity of 0.750, and specificity of 0.874. The most significant variable in the predictive model was the severity of depression, followed by life satisfaction and self-esteem factors. Basic demographic variables such as age and gender demonstrated a relatively small effect. CONCLUSION: Machine learning can be used to create algorithms for predicting suicidal ideation in community-dwelling elderly. However, there are limitations to predicting future suicidal ideation. A predictive model that includes both biological and cognitive indicators should be created in the future.

Development of a Suicide Prediction Model for the Elderly Using Health Screening Data.

Suicide poses a serious problem globally, especially among the elderly population. To tackle the issue, this study aimed to develop a model for predicting suicide by using machine learning based on the elderly population. To obtain a large sample, the study used the big data health screening cohort provided by the National Health Insurance Sharing Service. By applying a machine learning technique, a predictive model that comprehensively utilized various factors was developed to select the elderly aged > 65 years at risk of suicide. A total of 48,047 subjects were included in the analysis. Individuals who died by suicide were older, and the number of men was significantly greater. The suicide group had a more prominent history of depression, with the use of medicaments significantly higher. Specifically, the prescription of benzodiazepines alone was associated with a high suicide risk. Furthermore, body mass index, waist circumference, total cholesterol, and low-density lipoprotein level were lower in the suicide group. We developed a model for predicting suicide by using machine learning based on the elderly population. This suicide prediction model can satisfy the performance to some extent by employing only the medical service usage behavior without subjective reports.

Choquet Integral and Coalition Game-Based Ensemble of Deep Learning Models for COVID-19 Screening From Chest X-Ray Images.

Under the present circumstances, when we are still under the threat of different strains of coronavirus, and since the most widely used method for COVID-19 detection, RT-PCR is a tedious and time-consuming manual procedure with poor precision, the application of Artificial Intelligence (AI) and Computer-Aided Diagnosis (CAD) is inevitable. Though, some vaccines have now been authorized worldwide, it will take huge time to reach everyone, especially in developing countries. In this work, we have analyzed Chest X-ray (CXR) images for the detection of the coronavirus. The primary agenda of this proposed research study is to leverage the classification performance of the deep learning models using ensemble learning. Many papers have proposed different ensemble learning techniques in this field, some methods using aggregation functions like Weighted Arithmetic Mean (WAM) among others. However, none of these methods take into consideration the decisions that subsets of the classifiers take. In this paper, we have applied Choquet integral for ensemble and propose a novel method for the evaluation of fuzzy measures using coalition game theory, information theory, and Lambda fuzzy approximation. Three different sets of fuzzy measures are calculated using three different weighting schemes along with information theory and coalition game theory. Using these three sets of fuzzy measures, three Choquet integrals are calculated and their decisions are finally combined. Besides, we have created a database by combining several image repositories developed recently. Impressive results on the newly developed dataset and the challenging COVIDx dataset support the efficacy and robustness of the proposed method. Our experimental results outperform many recently proposed methods.

Pneumonia detection in chest X-ray images using an ensemble of deep learning models.

Pneumonia is a respiratory infection caused by bacteria or viruses; it affects many individuals, especially in developing and underdeveloped nations, where high levels of pollution, unhygienic living conditions, and overcrowding are relatively common, together with inadequate medical infrastructure. Pneumonia causes pleural effusion, a condition in which fluids fill the lung, causing respiratory difficulty. Early diagnosis of pneumonia is crucial to ensure curative treatment and increase survival rates. Chest X-ray imaging is the most frequently used method for diagnosing pneumonia. However, the examination of chest X-rays is a challenging task and is prone to subjective variability. In this study, we developed a computer-aided diagnosis system for automatic pneumonia detection using chest X-ray images. We employed deep transfer learning to handle the scarcity of available data and designed an ensemble of three convolutional neural network models: GoogLeNet, ResNet-18, and DenseNet-121. A weighted average ensemble technique was adopted, wherein the weights assigned to the base learners were determined using a novel approach. The scores of four standard evaluation metrics, precision, recall, f1-score, and the area under the curve, are fused to form the weight vector, which in studies in the literature was frequently set experimentally, a method that is prone to error. The proposed approach was evaluated on two publicly available pneumonia X-ray datasets, provided by Kermany et al. and the Radiological Society of North America (RSNA), respectively, using a five-fold cross-validation scheme. The proposed method achieved accuracy rates of 98.81% and 86.85% and sensitivity rates of 98.80% and 87.02% on the Kermany and RSNA datasets, respectively. The results were superior to those of state-of-the-art methods and our method performed better than the widely used ensemble techniques. Statistical analyses on the datasets using McNemar's and ANOVA tests showed the robustness of the approach. The codes for the proposed work are available at https://github.com/Rohit-Kundu/Ensemble-Pneumonia-Detection.

Predicting Depression in Community Dwellers Using a Machine Learning Algorithm.

Depression is one of the leading causes of disability worldwide. Given the socioeconomic burden of depression, appropriate depression screening for community dwellers is necessary. We used data from the 2014 and 2016 Korea National Health and Nutrition Examination Surveys. The 2014 dataset was used as a training set, whereas the 2016 dataset was used as the hold-out test set. The synthetic minority oversampling technique (SMOTE) was used to control for class imbalances between the depression and non-depression groups in the 2014 dataset. The least absolute shrinkage and selection operator (LASSO) was used for feature reduction and classifiers in the final model. Data obtained from 9488 participants were used for the machine learning process. The depression group had poorer socioeconomic, health, functional, and biological measures than the non-depression group. From the initial 37 variables, 13 were selected using LASSO. All performance measures were calculated based on the raw 2016 dataset without the SMOTE. The area under the receiver operating characteristic curve and overall accuracy in the hold-out test set were 0.903 and 0.828, respectively. Perceived stress had the strongest influence on the classifying model for depression. LASSO can be practically applied for depression screening of community dwellers with a few variables. Future studies are needed to develop a more efficient and accurate classification model for depression.

Hough Transform-Based Angular Features for Learning-Free Handwritten Keyword Spotting.

Handwritten keyword spotting (KWS) is of great interest to the document image research community. In this work, we propose a learning-free keyword spotting method following query by example (QBE) setting for handwritten documents. It consists of four key processes: pre-processing, vertical zone division, feature extraction, and feature matching. The pre-processing step deals with the noise found in the word images, and the skewness of the handwritings caused by the varied writing styles of the individuals. Next, the vertical zone division splits the word image into several zones. The number of vertical zones is guided by the number of letters in the query word image. To obtain this information (i.e., number of letters in a query word image) during experimentation, we use the text encoding of the query word image. The user provides the information to the system. The feature extraction process involves the use of the Hough transform. The last step is feature matching, which first compares the features extracted from the word images and then generates a similarity score. The performance of this algorithm has been tested on three publicly available datasets: IAM, QUWI, and ICDAR KWS 2015. It is noticed that the proposed method outperforms state-of-the-art learning-free KWS methods considered here for comparison while evaluated on the present datasets. We also evaluate the performance of the present KWS model using state-of-the-art deep features and it is found that the features used in the present work perform better than the deep features extracted using InceptionV3, VGG19, and DenseNet121 models.

Generation of Synthetic Chest X-ray Images and Detection of COVID-19: A Deep Learning Based Approach.

COVID-19 is a disease caused by the SARS-CoV-2 virus. The COVID-19 virus spreads when a person comes into contact with an affected individual. This is mainly through drops of saliva or nasal discharge. Most of the affected people have mild symptoms while some people develop acute respiratory distress syndrome (ARDS), which damages organs like the lungs and heart. Chest X-rays (CXRs) have been widely used to identify abnormalities that help in detecting the COVID-19 virus. They have also been used as an initial screening procedure for individuals highly suspected of being infected. However, the availability of radiographic CXRs is still scarce. This can limit the performance of deep learning (DL) based approaches for COVID-19 detection. To overcome these limitations, in this work, we developed an Auxiliary Classifier Generative Adversarial Network (ACGAN), to generate CXRs. Each generated X-ray belongs to one of the two classes COVID-19 positive or normal. To ensure the goodness of the synthetic images, we performed some experimentation on the obtained images using the latest Convolutional Neural Networks (CNNs) to detect COVID-19 in the CXRs. We fine-tuned the models and achieved more than 98% accuracy. After that, we also performed feature selection using the Harmony Search (HS) algorithm, which reduces the number of features while retaining classification accuracy. We further release a GAN-generated dataset consisting of 500 COVID-19 radiographic images.

COVID-19 Detection by Optimizing Deep Residual Features with Improved Clustering-Based Golden Ratio Optimizer.

The COVID-19 virus is spreading across the world very rapidly. The World Health Organization (WHO) declared it a global pandemic on 11 March 2020. Early detection of this virus is necessary because of the unavailability of any specific drug. The researchers have developed different techniques for COVID-19 detection, but only a few of them have achieved satisfactory results. There are three ways for COVID-19 detection to date, those are real-time reverse transcription-polymerize chain reaction (RT-PCR), Computed Tomography (CT), and X-ray plays. In this work, we have proposed a less expensive computational model for automatic COVID-19 detection from Chest X-ray and CT-scan images. Our paper has a two-fold contribution. Initially, we have extracted deep features from the image dataset and then introduced a completely novel meta-heuristic feature selection approach, named Clustering-based Golden Ratio Optimizer (CGRO). The model has been implemented on three publicly available datasets, namely the COVID CT-dataset, SARS-Cov-2 dataset, and Chest X-Ray dataset, and attained state-of-the-art accuracies of 99.31%, 98.65%, and 99.44%, respectively.

LARNet: Real-Time Detection of Facial Micro Expression Using Lossless Attention Residual Network.

Facial micro expressions are brief, spontaneous, and crucial emotions deep inside the mind, reflecting the actual thoughts for that moment. Humans can cover their emotions on a large scale, but their actual intentions and emotions can be extracted at a micro-level. Micro expressions are organic when compared with macro expressions, posing a challenge to both humans, as well as machines, to identify. In recent years, detection of facial expressions are widely used in commercial complexes, hotels, restaurants, psychology, security, offices, and education institutes. The aim and motivation of this paper are to provide an end-to-end architecture that accurately detects the actual expressions at the micro-scale features. However, the main research is to provide an analysis of the specific parts that are crucial for detecting the micro expressions from a face. Many states of the art approaches have been trained on the micro facial expressions and compared with our proposed Lossless Attention Residual Network (LARNet) approach. However, the main research on this is to provide analysis on the specific parts that are crucial for detecting the micro expressions from a face. Many CNN-based approaches extracts the features at local level which digs much deeper into the face pixels. However, the spatial and temporal information extracted from the face is encoded in LARNet for a feature fusion extraction on specific crucial locations, such as nose, cheeks, mouth, and eyes regions. LARNet outperforms the state-of-the-art methods with a slight margin by accurately detecting facial micro expressions in real-time. Lastly, the proposed LARNet becomes accurate and better by training with more annotated data.

Prediction of depression among medical check-ups of 433,190 patients: A nationwide population-based study.

Depression is a mental illness that causes significant disturbances in daily life. Depression is commonly associated with low mood, severe health problems, and substantial socioeconomic burden; hence, it is necessary to be able to detect depression earlier. We utilized the medical check-up cohort database of the National Health Insurance Sharing Service in Korea. We split the total dataset into training (70%) and test (30%) sets. Subsequently, five-fold cross validation was performed in the training set. The holdout test set was only used in the last step to evaluate the performance of the predictive model. Random forest algorithm was used for the predictive model. The analysis included 433,190 individuals who had a national medical check-up from 2009-2015, which included 10,824 (2.56%) patients in the depression group. The area under the receiver-operating curve was 0.849. Other performance metrics included a sensitivity of 0.737, specificity of 0.824, positive predictive value of 0.097, negative predictive value of 0.992, and accuracy of 0.780. Our predictive model could contribute to proactively reducing depression prevalence by administering interventions to prevent depression in patients receiving medical check-up. Future studies are needed to prospectively validate the predictability of this model.

Prediction of suicide among 372,813 individuals under medical check-up.

BACKGROUND: Suicide is a serious social and public health problem. Social stigma and prejudice reduce the accessibility of mental health care services for high-risk groups, resulting in them not receiving interventions and committing suicide. A suicide prediction model is necessary to identify high-risk groups in the general population. METHODS: We used national medical check-up data from 2009 to 2015 in Korea. The latest medical check-up data for each subject was set as an index point. Analysis was undertaken for an overall follow-up period (index point to the final tracking period) as well as for a one-year follow-up period. The training set was cross-validated fivefold. The predictive model was trained using a random forest algorithm, and its performance was measured using a separate test set not included in the training. RESULTS: The analysis covered 372,813 individuals, with an average (SD) overall follow-up duration of 1.52 (1.52) years. When we predicted suicide during the overall follow-up period, the area under the receiver operating characteristic curve (AUC) was 0.849, sensitivity was 0.817, and specificity was 0.754. The performance of the predicted suicide risk model for one year from the index point was AUC 0.818, sensitivity 0.788, and specificity 0.657. CONCLUSIONS: This is probably the first suicide predictive model using machine learning based on medical check-up data from the general population. It could be used to screen high-risk suicidal groups from the population through routine medical check-ups. Future studies may test preventive interventions such as exercise and alcohol in these high-risk groups.

Machine learning-based prediction of persistent oppositional defiant behavior for 5 years.

BACKGROUND: Early detection of oppositional defiant behavior is warranted for timely intervention in children at risk. This study aimed to build a predictive model of persistent oppositional defiant behavior based on a machine learning algorithm. METHODS: With nationwide cohort data collected from 2012 to 2017, a tree-based ensemble model, random forest, was exploited to build a predictive model for persistent oppositional defiant behavior. The persistent oppositional defiant behavior was defined by the presence of oppositional defiant behavior for all the five years. The area under the receiver operating characteristic curve (AUC), overall accuracy, sensitivity, specificity, and Matthew's correlation coefficients (MCC) were measured. RESULTS: Data of 1,323 children were used for building the machine learning-based predictive model. The baseline mean ± standard deviation month-age of the participants was 51.0 ± 1.2 months. The proportion of persistent oppositional defiant behavior was 0.98% (13/1323). In the hold-out test set, the overall accuracy, AUC, sensitivity, specificity, and MCC were 0.955, 0.982, 1.000, 0.954, and 0.417, respectively. CONCLUSION: Our study demonstrated that the machine learning-based approach is useful for predicting persistent oppositional defiant behavior in preschool-aged children.

Hyperparameter Optimization Method Based on Harmony Search Algorithm to Improve Performance of 1D CNN Human Respiration Pattern Recognition System.

In this study, we propose a method to find an optimal combination of hyperparameters to improve the accuracy of respiration pattern recognition in a 1D (Dimensional) convolutional neural network (CNN). The proposed method is designed to integrate with a 1D CNN using the harmony search algorithm. In an experiment, we used the depth of the convolutional layer of the 1D CNN, the number and size of kernels in each layer, and the number of neurons in the dense layer as hyperparameters for optimization. The experimental results demonstrate that the proposed method provided a recognition rate for five respiration patterns of approximately 96.7% on average, which is an approximately 2.8% improvement over an existing method. In addition, the number of iterations required to derive the optimal combination of hyperparameters was 2,000,000 in the previous study. In contrast, the proposed method required only 3652 iterations.

Harmony Search Optimisation of Dispersed Laminated Composite Plates.

One of the major goals in the process of designing structural components is to achieve the highest possible buckling load of the structural component while keeping the cost and weight at a minimum. This paper illustrates the application of the harmony search algorithm to the buckling load maximisation of dispersed laminated composite plates with rectangular geometry. The ply thicknesses and fiber orientation angles of the plies were chosen as the design variables. Besides the commonly used carbon fiber reinforced composites, boron/epoxy and glass/epoxy composite plates were also optimised using the harmony search algorithm. Furthermore, the optimisation algorithm was applied to plates with three different aspect ratios (ratio of the longer side length to the shorter side length of the plate). The buckling loads of the plates with optimised dispersed stacking sequences were compared to the buckling loads of plates with the commonly applied 0°, ±45°, and 90° fiber angle sequence and identical ply thicknesses. For all three aspect ratios and materials in this study, the dispersed stacking sequences performed better than the plates with regular stacking sequences.