Advanced Image Stitching Method for Dual-Sensor Inspection.

Efficient image stitching plays a vital role in the Non-Destructive Evaluation (NDE) of infrastructures. An essential challenge in the NDE of infrastructures is precisely visualizing defects within large structures. The existing literature predominantly relies on high-resolution close-distance images to detect surface or subsurface defects. While the automatic detection of all defect types represents a significant advancement, understanding the location and continuity of defects is imperative. It is worth noting that some defects may be too small to capture from a considerable distance. Consequently, multiple image sequences are captured and processed using image stitching techniques. Additionally, visible and infrared data fusion strategies prove essential for acquiring comprehensive information to detect defects across vast structures. Hence, there is a need for an effective image stitching method appropriate for infrared and visible images of structures and industrial assets, facilitating enhanced visualization and automated inspection for structural maintenance. This paper proposes an advanced image stitching method appropriate for dual-sensor inspections. The proposed image stitching technique employs self-supervised feature detection to enhance the quality and quantity of feature detection. Subsequently, a graph neural network is employed for robust feature matching. Ultimately, the proposed method results in image stitching that effectively eliminates perspective distortion in both infrared and visible images, a prerequisite for subsequent multi-modal fusion strategies. Our results substantially enhance the visualization capabilities for infrastructure inspection. Comparative analysis with popular state-of-the-art methods confirms the effectiveness of the proposed approach.

Robust Multi-Modal Image Registration for Image Fusion Enhancement in Infrastructure Inspection.

Efficient multi-modal image fusion plays an important role in the non-destructive evaluation (NDE) of infrastructures, where an essential challenge is the precise visualizing of defects. While automatic defect detection represents a significant advancement, the determination of the precise location of both surface and subsurface defects simultaneously is crucial. Hence, visible and infrared data fusion strategies are essential for acquiring comprehensive and complementary information to detect defects across vast structures. This paper proposes an infrared and visible image registration method based on Euclidean evaluation together with a trade-off between key-point threshold and non-maximum suppression. Moreover, we employ a multi-modal fusion strategy to investigate the robustness of our image registration results.

Simulation-aided infrared thermography with decomposition-based noise reduction for detecting defects in ancient polyptychs.

In recent years, the conservation and protection of ancient cultural heritage have received increasing attention, and non-destructive testing (NDT), which can minimize the damage done to the test subject, plays an integral role therein. For instance, NDT through active infrared thermal imaging can be applied to ancient polyptychs, which can realize accurate detection of damage and defects existing on the surface and interior of the polyptychs. In this study, infrared thermography is used for non-invasive investigation and evaluation of two polyptych samples with different pigments and artificial defects, but both reproduced based on a painting by Pietro Lorenzetti (1280/85-1348) using the typical tempera technique of the century. It is noted that, to avoid as far as possible secondary damages done to the ancient cultural heritages, repeated damage-detection experiments are rarely carried out on the test subjects. To that end, numerical simulation is used to reveal the heat transfer properties and temperature distributions, as to perform procedural verification and reduce the number of experiments that need to be conducted on actual samples. Technique-wise, to improve the observability of the experimental results, a total variation regularized low-rank tensor decomposition algorithm is implemented to reduce the background noise and improve the contrast of the images. Furthermore, the efficacy of image processing is quantified through the structural-similarity evaluation.

Deep Learning Neural Network Performance on NDT Digital X-ray Radiography Images: Analyzing the Impact of Image Quality Parameters-An Experimental Study.

In response to the growing inspection demand exerted by process automation in component manufacturing, non-destructive testing (NDT) continues to explore automated approaches that utilize deep-learning algorithms for defect identification, including within digital X-ray radiography images. This necessitates a thorough understanding of the implication of image quality parameters on the performance of these deep-learning models. This study investigated the influence of two image-quality parameters, namely signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR), on the performance of a U-net deep-learning semantic segmentation model. Input images were acquired with varying combinations of exposure factors, such as kilovoltage, milli-ampere, and exposure time, which altered the resultant radiographic image quality. The data were sorted into five different datasets according to their measured SNR and CNR values. The deep-learning model was trained five distinct times, utilizing a unique dataset for each training session. Training the model with high CNR values yielded an intersection-over-union (IoU) metric of 0.9594 on test data of the same category but dropped to 0.5875 when tested on lower CNR test data. The result of this study emphasizes the importance of achieving a balance in training dataset according to the investigated quality parameters in order to enhance the performance of deep-learning segmentation models for NDT digital X-ray radiography applications.

Automatic Detection and Identification of Defects by Deep Learning Algorithms from Pulsed Thermography Data.

Infrared thermography (IRT), is one of the most interesting techniques to identify different kinds of defects, such as delamination and damage existing for quality management of material. Objective detection and segmentation algorithms in deep learning have been widely applied in image processing, although very rarely in the IRT field. In this paper, spatial deep-learning image processing methods for defect detection and identification were discussed and investigated. The aim in this work is to integrate such deep-learning (DL) models to enable interpretations of thermal images automatically for quality management (QM). That requires achieving a high enough accuracy for each deep-learning method so that they can be used to assist human inspectors based on the training. There are several alternatives of deep Convolutional Neural Networks for detecting the images that were employed in this work. These included: 1. The instance segmentation methods Mask-RCNN (Mask Region-based Convolutional Neural Networks) and Center-Mask; 2. The independent semantic segmentation methods: U-net and Resnet-U-net; 3. The objective localization methods: You Only Look Once (YOLO-v3) and Faster Region-based Convolutional Neural Networks (Fast-er-RCNN). In addition, a regular infrared image segmentation processing combination method (Absolute thermal contrast (ATC) and global threshold) was introduced for comparison. A series of academic samples composed of different materials and containing artificial defects of different shapes and nature (flat-bottom holes, Teflon inserts) were evaluated, and all results were studied to evaluate the efficacy and performance of the proposed algorithms.

Defect Detection and Depth Estimation in Composite Materials for Pulsed Thermography Images by Nonuniform Heating Correction and Oriented Gradient Information.

Pulsed thermography is a nondestructive method commonly used to explore anomalies in composite materials. This paper presents a procedure for the automated detection of defects in thermal images of composite materials obtained with pulsed thermography experiments. The proposed methodology is simple and novel as it is reliable in low-contrast and nonuniform heating conditions and does not require data preprocessing. Nonuniform heating correction and the gradient direction information combined with a local and global segmentation phase are used to analyze carbon fiber-reinforced plastic (CFRP) thermal images with Teflon inserts with different length/depth ratios. Additionally, a comparison between the actual depths and estimated depths of detected defects is performed. The performance of the nonuniform heating correction proposed method is superior to that obtained on the same CFRP sample analyzed with a deep learning algorithm and the background thermal compensation by filtering strategy.

Experimental Study and FEM Simulations for Detection of Rebars in Concrete Slabs by Coplanar Capacitive Sensing Technique.

In the present study, a relatively novel non-destructive testing (NDT) method called the coplanar capacitive sensing technique was applied in order to detect different sizes of rebars in a reinforced concrete (RC) structure. This technique effectively detects changes in the dielectric properties during scanning in various sections of concrete with and without rebars. Numerical simulations were carried out by three-dimensional (3D) finite element modelling (FEM) in COMSOL Multiphysics software to analyse the impact of the presence of rebars on the electric field generated by the coplanar capacitive probe. In addition, the effect of the presence of a surface defect on the rebar embedded in the concrete slab was demonstrated by the same software for the first time. Experiments were performed on a concrete slab containing rebars, and were compared with FEM results. The results showed that there is a good qualitative agreement between the numerical simulations and experimental results.

Dual-Intended Deep Learning Model for Breast Cancer Diagnosis in Ultrasound Imaging.

Automated medical data analysis demonstrated a significant role in modern medicine, and cancer diagnosis/prognosis to achieve highly reliable and generalizable systems. In this study, an automated breast cancer screening method in ultrasound imaging is proposed. A convolutional deep autoencoder model is presented for simultaneous segmentation and radiomic extraction. The model segments the breast lesions while concurrently extracting radiomic features. With our deep model, we perform breast lesion segmentation, which is linked to low-dimensional deep-radiomic extraction (four features). Similarly, we used high dimensional conventional imaging throughputs and applied spectral embedding techniques to reduce its size from 354 to 12 radiomics. A total of 780 ultrasound images-437 benign, 210, malignant, and 133 normal-were used to train and validate the models in this study. To diagnose malignant lesions, we have performed training, hyperparameter tuning, cross-validation, and testing with a random forest model. This resulted in a binary classification accuracy of 78.5% (65.1-84.1%) for the maximal (full multivariate) cross-validated model for a combination of radiomic groups.

Data Enhancement via Low-Rank Matrix Reconstruction in Pulsed Thermography for Carbon-Fibre-Reinforced Polymers.

Pulsed thermography is a commonly used non-destructive testing method and is increasingly studied for the assessment of advanced materials such as carbon fibre-reinforced polymer (CFRP). Different processing approaches are proposed to detect and characterize anomalies that may be generated in structures during the manufacturing cycle or service period. In this study, matrix decomposition using Robust PCA via Inexact-ALM is investigated as a pre- and post-processing approach in combination with state-of-the-art approaches (i.e., PCT, PPT and PLST) on pulsed thermography thermal data. An academic sample with several artificial defects of different types, i.e., flat-bottom-holes (FBH), pull-outs (PO) and Teflon inserts (TEF), was employed to assess and compare defect detection and segmentation capabilities of different processing approaches. For this purpose, the contrast-to-noise ratio (CNR) and similarity coefficient were used as quantitative metrics. The results show a clear improvement in CNR when Robust PCA is applied as a pre-processing technique, CNR values for FBH, PO and TEF improve up to 164%, 237% and 80%, respectively, when compared to principal component thermography (PCT), whilst the CNR improvement with respect to pulsed phase thermography (PPT) was 77%, 101% and 289%, respectively. In the case of partial least squares thermography, Robust PCA results improved not only only when used as a pre-processing technique but also when used as a post-processing technique; however, this improvement is higher for FBHs and POs after pre-processing. Pre-processing increases CNR scores for FBHs and POs with a ratio from 0.43% to 115.88% and from 13.48% to 216.63%, respectively. Similarly, post-processing enhances the FBHs and POs results with a ratio between 9.62% and 296.9% and 16.98% to 92.6%, respectively. A low-rank matrix computed from Robust PCA as a pre-processing technique on raw data before using PCT and PPT can enhance the results of 67% of the defects. Using low-rank matrix decomposition from Robust PCA as a pre- and post-processing technique outperforms PLST results of 69% and 67% of the defects. These results clearly indicate that pre-processing pulsed thermography data by Robust PCA can elevate the defect detectability of advanced processing techniques, such as PCT, PPT and PLST, while post-processing using the same methods, in some cases, can deteriorate the results.

Multi-Excitation Infrared Fusion for Impact Evaluation of Aluminium-BFRP/GFRP Hybrid Composites.

Fibre metal laminates are widely implemented in the aerospace industry owing to the merits of fatigue resistance and plastic properties. An effective defect assessment technique needs to be investigated for this type of composite materials. In order to achieve accurate impact-induced damage evaluation, a multi-excitation infrared fusion method is introduced in this study. Optical excitation thermography with high performance on revealing surface and subsurface defects is combined with vibro-thermography to improve the capability of detection on defects. Quantitative analysis is carried out on the temperature curve to assess the impact-induced deformation. A new image fusion framework including feature extraction, feature selection and fusion steps is proposed to fully utilize the information from two excitation modalities. Six fibre metal laminates which contain aluminium-basalt fibre reinforced plastic and aluminium-glass fibre reinforced plastic are investigated. Features from different perspectives are compared and selected via intensity contrast on deformation area for fusion imaging. Both types of defects (i.e., surface and sub-surface) and the internal deformation situation of these six samples are characterized clearly and intuitively.

Impartially Validated Multiple Deep-Chain Models to Detect COVID-19 in Chest X-ray Using Latent Space Radiomics.

The COVID-19 pandemic continues to spread globally at a rapid pace, and its rapid detection remains a challenge due to its rapid infectivity and limited testing availability. One of the simply available imaging modalities in clinical routine involves chest X-ray (CXR), which is often used for diagnostic purposes. Here, we proposed a computer-aided detection of COVID-19 in CXR imaging using deep and conventional radiomic features. First, we used a 2D U-Net model to segment the lung lobes. Then, we extracted deep latent space radiomics by applying deep convolutional autoencoder (ConvAE) with internal dense layers to extract low-dimensional deep radiomics. We used Johnson-Lindenstrauss (JL) lemma, Laplacian scoring (LS), and principal component analysis (PCA) to reduce dimensionality in conventional radiomics. The generated low-dimensional deep and conventional radiomics were integrated to classify COVID-19 from pneumonia and healthy patients. We used 704 CXR images for training the entire model (i.e., U-Net, ConvAE, and feature selection in conventional radiomics). Afterward, we independently validated the whole system using a study cohort of 1597 cases. We trained and tested a random forest model for detecting COVID-19 cases through multivariate binary-class and multiclass classification. The maximal (full multivariate) model using a combination of the two radiomic groups yields performance in classification cross-validated accuracy of 72.6% (69.4-74.4%) for multiclass and 89.6% (88.4-90.7%) for binary-class classification.

Multiscale Analysis of Solar Loading Thermographic Signals for Wall Structure Inspection.

Infrared thermography has been widely adopted in many applications for material structure inspection, where data analysis methods are often implemented to elaborate raw thermal data and to characterize material structural properties. Herein, a multiscale thermographic data analysis framework is proposed and applied to building structure inspection. In detail, thermograms are first collected by conducting solar loading thermography, which are then decomposed into several intrinsic mode functions under different spatial scales by multidimensional ensemble empirical mode decomposition. At each scale, principal component analysis (PCA) is implemented for feature extraction. By visualizing the loading vectors of PCA, the important building structures are highlighted. Compared with principal component thermography that applies PCA directly to raw thermal data, the proposed multiscale analysis method is able to zoom in on different types of structural features.

Robust Principal Component Thermography for Defect Detection in Composites.

Pulsed Thermography (PT) data are usually affected by noise and as such most of the research effort in the last few years has been directed towards the development of advanced signal processing methods to improve defect detection. Among the numerous techniques that have been proposed, principal component thermography (PCT)-based on principal component analysis (PCA)-is one of the most effective in terms of defect contrast enhancement and data compression. However, it is well-known that PCA can be significantly affected in the presence of corrupted data (e.g., noise and outliers). Robust PCA (RPCA) has been recently proposed as an alternative statistical method that handles noisy data more properly by decomposing the input data into a low-rank matrix and a sparse matrix. We propose to process PT data by RPCA instead of PCA in order to improve defect detectability. The performance of the resulting approach, Robust Principal Component Thermography (RPCT)-based on RPCA, was evaluated with respect to PCT-based on PCA, using a CFRP sample containing artificially produced defects. We compared results quantitatively based on two metrics, Contrast-to-Noise Ratio (CNR), for defect detection capabilities, and the Jaccard similarity coefficient, for defect segmentation potential. CNR results were on average 40% higher for RPCT than for PCT, and the Jaccard index was slightly higher for RPCT (0.7395) than for PCT (0.7010). In terms of computational time, however, PCT was 11.5 times faster than RPCT. Further investigations are needed to assess RPCT performance on a wider range of materials and to optimize computational time.

Evaluation and Selection of Video Stabilization Techniques for UAV-Based Active Infrared Thermography Application.

Unmanned Aerial Vehicles (UAVs) that can fly around an aircraft carrying several sensors, e.g., thermal and optical cameras, to inspect the parts of interest without removing them can have significant impact in reducing inspection time and cost. One of the main challenges in the UAV based active InfraRed Thermography (IRT) inspection is the UAV's unexpected motions. Since active thermography is mainly concerned with the analysis of thermal sequences, unexpected motions can disturb the thermal profiling and cause data misinterpretation especially for providing an automated process pipeline of such inspections. Additionally, in the scenarios where post-analysis is intended to be applied by an inspector, the UAV's unexpected motions can increase the risk of human error, data misinterpretation, and incorrect characterization of possible defects. Therefore, post-processing is required to minimize/eliminate such undesired motions using digital video stabilization techniques. There are number of video stabilization algorithms that are readily available; however, selecting the best suited one is also challenging. Therefore, this paper evaluates video stabilization algorithms to minimize/mitigate undesired UAV motion and proposes a simple method to find the best suited stabilization algorithm as a fundamental first step towards a fully operational UAV-IRT inspection system.

Introduction of Deep Learning in Thermographic Monitoring of Cultural Heritage and Improvement by Automatic Thermogram Pre-Processing Algorithms.

The monitoring of heritage objects is necessary due to their continuous deterioration over time. Therefore, the joint use of the most up-to-date inspection techniques with the most innovative data processing algorithms plays an important role to apply the required prevention and conservation tasks in each case study. InfraRed Thermography (IRT) is one of the most used Non-Destructive Testing (NDT) techniques in the cultural heritage field due to its advantages in the analysis of delicate objects (i.e., undisturbed, non-contact and fast inspection of large surfaces) and its continuous evolution in both the acquisition and the processing of the data acquired. Despite the good qualitative and quantitative results obtained so far, the lack of automation in the IRT data interpretation predominates, with few automatic analyses that are limited to specific conditions and the technology of the thermographic camera. Deep Learning (DL) is a data processor with a versatile solution for highly automated analysis. Then, this paper introduces the latest state-of-the-art DL model for instance segmentation, Mask Region-Convolution Neural Network (Mask R-CNN), for the automatic detection and segmentation of the position and area of different surface and subsurface defects, respectively, in two different artistic objects belonging to the same family: Marquetry. For that, active IRT experiments are applied to each marquetry. The thermal image sequences acquired are used as input dataset in the Mask R-CNN learning process. Previously, two automatic thermal image pre-processing algorithms based on thermal fundamentals are applied to the acquired data in order to improve the contrast between defective and sound areas. Good detection and segmentation results are obtained regarding state-of-the-art IRT data processing algorithms, which experience difficulty in identifying the deepest defects in the tests. In addition, the performance of the Mask R-CNN is improved by the prior application of the proposed pre-processing algorithms.

Enhanced Infrared Sparse Pattern Extraction and Usage for Impact Evaluation of Basalt-Carbon Hybrid Composites by Pulsed Thermography.

Nowadays, infrared thermography, as a widely used non-destructive testing method, is increasingly studied for impact evaluation of composite structures. Sparse pattern extraction is attracting increasing attention as an advanced post-processing method. In this paper, an enhanced sparse pattern extraction framework is presented for thermographic sequence processing and defect detection. This framework adapts cropping operator and typical component extraction as a preprocessing step to reduce the dimensions of raw data and applies sparse pattern extraction algorithms to enhance the contrast on the defect area. Different cases are studied involving several defects in four basalt-carbon hybrid fiber-reinforced polymer composite laminates. Finally, comparative analysis with intensity distribution is carried out to verify the effectiveness of contrast enhancement using this framework.

Detecting Vasodilation as Potential Diagnostic Biomarker in Breast Cancer Using Deep Learning-Driven Thermomics.

Breast cancer is the most common cancer in women. Early diagnosis improves outcome and survival, which is the cornerstone of breast cancer treatment. Thermography has been utilized as a complementary diagnostic technique in breast cancer detection. Artificial intelligence (AI) has the capacity to capture and analyze the entire concealed information in thermography. In this study, we propose a method to potentially detect the immunohistochemical response to breast cancer by finding thermal heterogeneous patterns in the targeted area. In this study for breast cancer screening 208 subjects participated and normal and abnormal (diagnosed by mammography or clinical diagnosis) conditions were analyzed. High-dimensional deep thermomic features were extracted from the ResNet-50 pre-trained model from low-rank thermal matrix approximation using sparse principal component analysis. Then, a sparse deep autoencoder designed and trained for such data decreases the dimensionality to 16 latent space thermomic features. A random forest model was used to classify the participants. The proposed method preserves thermal heterogeneity, which leads to successful classification between normal and abnormal subjects with an accuracy of 78.16% (73.3-81.07%). By non-invasively capturing a thermal map of the entire tumor, the proposed method can assist in screening and diagnosing this malignancy. These thermal signatures may preoperatively stratify the patients for personalized treatment planning and potentially monitor the patients during treatment.

Thermal imaging dataset from composite material academic samples inspected by pulsed thermography.

This paper presents a thermal imaging dataset from composite material samples (carbon and glass fiber reinforced plastic) that were inspected by pulsed thermography with the goal of detecting and characterizing subsurface defective zones (Teflon inserts representing delaminations between plies). The pulsed thermography experiment was applied to 6 academic plates (inspected from both sides) all having the dimensions of 300 mm x 300 mm x 2 mm and same distribution of defects but made of different materials: three plates on carbon fiber-reinforced plastic (CFRP) and three plates made on glass fiber reinforced plastic (GFRP) specimens with three different geometries: planar, curved and trapezoidal. Each plate contains 25 inserts having length/depth ratios between 1.7 and 75. Two FX60 BALCAR photographic flashes (6.2 kJ per flash) were used to generate the heat pulse (2 ms duration), an X6900 FLIR infrared camera using ResearchIR software to record the thermal images and a custom-built software/control unit to synchronize data recording with pulse generation. Finally, the dataset proposed consists of 12 sequences of approximately 2000 images of 512 × 512 pixels each.

Comparison of Cooled and Uncooled IR Sensors by Means of Signal-to-Noise Ratio for NDT Diagnostics of Aerospace Grade Composites.

This work aims to address the effectiveness and challenges of non-destructive testing (NDT) by active infrared thermography (IRT) for the inspection of aerospace-grade composite samples and seeks to compare uncooled and cooled thermal cameras using the signal-to-noise ratio (SNR) as a performance parameter. It focuses on locating impact damages and optimising the results using several signal processing techniques. The work successfully compares both types of cameras using seven different SNR definitions, to understand if a lower-resolution uncooled IR camera can achieve an acceptable NDT standard. Due to most uncooled cameras being small, lightweight, and cheap, they are more accessible to use on an unmanned aerial vehicle (UAV). The concept of using a UAV for NDT on a composite wing is explored, and the UAV is also tracked using a localisation system to observe the exact movement in millimetres and how it affects the thermal data. It was observed that an NDT UAV can access difficult areas and, therefore, can be suggested for significant reduction of time and cost.

Validation of Anatomical Sites for the Measurement of Infrared Body Surface Temperature Variation in Response to Handling and Transport.

This study aimed at validating the anatomical sites for the measurement of infrared (IR) body surface temperature as a tool to monitor the pigs' response to handling and transport stress. The selected anatomical sites were the neck (infrared neck temperature-IRNT), rump (infrared rump temperature-IRRT), orbital (infrared orbital temperature-IROT) and behind ears (infrared behind ears temperature-IRBET) regions. A total of 120 pigs were handled from the finishing pen to the loading dock through a handling test course. Two handling types (gentle vs. rough) and number of laps (1 vs. 3) were applied according to a 2 × 2 factorial design. After loading, pigs were transported for 40 min and returned to their home pens. Animal behavior, heart rate, rectal temperature and salivary cortisol concentration were measured for validation. Increased IR body temperature, heart rate and salivary cortisol levels were observed in response to rough handling and longer distance walk (P < 0.05 for all). The greatest correlations were found between IROT and IRBET temperatures and salivary cortisol concentration at the end of the handling test (r = 0.49 and r = 0.50, respectively; P < 0.001 for both). Therefore, IR pig's head surface temperature may be useful for a comprehensive assessment of the physiological response to handling and transport stress.