An End-to-End Deep Learning Framework for Fault Detection in Marine Machinery.

The Industrial Internet of Things has enabled the integration and analysis of vast volumes of data across various industries, with the maritime sector being no exception. Advances in cloud computing and deep learning (DL) are continuously reshaping the industry, particularly in optimizing maritime operations such as Predictive Maintenance (PdM). In this study, we propose a novel DL-based framework focusing on the fault detection task of PdM in marine operations, leveraging time-series data from sensors installed on shipboard machinery. The framework is designed as a scalable and cost-efficient software solution, encompassing all stages from data collection and pre-processing at the edge to the deployment and lifecycle management of DL models. The proposed DL architecture utilizes Graph Attention Networks (GATs) to extract spatio-temporal information from the time-series data and provides explainable predictions through a feature-wise scoring mechanism. Additionally, a custom evaluation metric with real-world applicability is employed, prioritizing both prediction accuracy and the timeliness of fault identification. To demonstrate the effectiveness of our framework, we conduct experiments on three types of open-source datasets relevant to PdM: electrical data, bearing datasets, and data from water circulation experiments.

Insights into Object Semantics: Leveraging Transformer Networks for Advanced Image Captioning.

Image captioning is a technique used to generate descriptive captions for images. Typically, it involves employing a Convolutional Neural Network (CNN) as the encoder to extract visual features, and a decoder model, often based on Recurrent Neural Networks (RNNs), to generate the captions. Recently, the encoder-decoder architecture has witnessed the widespread adoption of the self-attention mechanism. However, this approach faces certain challenges that require further research. One such challenge is that the extracted visual features do not fully exploit the available image information, primarily due to the absence of semantic concepts. This limitation restricts the ability to fully comprehend the content depicted in the image. To address this issue, we present a new image-Transformer-based model boosted with image object semantic representation. Our model incorporates semantic representation in encoder attention, enhancing visual features by integrating instance-level concepts. Additionally, we employ Transformer as the decoder in the language generation module. By doing so, we achieve improved performance in generating accurate and diverse captions. We evaluated the performance of our model on the MS-COCO and novel MACE datasets. The results illustrate that our model aligns with state-of-the-art approaches in terms of caption generation.

A Scalable Test Suite for Continuous Dynamic Multiobjective Optimization.

Dynamic multiobjective optimization (DMO) has gained increasing attention in recent years. Test problems are of great importance in order to facilitate the development of advanced algorithms that can handle dynamic environments well. However, many of the existing dynamic multiobjective test problems have not been rigorously constructed and analyzed, which may induce some unexpected bias when they are used for algorithmic analysis. In this paper, some of these biases are identified after a review of widely used test problems. These include poor scalability of objectives and, more important, problematic overemphasis of static properties rather than dynamics making it difficult to draw accurate conclusion about the strengths and weaknesses of the algorithms studied. A diverse set of dynamics and features is then highlighted that a good test suite should have. We further develop a scalable continuous test suite, which includes a number of dynamics or features that have been rarely considered in literature but frequently occur in real life. It is demonstrated with empirical studies that the proposed test suite is more challenging to the DMO algorithms found in the literature. The test suite can also test algorithms in ways that existing test suites cannot.

DeepMoCap: Deep Optical Motion Capture Using Multiple Depth Sensors and Retro-Reflectors.

In this paper, a marker-based, single-person optical motion capture method (DeepMoCap) is proposed using multiple spatio-temporally aligned infrared-depth sensors and retro-reflective straps and patches (reflectors). DeepMoCap explores motion capture by automatically localizing and labeling reflectors on depth images and, subsequently, on 3D space. Introducing a non-parametric representation to encode the temporal correlation among pairs of colorized depthmaps and 3D optical flow frames, a multi-stage Fully Convolutional Network (FCN) architecture is proposed to jointly learn reflector locations and their temporal dependency among sequential frames. The extracted reflector 2D locations are spatially mapped in 3D space, resulting in robust 3D optical data extraction. The subject's motion is efficiently captured by applying a template-based fitting technique on the extracted optical data. Two datasets have been created and made publicly available for evaluation purposes; one comprising multi-view depth and 3D optical flow annotated images (DMC2.5D), and a second, consisting of spatio-temporally aligned multi-view depth images along with skeleton, inertial and ground truth MoCap data (DMC3D). The FCN model outperforms its competitors on the DMC2.5D dataset using 2D Percentage of Correct Keypoints (PCK) metric, while the motion capture outcome is evaluated against RGB-D and inertial data fusion approaches on DMC3D, outperforming the next best method by 4 . 5 % in total 3D PCK accuracy.

Utilization of evidence theory in the detection of salient regions in successive CT images.

This study presents an integrated approach to locating and presenting the medical practitioner with salient regions in a computed tomography (CT) scan when focusing on the area of the liver. A number of image processing tasks are performed in successive scans to extract areas with a different features than that of the greater part of the organ. In general, these areas do not always correspond to pathological patterns, but may be the result of noise in the scanned image or related to veins passing through the tissue. The result of the algorithm is the original image with a mask indicating these regions, so the attention of the medical practitioner is drawn to them for further examination. The algorithm also calculates a measure of confidence of the system, with respect to the extraction of the salient region, based on the fact that a region with a similar pattern is also located in successive scans. This essentially represents the hypothesis that the volume of both pathological patterns and blood vessels, but not noise patterns, is large enough to be captured in successive scans.

Emotion recognition through facial expression analysis based on a neurofuzzy network.

Extracting and validating emotional cues through analysis of users' facial expressions is of high importance for improving the level of interaction in man machine communication systems. Extraction of appropriate facial features and consequent recognition of the user's emotional state that can be robust to facial expression variations among different users is the topic of this paper. Facial animation parameters (FAPs) defined according to the ISO MPEG-4 standard are extracted by a robust facial analysis system, accompanied by appropriate confidence measures of the estimation accuracy. A novel neurofuzzy system is then created, based on rules that have been defined through analysis of FAP variations both at the discrete emotional space, as well as in the 2D continuous activation-evaluation one. The neurofuzzy system allows for further learning and adaptation to specific users' facial expression characteristics, measured though FAP estimation in real life application of the system, using analysis by clustering of the obtained FAP values. Experimental studies with emotionally expressive datasets, generated in the EC IST ERMIS project indicate the good performance and potential of the developed technologies.

Intelligent initialization of resource allocating RBF networks.

In any neural network system, proper parameter initialization reduces training time and effort, and generally leads to compact modeling of the process under examination, i.e. less complex network structures and better generalization. However, in cases of multi-dimensional data, parameter initialization is both difficult and time consuming. In the proposed scheme a novel, multi-dimensional, unsupervised clustering method is used to properly initialize neural network architectures, focusing on resource allocating networks (RAN); both the hidden and output layer parameters are determined by the output of the clustering process, without the need for any user interference. The main contribution of this work is that the proposed approach leads to network structures that are compact, efficient and achieve best classification results, without the need for manual selection of suitable initial network parameters. The efficiency of the proposed method has been tested on several classes of publicly available data, such as iris, Wisconsin and ionosphere data.