Unsupervised learning-based approach for detecting 3D edges in depth maps.

3D edge features, which represent the boundaries between different objects or surfaces in a 3D scene, are crucial for many computer vision tasks, including object recognition, tracking, and segmentation. They also have numerous real-world applications in the field of robotics, such as vision-guided grasping and manipulation of objects. To extract these features in the noisy real-world depth data, reliable 3D edge detectors are indispensable. However, currently available 3D edge detection methods are either highly parameterized or require ground truth labelling, which makes them challenging to use for practical applications. To this extent, we present a new 3D edge detection approach using unsupervised classification. Our method learns features from depth maps at three different scales using an encoder-decoder network, from which edge-specific features are extracted. These edge features are then clustered using learning to classify each point as an edge or not. The proposed method has two key benefits. First, it eliminates the need for manual fine-tuning of data-specific hyper-parameters and automatically selects threshold values for edge classification. Second, the method does not require any labelled training data, unlike many state-of-the-art methods that require supervised training with extensive hand-labelled datasets. The proposed method is evaluated on five benchmark datasets with single and multi-object scenes, and compared with four state-of-the-art edge detection methods from the literature. Results demonstrate that the proposed method achieves competitive performance, despite not using any labelled data or relying on hand-tuning of key parameters.

Towards reuse and recycling of lithium-ion batteries: tele-robotics for disassembly of electric vehicle batteries.

Disassembly of electric vehicle batteries is a critical stage in recovery, recycling and re-use of high-value battery materials, but is complicated by limited standardisation, design complexity, compounded by uncertainty and safety issues from varying end-of-life condition. Telerobotics presents an avenue for semi-autonomous robotic disassembly that addresses these challenges. However, it is suggested that quality and realism of the user's haptic interactions with the environment is important for precise, contact-rich and safety-critical tasks. To investigate this proposition, we demonstrate the disassembly of a Nissan Leaf 2011 module stack as a basis for a comparative study between a traditional asymmetric haptic-"cobot" master-slave framework and identical master and slave cobots based on task completion time and success rate metrics. We demonstrate across a range of disassembly tasks a time reduction of 22%-57% is achieved using identical cobots, yet this improvement arises chiefly from an expanded workspace and 1:1 positional mapping, and suffers a 10%-30% reduction in first attempt success rate. For unbolting and grasping, the realism of force feedback was comparatively less important than directional information encoded in the interaction, however, 1:1 force mapping strengthened environmental tactile cues for vacuum pick-and-place and contact cutting tasks.

Real-time nonlinear parameter estimation and tracking control of unmanned aerial vehicles in closed-loop.

The real-time unknown parameter estimation and adaptive tracking control problems are investigated in this paper for a six degrees of freedom (6-DOF) of under-actuated quadrotor unmanned aerial vehicle (UAV). A virtual proportional derivative (PD) controller is designed to maintain the translational dynamics. Two adaptive schemes are developed to handle the attitude dynamics of the UAV with several unknown parameters. In the beginning, a classical adaptive scheme (CAS) using the certainty equivalence principle is proposed and designed. The idea is to design a controller for an ideal situation by assuming the unknown parameters were known. Then the unknown parameters are replaced by their estimation. A theoretical analysis is provided to ensure the trajectory tracking of the adaptive controller. However, an inherent drawback of this scheme is that there is no guarantee for the estimated parameters to converge to the actual values. To address this issue, a new adaptive scheme (NAS) is developed as the next step by adding a continuously differentiable function to the control structure. The proposed technique guarantees handling of the parametric uncertainties with an appropriate design manifold. A rigorous analytical proof, numerical simulation analyses, and experimental validation are presented to show the effectiveness of the proposed control design.

Local Community Detection Algorithm Based on Alternating Strategy of Strong Fusion and Weak Fusion.

Existing fusion-based local community detection algorithms have achieved good results. However, when assigning a node to a community, similarity functions are sometimes used, which only use node information, while ignoring connection information within the community. These algorithms sometimes fail to find influential nodes, which eventually leads to the failure to find a complete local community. To address these problems, a new local community detection algorithm is proposed in this article. Two strategies, of strong fusion followed by weak fusion, are used alternately to fuse nodes. Compared with using two fusion strategies alone, the alternating loop method can improve the solution of the algorithm in each stage. In strong fusion, we propose a new membership function that considers both node information and connection information in the local community. This improves the quality of the fused node while preserving the structure of the current community. In weak fusion, we propose a parameter-based similarity measure, which can detect influential nodes for a local community. We also propose a local community evaluation metric, which does not require true division to determine the optimal local community under different parameters. Experiments, compared to six state-of-the-art algorithms, show that the proposed algorithm improves accuracy and stability, and also demonstrate the effectiveness of the new local community evaluation metrics in parameter selection.

Dynamic Immunization Node Model for Complex Networks Based on Community Structure and Threshold.

In the information age of big data, and increasingly large and complex networks, there is a growing challenge of understanding how best to restrain the spread of harmful information, for example, a computer virus. Establishing models of propagation and node immunity are important parts of this problem. In this article, a dynamic node immune model, based on the community structure and threshold (NICT), is proposed. First, a network model is established, which regards nodes carrying harmful information as new nodes in the network. The method of establishing the edge between the new node and the original node can be changed according to the needs of different networks. The propagation probability between nodes is determined by using community structure information and a similarity function between nodes. Second, an improved immune gain, based on the propagation probability of the community structure and node similarity, is proposed. The improved immune gain value is calculated for neighbors of the infected node at each time step, and the node is immunized according to the hand-coded parameter: immune threshold. This can effectively prevent invalid or insufficient immunization at each time step. Finally, an evaluation index, considering both the number of immune nodes and the number of infected nodes at each time step, is proposed. The immune effect of nodes can be evaluated more effectively. The results of network immunization experiments, on eight real networks, suggest that the proposed method can deliver better network immunization than several other well-known methods from the literature.

An Automatic and Optimal MPA Design Method.

Raw polarimetric images are captured by a focal plane polarimeter which is covered by a micro-polarizer array (MPA). The design of the MPA plays a crucial role in polarimetric imaging. MPAs are predominantly designed according to expert engineering experience and rules of thumb. Typically, only one optimization criterion, maximizing bandwidth, is used to design the MPA. To select a design, an exhaustive search is usually performed on a very limited set of available polarizing patterns, which must be constrained in order to make the search tractable. In contrast, this paper proposes a fully automated and optimal MPA design method (AO-MPA) which generates significantly improved MPAs. Instead of the single criterion of bandwidth, we propose six design principles, and show how they can be utilized to mutually optimize the MPA design by formulating a tri-objective optimization problem with multiple constraints. A much larger set of possible MPA patterns is rapidly and automatically searched by applying advanced multi-objective optimization techniques. We have tested AO-MPA using two groups of experiments, in which AO-MPA is compared against several other leading MPA design methods, and the patterns generated by AO-MPA are compared against state-of-the-art patterns from the literature. The results, obtained using a public benchmark dataset, show that the AO-MPA method is very computationally efficient, and can find all optimal MPA patterns for all array sizes. Moreover, for each size, AO-MPA obtains all optimal layouts simultaneously. AO-MPA generates designs which require fewer polarization orientations, while also yielding better performance in estimating intensity measurements, Stokes vector and the degree of linear polarization. This results in MPAs which are easier to manufacture while also being more robust to noise.

Vision-Guided MPC for Robotic Path Following Using Learned Memory-Augmented Model.

The control of the interaction between the robot and environment, following a predefined geometric surface path with high accuracy, is a fundamental problem for contact-rich tasks such as machining, polishing, or grinding. Flexible path-following control presents numerous applications in emerging industry fields such as disassembly and recycling, where the control system must adapt to a range of dissimilar object classes, where the properties of the environment are uncertain. We present an end-to-end framework for trajectory-independent robotic path following for contact-rich tasks in the presence of parametric uncertainties. We formulate a combination of model predictive control with image-based path planning and real-time visual feedback, based on a learned state-space dynamic model. For modeling the dynamics of the robot-environment system during contact, we introduce the application of the differentiable neural computer, a type of memory augmented neural network (MANN). Although MANNs have been as yet unexplored in a control context, we demonstrate a reduction in RMS error of ∼ 21.0% compared with an equivalent Long Short-Term Memory (LSTM) architecture. Our framework was validated in simulation, demonstrating the ability to generalize to materials previously unseen in the training dataset.

Publisher Correction: Recycling lithium-ion batteries from electric vehicles.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Let's Push Things Forward: A Survey on Robot Pushing.

As robots make their way out of factories into human environments, outer space, and beyond, they require the skill to manipulate their environment in multifarious, unforeseeable circumstances. With this regard, pushing is an essential motion primitive that dramatically extends a robot's manipulation repertoire. In this work, we review the robotic pushing literature. While focusing on work concerned with predicting the motion of pushed objects, we also cover relevant applications of pushing for planning and control. Beginning with analytical approaches, under which we also subsume physics engines, we then proceed to discuss work on learning models from data. In doing so, we dedicate a separate section to deep learning approaches which have seen a recent upsurge in the literature. Concluding remarks and further research perspectives are given at the end of the paper.

Simultaneous Material Segmentation and 3D Reconstruction in Industrial Scenarios.

Recognizing material categories is one of the core challenges in robotic nuclear waste decommissioning. All nuclear waste should be sorted and segregated according to its materials, and then different disposal post-process can be applied. In this paper, we propose a novel transfer learning approach to learn boundary-aware material segmentation from a meta-dataset and weakly annotated data. The proposed method is data-efficient, leveraging a publically available dataset for general computer vision tasks and coarsely labeled material recognition data, with only a limited number of fine pixel-wise annotations required. Importantly, our approach is integrated with a Simultaneous Localization and Mapping (SLAM) system to fuse the per-frame understanding delicately into a 3D global semantic map to facilitate robot manipulation in self-occluded object heaps or robot navigation in disaster zones. We evaluate the proposed method on the Materials in Context dataset over 23 categories and that our integrated system delivers quasi-real-time 3D semantic mapping with high-resolution images. The trained model is also verified in an industrial environment as part of the EU RoMaNs project, and promising qualitative results are presented. A video demo and the newly generated data can be found at the project website (Supplementary Material).

The Grasp Strategy of a Robot Passer Influences Performance and Quality of the Robot-Human Object Handover.

Task-aware robotic grasping is critical if robots are to successfully cooperate with humans. The choice of a grasp is multi-faceted; however, the task to perform primes this choice in terms of hand shaping and placement on the object. This grasping strategy is particularly important for a robot companion, as it can potentially hinder the success of the collaboration with humans. In this work, we investigate how different grasping strategies of a robot passer influence the performance and the perceptions of the interaction of a human receiver. Our findings suggest that a grasping strategy that accounts for the subsequent task of the receiver improves substantially the performance of the human receiver in executing the subsequent task. The time to complete the task is reduced by eliminating the need of a post-handover re-adjustment of the object. Furthermore, the human perceptions of the interaction improve when a task-oriented grasping strategy is adopted. The influence of the robotic grasp strategy increases as the constraints induced by the object's affordances become more restrictive. The results of this work can benefit the wider robotics community, with application ranging from industrial to household human-robot interaction for cooperative and collaborative object manipulation.

Semi-Supervised Graph Regularized Deep NMF With Bi-Orthogonal Constraints for Data Representation.

Semi-supervised non-negative matrix factorization (NMF) exploits the strengths of NMF in effectively learning local information contained in data and is also able to achieve effective learning when only a small fraction of data is labeled. NMF is particularly useful for dimensionality reduction of high-dimensional data. However, the mapping between the low-dimensional representation, learned by semi-supervised NMF, and the original high-dimensional data contains complex hierarchical and structural information, which is hard to extract by using only single-layer clustering methods. Therefore, in this article, we propose a new deep learning method, called semi-supervised graph regularized deep NMF with bi-orthogonal constraints (SGDNMF). SGDNMF learns a representation from the hidden layers of a deep network for clustering, which contains varied and unknown attributes. Bi-orthogonal constraints on two factor matrices are introduced into our SGDNMF model, which can make the solution unique and improve clustering performance. This improves the effect of dimensionality reduction because it only requires a small fraction of data to be labeled. In addition, SGDNMF incorporates dual-hypergraph Laplacian regularization, which can reinforce high-order relationships in both data and feature spaces and fully retain the intrinsic geometric structure of the original data. This article presents the details of the SGDNMF algorithm, including the objective function and the iterative updating rules. Empirical experiments on four different data sets demonstrate state-of-the-art performance of SGDNMF in comparison with six other prominent algorithms.

Recycling lithium-ion batteries from electric vehicles.

Rapid growth in the market for electric vehicles is imperative, to meet global targets for reducing greenhouse gas emissions, to improve air quality in urban centres and to meet the needs of consumers, with whom electric vehicles are increasingly popular. However, growing numbers of electric vehicles present a serious waste-management challenge for recyclers at end-of-life. Nevertheless, spent batteries may also present an opportunity as manufacturers require access to strategic elements and critical materials for key components in electric-vehicle manufacture: recycled lithium-ion batteries from electric vehicles could provide a valuable secondary source of materials. Here we outline and evaluate the current range of approaches to electric-vehicle lithium-ion battery recycling and re-use, and highlight areas for future progress.

Automatic Detection of Myocontrol Failures Based upon Situational Context Information.

Myoelectric control systems for assistive devices are still unreliable. The user's input signals can become unstable over time due to e.g. fatigue, electrode displacement, or sweat. Hence, such controllers need to be constantly updated and heavily rely on user feedback. In this paper, we present an automatic failure detection method which learns when plausible predictions become unreliable and model updates are necessary. Our key insight is to enhance the control system with a set of generative models that learn sensible behaviour for a desired task from human demonstration. We illustrate our approach on a grasping scenario in Virtual Reality, in which the user is asked to grasp a bottle on a table. From demonstration our model learns the reach-to-grasp motion from a resting position to two grasps (power grasp and tridigital grasp) and how to predict the most adequate grasp from local context, e.g. tridigital grasp on the bottle cap or around the bottleneck. By measuring the error between new grasp attempts and the model prediction, the system can effectively detect which input commands do not reflect the user's intention. We evaluated our model in two cases: i) with both position and rotation information of the wrist pose, and ii) with only rotational information. Our results show that our approach detects statistically highly significant differences in error distributions with p<0.001 between successful and failed grasp attempts in both cases.

Dense RGB-D Semantic Mapping with Pixel-Voxel Neural Network.

In this paper, a novel Pixel-Voxel network is proposed for dense 3D semantic mapping, which can perform dense 3D mapping while simultaneously recognizing and labelling the semantic category each point in the 3D map. In our approach, we fully leverage the advantages of different modalities. That is, the PixelNet can learn the high-level contextual information from 2D RGB images, and the VoxelNet can learn 3D geometrical shapes from the 3D point cloud. Unlike the existing architecture that fuses score maps from different modalities with equal weights, we propose a softmax weighted fusion stack that adaptively learns the varying contributions of PixelNet and VoxelNet and fuses the score maps according to their respective confidence levels. Our approach achieved competitive results on both the SUN RGB-D and NYU V2 benchmarks, while the runtime of the proposed system is boosted to around 13 Hz, enabling near-real-time performance using an i7 eight-cores PC with a single Titan X GPU.

Non-Negative Spectral Learning and Sparse Regression-Based Dual-Graph Regularized Feature Selection.

Feature selection is an important approach for reducing the dimension of high-dimensional data. In recent years, many feature selection algorithms have been proposed, but most of them only exploit information from the data space. They often neglect useful information contained in the feature space, and do not make full use of the characteristics of the data. To overcome this problem, this paper proposes a new unsupervised feature selection algorithm, called non-negative spectral learning and sparse regression-based dual-graph regularized feature selection (NSSRD). NSSRD is based on the feature selection framework of joint embedding learning and sparse regression, but extends this framework by introducing the feature graph. By using low dimensional embedding learning in both data space and feature space, NSSRD simultaneously exploits the geometric information of both spaces. Second, the algorithm uses non-negative constraints to constrain the low-dimensional embedding matrix of both feature space and data space, ensuring that the elements in the matrix are non-negative. Third, NSSRD unifies the embedding matrix of the feature space and the sparse transformation matrix. To ensure the sparsity of the feature array, the sparse transformation matrix is constrained using the -norm. Thus feature selection can obtain accurate discriminative information from these matrices. Finally, NSSRD uses an iterative and alternative updating rule to optimize the objective function, enabling it to select the representative features more quickly and efficiently. This paper explains the objective function, the iterative updating rules and a proof of convergence. Experimental results show that NSSRD is significantly more effective than several other feature selection algorithms from the literature, on a variety of test data.

Robust Fusion of Color and Depth Data for RGB-D Target Tracking Using Adaptive Range-Invariant Depth Models and Spatio-Temporal Consistency Constraints.

This paper presents a novel robust method for single target tracking in RGB-D images, and also contributes a substantial new benchmark dataset for evaluating RGB-D trackers. While a target object's color distribution is reasonably motion-invariant, this is not true for the target's depth distribution, which continually varies as the target moves relative to the camera. It is therefore nontrivial to design target models which can fully exploit (potentially very rich) depth information for target tracking. For this reason, much of the previous RGB-D literature relies on color information for tracking, while exploiting depth information only for occlusion reasoning. In contrast, we propose an adaptive range-invariant target depth model, and show how both depth and color information can be fully and adaptively fused during the search for the target in each new RGB-D image. We introduce a new, hierarchical, two-layered target model (comprising local and global models) which uses spatio-temporal consistency constraints to achieve stable and robust on-the-fly target relearning. In the global layer, multiple features, derived from both color and depth data, are adaptively fused to find a candidate target region. In ambiguous frames, where one or more features disagree, this global candidate region is further decomposed into smaller local candidate regions for matching to local-layer models of small target parts. We also note that conventional use of depth data, for occlusion reasoning, can easily trigger false occlusion detections when the target moves rapidly toward the camera. To overcome this problem, we show how combining target information with contextual information enables the target's depth constraint to be relaxed. Our adaptively relaxed depth constraints can robustly accommodate large and rapid target motion in the depth direction, while still enabling the use of depth data for highly accurate reasoning about occlusions. For evaluation, we introduce a new RGB-D benchmark dataset with per-frame annotated attributes and extensive bias analysis. Our tracker is evaluated using two different state-of-the-art methodologies, VOT and object tracking benchmark, and in both cases it significantly outperforms four other state-of-the-art RGB-D trackers from the literature.

Nonnegative Matrix Factorization with Rank Regularization and Hard Constraint.

Nonnegative matrix factorization (NMF) is well known to be an effective tool for dimensionality reduction in problems involving big data. For this reason, it frequently appears in many areas of scientific and engineering literature. This letter proposes a novel semisupervised NMF algorithm for overcoming a variety of problems associated with NMF algorithms, including poor use of prior information, negative impact on manifold structure of the sparse constraint, and inaccurate graph construction. Our proposed algorithm, nonnegative matrix factorization with rank regularization and hard constraint (NMFRC), incorporates label information into data representation as a hard constraint, which makes full use of prior information. NMFRC also measures pairwise similarity according to geodesic distance rather than Euclidean distance. This results in more accurate measurement of pairwise relationships, resulting in more effective manifold information. Furthermore, NMFRC adopts rank constraint instead of norm constraints for regularization to balance the sparseness and smoothness of data. In this way, the new data representation is more representative and has better interpretability. Experiments on real data sets suggest that NMFRC outperforms four other state-of-the-art algorithms in terms of clustering accuracy.

Improved Memetic Algorithm Based on Route Distance Grouping for Multiobjective Large Scale Capacitated Arc Routing Problems.

The capacitated arc routing problem (CARP) has attracted considerable attention from researchers due to its broad potential for social applications. This paper builds on, and develops beyond, the cooperative coevolutionary algorithm based on route distance grouping (RDG-MAENS), recently proposed by Mei et al. Although Mei's method has proved superior to previous algorithms, we discuss several remaining drawbacks and propose solutions to overcome them. First, although RDG is used in searching for potential better solutions, the solution generated from the decomposed problem at each generation is not the best one, and the best solution found so far is not used for solving the current generation. Second, to determine which sub-population the individual belongs to simply according to the distance can lead to an imbalance in the number of the individuals among different sub-populations and the allocation of resources. Third, the method of Mei et al. was only used to solve single-objective CARP. To overcome the above issues, this paper proposes improving RDG-MAENS by updating the solutions immediately and applying them to solve the current solution through areas shared, and then according to the magnitude of the vector of the route direction, and a fast and simple allocation scheme is proposed to determine which decomposed problem the route belongs to. Finally, we combine the improved algorithm with an improved decomposition-based memetic algorithm to solve the multiobjective large scale CARP (LSCARP). Experimental results suggest that the proposed improved algorithm can achieve better results on both single-objective LSCARP and multiobjective LSCARP.