Shaped-Based Tightly Coupled IMU/Camera Object-Level SLAM.

Object-level simultaneous localization and mapping (SLAM) has gained popularity in recent years since it can provide a means for intelligent robot-to-environment interactions. However, most of these methods assume that the distribution of the errors is Gaussian. This assumption is not valid under many circumstances. Further, these methods use a delayed initialization of the objects in the map. During this delayed period, the solution relies on the motion model provided by an inertial measurement unit (IMU). Unfortunately, the errors tend to accumulate quickly due to the dead-reckoning nature of these motion models. Finally, the current solutions depend on a set of salient features on the object's surface and not the object's shape. This research proposes an accurate object-level solution to the SLAM problem with a 4.1 to 13.1 cm error in the position (0.005 to 0.021 of the total path). The developed solution is based on Rao-Blackwellized Particle Filtering (RBPF) that does not assume any predefined error distribution for the parameters. Further, the solution relies on the shape and thus can be used for objects that lack texture on their surface. Finally, the developed tightly coupled IMU/camera solution is based on an undelayed initialization of the objects in the map.

Radar/INS Integration and Map Matching for Land Vehicle Navigation in Urban Environments.

Autonomous navigation requires multi-sensor fusion to achieve a high level of accuracy in different environments. Global navigation satellite system (GNSS) receivers are the main components in most navigation systems. However, GNSS signals are subject to blockage and multipath effects in challenging areas, e.g., tunnels, underground parking, and downtown or urban areas. Therefore, different sensors, such as inertial navigation systems (INSs) and radar, can be used to compensate for GNSS signal deterioration and to meet continuity requirements. In this paper, a novel algorithm was applied to improve land vehicle navigation in GNSS-challenging environments through radar/INS integration and map matching. Four radar units were utilized in this work. Two units were used to estimate the vehicle's forward velocity, and the four units were used together to estimate the vehicle's position. The integrated solution was estimated in two steps. First, the radar solution was fused with an INS through an extended Kalman filter (EKF). Second, map matching was used to correct the radar/INS integrated position using OpenStreetMap (OSM). The developed algorithm was evaluated using real data collected in Calgary's urban area and downtown Toronto. The results show the efficiency of the proposed method, which had a horizontal position RMS error percentage of less than 1% of the distance traveled for three minutes of a simulated GNSS outage.

Accelerometer-Based Wheel Odometer for Kinematics Determination.

Various high budget industries that utilize wheel-based vehicles rely on wheel odometry as an integral aspect of their navigation process. This research introduces a low-cost alternative for typical wheel encoders that are typically used to determine the on-track speed of vehicles. The proposed system is referred to as an Accelerometer-based Wheel Odometer for Kinematics determination (AWOK). The AWOK system comprises just a single axis accelerometer mounted radially at the center of any given wheel. The AWOK system can provide direct distances instead of just velocities, which are provided by typical wheel speedometers. Hence, the AWOK system is advantageous in comparison to typical wheel odometers. Besides, the AWOK system comprises a simple assembly with a highly efficient data processing algorithm. Additionally, the AWOK system provides a high capacity to handle high dynamics in comparison to similar approaches found in previous related work. Furthermore, the AWOK system is not affected by the inherited stochastic errors in micro-machined electro-mechanical systems (MEMS) inertial sensors, whether short-term or long-term errors. Above all, the AWOK system reported a relative accuracy of 0.15% in determining the distance covered by a car.

Optical and Mass Flow Sensors for Aiding Vehicle Navigation in GNSS Denied Environment.

Nowadays, autonomous vehicles have achieved a lot of research interest regarding the navigation, the surrounding environmental perception, and control. Global Navigation Satellite System/Inertial Navigation System (GNSS/INS) is one of the significant components of any vehicle navigation system. However, GNSS has limitations in some operating scenarios such as urban regions and indoor environments where the GNSS signal suffers from multipath or outage. On the other hand, INS standalone navigation solution degrades over time due to the INS errors. Therefore, a modern vehicle navigation system depends on integration between different sensors to aid INS for mitigating its drift during GNSS signal outage. However, there are some challenges for the aiding sensors related to their high price, high computational costs, and environmental and weather effects. This paper proposes an integrated aiding navigation system for vehicles in an indoor environment (e.g., underground parking). This proposed system is based on optical flow and multiple mass flow sensors integrations to aid the low-cost INS by providing the navigation extended Kalman filter (EKF) with forward velocity and change of heading updates to enhance the vehicle navigation. The optical flow is computed for frames taken using a consumer portable device (CPD) camera mounted in the upward-looking direction to avoid moving objects in front of the camera and to exploit the typical features of the underground parking or tunnels such as ducts and pipes. On the other hand, the multiple mass flow sensors measurements are modeled to provide forward velocity information. Moreover, a mass flow differential odometry is proposed where the vehicle change of heading is estimated from the multiple mass flow sensors measurements. This integrated aiding system can be used for unmanned aerial vehicles (UAV) and land vehicle navigations. However, the experimental results are implemented for land vehicles through the integration of CPD with mass flow sensors to aid the navigation system.

LocSpeck: A Collaborative and Distributed Positioning System for Asymmetric Nodes Based on UWB Ad-Hoc Network and Wi-Fi Fingerprinting.

This paper presents LocSpeck, a collaborative and distributed indoor positioning system for dynamic nodes connected using an ad-hoc network, based on inter-node relative range measurements and Wi-Fi fingerprinting. The proposed system operates using peer-to-peer range measurements and does not need ultra-wideband (UWB) fixed anchor, nor it needs a predefined network topology. The nodes could be asymmetric in terms of the available sensors onboard, the computational resources, and the power capacity. This asymmetry adversely affects the positioning performance of the weaker nodes. Collaboration between different nodes is achieved through a distributed estimator without the need of a single centralized computing element. The ranging measurement component of the system is based on the DW1000 UWB transceiver chip from Decawave, which is attached to a set of smartphones equipped with asymmetric sensors. The distributed positioning filter fuses, locally on each node, the relative range measurements, the reading from the internal sensors, and the Wi-Fi received signal strength indicator (RSSI) readings to obtain an estimate of the position of each node. The described system does not depend on fixed UWB anchors and supports online addition and removal of nodes and dynamic node role assignment, either as an anchor or as a rover. The performance of the system is evaluated by real-world test scenarios using a set of four smartphones navigating an indoor environment on foot. The performance is compared to that of a commercial UWB-based system. The results presented in this paper show that weak mobile nodes, in terms of available positioning sensors, can benefit from collaboration with other nearby nodes.

Particle Imaging Velocimetry Gyroscope.

Inertial measurement units (IMUs) are typically classified as per the performance of the gyroscopes within each system. Consequently, it is critical for a system to have a low bias instability to have better performance. Nonetheless, there is no IMU available commercially that does not actually suffer from bias-instability, even for the navigation grade IMUs. This paper introduces the proposition of a novel fluid-based gyroscope, which is referred to hereafter as a particle imaging velocimetry gyroscope (PIVG). The main advantages of the PIVG include being nearly drift-free, a high signal-to-noise ratio (SNR) in comparison to commercially available high-end gyroscopes, and its low cost.

Steering Angle Assisted Vehicular Navigation Using Portable Devices in GNSS-Denied Environments.

Recently, land vehicle navigation, and especially by the use of low-cost sensors, has been the object of a huge level of research interest. Consumer Portable Devices (CPDs) such as tablets and smartphones are being widely used by many consumers all over the world. CPDs contain sensors (accelerometers, gyroscopes, magnetometer, etc.) that can be used for many land vehicle applications such as navigation. This paper presents a novel approach for estimating steering wheel angles using CPD accelerometers by attaching CPDs to the steering wheel. The land vehicle change of heading is then computed from the estimated steering wheel angle. The calculated change of heading is used to update the navigation filter to aid the onboard Inertial Measurement Unit (IMU) through the use of an Extended Kalman Filter (EKF) in GNSS-denied environments. Four main factors that may affect the steering wheel angle accuracy are considered and modeled during steering angle estimations: static onboard IMU leveling, inclination angle of the steering wheel, vehicle acceleration, and vehicle inclination. In addition, these factors are assessed for their effects on the final result. Therefore, three methods are proposed for steering angle estimation: non-compensated, partially-compensated, and fully-compensated methods. A road experimental test was carried out using a Pixhawk (PX4) navigation system, iPad Air, and the OBD-II interface. The average Root Mean Square Error (RMSE) of the change of heading estimated by the proposed method was 0.033 rad/s. A navigation solution was estimated while changes of heading and forward velocity updates were used to aid the IMU during different GNSS signal outages. The estimated navigation solution is enhanced when applying the proposed updates to the navigation filter by 91% and 97% for 60 s and 120 s of GNSS signal outage, respectively, compared to the IMU standalone solution.

Wireless Fingerprinting Uncertainty Prediction Based on Machine Learning.

Although wireless fingerprinting has been well researched and widely used for indoor localization, its performance is difficult to quantify. Therefore, when wireless fingerprinting solutions are used as location updates in multi-sensor integration, it is challenging to set their weight accurately. To alleviate this issue, this paper focuses on predicting wireless fingerprinting location uncertainty by given received signal strength (RSS) measurements through the use of machine learning (ML). Two ML methods are used, including an artificial neural network (ANN)-based approach and a Gaussian distribution (GD)-based method. The predicted location uncertainty is evaluated and further used to set the measurement noises in the dead-reckoning/wireless fingerprinting integrated localization extended Kalman filter (EKF). Indoor walking test results indicated the possibility of predicting the wireless fingerprinting uncertainty through ANN the effectiveness of setting measurement noises adaptively in the integrated localization EKF.

Motion Constraints and Vanishing Point Aided Land Vehicle Navigation.

In the typical Inertial Navigation System (INS)/ Global Navigation Satellite System (GNSS) setup for ground vehicle navigation, measures should be taken to maintain the performance when there are GNSS signal outages. Usually, aiding sensors are utilized to reduce the INS drift. A full motion constraint model is developed allowing the online calibration of INS frame with respect to (w.r.t) the motion frame. To obtain better heading and lateral positioning performance, we propose to use of vanishing point (VP) observations of parallel lane markings from a single forward-looking camera to aid the INS. In the VP module, the relative attitude of the camera w.r.t the road frame is derived from the VP coordinates. The state-space model is developed with augmented vertical attitude error state. Finally, the VP module is added to a modified motion constrains module in the Extended Kalman filter (EKF) framework. Simulations and real-world experiments have shown the validity of VP-based method and improved heading and cross-track position accuracy compared with the solution without VP. The proposed method can work jointly with conventional visual odometry to aid INS for better accuracy and robustness.

Radar and Visual Odometry Integrated System Aided Navigation for UAVS in GNSS Denied Environment.

Drones are becoming increasingly significant for vast applications, such as firefighting, and rescue. While flying in challenging environments, reliable Global Navigation Satellite System (GNSS) measurements cannot be guaranteed all the time, and the Inertial Navigation System (INS) navigation solution will deteriorate dramatically. Although different aiding sensors, such as cameras, are proposed to reduce the effect of these drift errors, the positioning accuracy by using these techniques is still affected by some challenges, such as the lack of the observed features, inconsistent matches, illumination, and environmental conditions. This paper presents an integrated navigation system for Unmanned Aerial Vehicles (UAVs) in GNSS denied environments based on a Radar Odometry (RO) and an enhanced Visual Odometry (VO) to handle such challenges since the radar is immune against these issues. The estimated forward velocities of a vehicle from both the RO and the enhanced VO are fused with the Inertial Measurement Unit (IMU), barometer, and magnetometer measurements via an Extended Kalman Filter (EKF) to enhance the navigation accuracy during GNSS signal outages. The RO and VO are integrated into one integrated system to help overcome their limitations, since the RO measurements are affected while flying over non-flat terrain. Therefore, the integration of the VO is important in such scenarios. The experimental results demonstrate the proposed system's ability to significantly enhance the 3D positioning accuracy during the GNSS signal outage.

A New Vegetation Segmentation Approach for Cropped Fields Based on Threshold Detection from Hue Histograms.

Over the last decade, the use of unmanned aerial vehicle (UAV) technology has evolved significantly in different applications as it provides a special platform capable of combining the benefits of terrestrial and aerial remote sensing. Therefore, such technology has been established as an important source of data collection for different precision agriculture (PA) applications such as crop health monitoring and weed management. Generally, these PA applications depend on performing a vegetation segmentation process as an initial step, which aims to detect the vegetation objects in collected agriculture fields’ images. The main result of the vegetation segmentation process is a binary image, where vegetations are presented in white color and the remaining objects are presented in black. Such process could easily be performed using different vegetation indexes derived from multispectral imagery. Recently, to expand the use of UAV imagery systems for PA applications, it was important to reduce the cost of such systems through using low-cost RGB cameras Thus, developing vegetation segmentation techniques for RGB images is a challenging problem. The proposed paper introduces a new vegetation segmentation methodology for low-cost UAV RGB images, which depends on using Hue color channel. The proposed methodology follows the assumption that the colors in any agriculture field image can be distributed into vegetation and non-vegetations colors. Therefore, four main steps are developed to detect five different threshold values using the hue histogram of the RGB image, these thresholds are capable to discriminate the dominant color, either vegetation or non-vegetation, within the agriculture field image. The achieved results for implementing the proposed methodology showed its ability to generate accurate and stable vegetation segmentation performance with mean accuracy equal to 87.29% and standard deviation as 12.5%.

Time Series UAV Image-Based Point Clouds for Landslide Progression Evaluation Applications.

Landslides are major and constantly changing threats to urban landscapes and infrastructure. It is essential to detect and capture landslide changes regularly. Traditional methods for monitoring landslides are time-consuming, costly, dangerous, and the quality and quantity of the data is sometimes unable to meet the necessary requirements of geotechnical projects. This motivates the development of more automatic and efficient remote sensing approaches for landslide progression evaluation. Automatic change detection involving low-altitude unmanned aerial vehicle image-based point clouds, although proven, is relatively unexplored, and little research has been done in terms of accounting for volumetric changes. In this study, a methodology for automatically deriving change displacement rates, in a horizontal direction based on comparisons between extracted landslide scarps from multiple time periods, has been developed. Compared with the iterative closest projected point (ICPP) registration method, the developed method takes full advantage of automated geometric measuring, leading to fast processing. The proposed approach easily processes a large number of images from different epochs and enables the creation of registered image-based point clouds without the use of extensive ground control point information or further processing such as interpretation and image correlation. The produced results are promising for use in the field of landslide research.

Improving the Accuracy of Direct Geo-referencing of Smartphone-Based Mobile Mapping Systems Using Relative Orientation and Scene Geometric Constraints.

This paper introduces a new method which facilitate the use of smartphones as a handheld low-cost mobile mapping system (MMS). Smartphones are becoming more sophisticated and smarter and are quickly closing the gap between computers and portable tablet devices. The current generation of smartphones are equipped with low-cost GPS receivers, high-resolution digital cameras, and micro-electro mechanical systems (MEMS)-based navigation sensors (e.g., accelerometers, gyroscopes, magnetic compasses, and barometers). These sensors are in fact the essential components for a MMS. However, smartphone navigation sensors suffer from the poor accuracy of global navigation satellite System (GNSS), accumulated drift, and high signal noise. These issues affect the accuracy of the initial Exterior Orientation Parameters (EOPs) that are inputted into the bundle adjustment algorithm, which then produces inaccurate 3D mapping solutions. This paper proposes new methodologies for increasing the accuracy of direct geo-referencing of smartphones using relative orientation and smartphone motion sensor measurements as well as integrating geometric scene constraints into free network bundle adjustment. The new methodologies incorporate fusing the relative orientations of the captured images and their corresponding motion sensor measurements to improve the initial EOPs. Then, the geometric features (e.g., horizontal and vertical linear lines) visible in each image are extracted and used as constraints in the bundle adjustment procedure which correct the relative position and orientation of the 3D mapping solution.

A Map/INS/Wi-Fi Integrated System for Indoor Location-Based Service Applications.

In this research, a new Map/INS/Wi-Fi integrated system for indoor location-based service (LBS) applications based on a cascaded Particle/Kalman filter framework structure is proposed. Two-dimension indoor map information, together with measurements from an inertial measurement unit (IMU) and Received Signal Strength Indicator (RSSI) value, are integrated for estimating positioning information. The main challenge of this research is how to make effective use of various measurements that complement each other in order to obtain an accurate, continuous, and low-cost position solution without increasing the computational burden of the system. Therefore, to eliminate the cumulative drift caused by low-cost IMU sensor errors, the ubiquitous Wi-Fi signal and non-holonomic constraints are rationally used to correct the IMU-derived navigation solution through the extended Kalman Filter (EKF). Moreover, the map-aiding method and map-matching method are innovatively combined to constrain the primary Wi-Fi/IMU-derived position through an Auxiliary Value Particle Filter (AVPF). Different sources of information are incorporated through a cascaded structure EKF/AVPF filter algorithm. Indoor tests show that the proposed method can effectively reduce the accumulation of positioning errors of a stand-alone Inertial Navigation System (INS), and provide a stable, continuous and reliable indoor location service.

A Novel Real-Time Reference Key Frame Scan Matching Method.

Unmanned aerial vehicles represent an effective technology for indoor search and rescue operations. Typically, most indoor missions' environments would be unknown, unstructured, and/or dynamic. Navigation of UAVs in such environments is addressed by simultaneous localization and mapping approach using either local or global approaches. Both approaches suffer from accumulated errors and high processing time due to the iterative nature of the scan matching method. Moreover, point-to-point scan matching is prone to outlier association processes. This paper proposes a low-cost novel method for 2D real-time scan matching based on a reference key frame (RKF). RKF is a hybrid scan matching technique comprised of feature-to-feature and point-to-point approaches. This algorithm aims at mitigating errors accumulation using the key frame technique, which is inspired from video streaming broadcast process. The algorithm depends on the iterative closest point algorithm during the lack of linear features which is typically exhibited in unstructured environments. The algorithm switches back to the RKF once linear features are detected. To validate and evaluate the algorithm, the mapping performance and time consumption are compared with various algorithms in static and dynamic environments. The performance of the algorithm exhibits promising navigational, mapping results and very short computational time, that indicates the potential use of the new algorithm with real-time systems.

Map-Based Indoor Pedestrian Navigation Using an Auxiliary Particle Filter.

In this research, a non-infrastructure-based and low-cost indoor navigation method is proposed through the integration of smartphone built-in microelectromechanical systems (MEMS) sensors and indoor map information using an auxiliary particle filter (APF). A cascade structure Kalman particle filter algorithm is designed to reduce the computational burden and improve the estimation speed of the APF by decreasing its update frequency and the number of particles used in this research. In the lower filter (Kalman filter), zero velocity update and non-holonomic constraints are used to correct the error of the inertial navigation-derived solutions. The innovation of the design lies in the combination of upper filter (particle filter) map-matching and map-aiding methods to further constrain the navigation solutions. This proposed navigation method simplifies indoor positioning and makes it accessible to individual and group users, while guaranteeing the system's accuracy. The availability and accuracy of the proposed algorithm are tested and validated through experiments in various practical scenarios.

Design and Implementation of Foot-Mounted Inertial Sensor Based Wearable Electronic Device for Game Play Application.

Wearable electronic devices have experienced increasing development with the advances in the semiconductor industry and have received more attention during the last decades. This paper presents the development and implementation of a novel inertial sensor-based foot-mounted wearable electronic device for a brand new application: game playing. The main objective of the introduced system is to monitor and identify the human foot stepping direction in real time, and coordinate these motions to control the player operation in games. This proposed system extends the utilized field of currently available wearable devices and introduces a convenient and portable medium to perform exercise in a more compelling way in the near future. This paper provides an overview of the previously-developed system platforms, introduces the main idea behind this novel application, and describes the implemented human foot moving direction identification algorithm. Practical experiment results demonstrate that the proposed system is capable of recognizing five foot motions, jump, step left, step right, step forward, and step backward, and has achieved an over 97% accuracy performance for different users. The functionality of the system for real-time application has also been verified through the practical experiments.

A Novel Method to Enhance Pipeline Trajectory Determination Using Pipeline Junctions.

Pipeline inspection gauges (pigs) have been used for many years to perform various maintenance operations in oil and gas pipelines. Different pipeline parameters can be inspected during the pig journey. Although pigs use many sensors to detect the required pipeline parameters, matching these data with the corresponding pipeline location is considered a very important parameter. High-end, tactical-grade inertial measurement units (IMUs) are used in pigging applications to locate the detected problems of pipeline using other sensors, and to reconstruct the trajectories of the pig. These IMUs are accurate; however, their high cost and large sizes limit their use in small diameter pipelines (8″ or less). This paper describes a new methodology for the use of MEMS-based IMUs using an extended Kalman filter (EKF) and the pipeline junctions to increase the position parameters' accuracy and to reduce the total RMS errors even during the unavailability of above ground markers (AGMs). The results of this new proposed method using a micro-electro-mechanical systems (MEMS)-based IMU revealed that the position RMS errors were reduced by approximately 85% compared to the standard EKF solution. Therefore, this approach will enable the mapping of small diameter pipelines, which was not possible before.

Smartphone-Based Indoor Localization with Bluetooth Low Energy Beacons.

Indoor wireless localization using Bluetooth Low Energy (BLE) beacons has attracted considerable attention after the release of the BLE protocol. In this paper, we propose an algorithm that uses the combination of channel-separate polynomial regression model (PRM), channel-separate fingerprinting (FP), outlier detection and extended Kalman filtering (EKF) for smartphone-based indoor localization with BLE beacons. The proposed algorithm uses FP and PRM to estimate the target's location and the distances between the target and BLE beacons respectively. We compare the performance of distance estimation that uses separate PRM for three advertisement channels (i.e., the separate strategy) with that use an aggregate PRM generated through the combination of information from all channels (i.e., the aggregate strategy). The performance of FP-based location estimation results of the separate strategy and the aggregate strategy are also compared. It was found that the separate strategy can provide higher accuracy; thus, it is preferred to adopt PRM and FP for each BLE advertisement channel separately. Furthermore, to enhance the robustness of the algorithm, a two-level outlier detection mechanism is designed. Distance and location estimates obtained from PRM and FP are passed to the first outlier detection to generate improved distance estimates for the EKF. After the EKF process, the second outlier detection algorithm based on statistical testing is further performed to remove the outliers. The proposed algorithm was evaluated by various field experiments. Results show that the proposed algorithm achieved the accuracy of <2.56 m at 90% of the time with dense deployment of BLE beacons (1 beacon per 9 m), which performs 35.82% better than <3.99 m from the Propagation Model (PM) + EKF algorithm and 15.77% more accurate than <3.04 m from the FP + EKF algorithm. With sparse deployment (1 beacon per 18 m), the proposed algorithm achieves the accuracies of <3.88 m at 90% of the time, which performs 49.58% more accurate than <8.00 m from the PM + EKF algorithm and 21.41% better than <4.94 m from the FP + EKF algorithm. Therefore, the proposed algorithm is especially useful to improve the localization accuracy in environments with sparse beacon deployment.

Context-aware personal navigation using embedded sensor fusion in smartphones.

Context-awareness is an interesting topic in mobile navigation scenarios where the context of the application is highly dynamic. Using context-aware computing, navigation services consider the situation of user, not only in the design process, but in real time while the device is in use. The basic idea is that mobile navigation services can provide different services based on different contexts-where contexts are related to the user's activity and the device placement. Context-aware systems are concerned with the following challenges which are addressed in this paper: context acquisition, context understanding, and context-aware application adaptation. The proposed approach in this paper is using low-cost sensors in a multi-level fusion scheme to improve the accuracy and robustness of context-aware navigation system. The experimental results demonstrate the capabilities of the context-aware Personal Navigation Systems (PNS) for outdoor personal navigation using a smartphone.