Bayesian reinforcement learning for navigation planning in unknown environments.

This study focuses on a rescue mission problem, particularly enabling agents/robots to navigate efficiently in unknown environments. Technological advances, including manufacturing, sensing, and communication systems, have raised interest in using robots or drones for rescue operations. Effective rescue operations require quick identification of changes in the environment and/or locating the victims/injuries as soon as possible. Several techniques have been developed in recent years for autonomy in rescue missions, including motion planning, adaptive control, and more recently, reinforcement learning techniques. These techniques rely on full knowledge of the environment or the availability of simulators that can represent real environments during rescue operations. However, in practice, agents might have little or no information about the environment or the number or locations of injuries, preventing/limiting the application of most existing techniques. This study provides a probabilistic/Bayesian representation of the unknown environment, which jointly models the stochasticity in the agent's navigation and the environment uncertainty into a vector called the belief state. This belief state allows offline learning of the optimal Bayesian policy in an unknown environment without the need for any real data/interactions, which guarantees taking actions that are optimal given all available information. To address the large size of belief space, deep reinforcement learning is developed for computing an approximate Bayesian planning policy. The numerical experiments using different maze problems demonstrate the high performance of the proposed policy.

End-Cloud Collaboration Navigation Planning Method for Unmanned Aerial Vehicles Used in Small Areas.

Unmanned aerial vehicle (UAV) collaboration has become the main means of indoor and outdoor regional search, railway patrol, and other tasks, and navigation planning is one of the key, albeit difficult, technologies. The purpose of UAV navigation planning is to plan reasonable trajectories for UAVs to avoid obstacles and reach the task area. Essentially, it is a complex optimization problem that requires the use of navigation planning algorithms to search for path-point solutions that meet the requirements under the guide of objective functions and constraints. At present, there are autonomous navigation modes of UAVs relying on airborne sensors and navigation control modes of UAVs relying on ground control stations (GCSs). However, due to the limitation of airborne processor computing power, and background command and control communication delay, a navigation planning method that takes into account accuracy and timeliness is needed. First, the navigation planning architecture of UAVs of end-cloud collaboration was designed. Then, the background cloud navigation planning algorithm of UAVs was designed based on the improved particle swarm optimization (PSO). Next, the navigation control algorithm of the UAV terminals was designed based on the multi-objective hybrid swarm intelligent optimization algorithm. Finally, the computer simulation and actual indoor-environment flight test based on small rotor UAVs were designed and conducted. The results showed that the proposed method is correct and feasible, and can improve the effectiveness and efficiency of navigation planning of UAVs.

Evolutionary Algorithm-Based Complete Coverage Path Planning for Tetriamond Tiling Robots.

Tiling robots with fixed morphology face major challenges in terms of covering the cleaning area and generating the optimal trajectory during navigation. Developing a self-reconfigurable autonomous robot is a probable solution to these issues, as it adapts various forms and accesses narrow spaces during navigation. The total navigation energy includes the energy expenditure during locomotion and the shape-shifting of the platform. Thus, during motion planning, the optimal navigation sequence of a self-reconfigurable robot must include the components of the navigation energy and the area coverage. This paper addresses the framework to generate an optimal navigation path for reconfigurable cleaning robots made of tetriamonds. During formulation, the cleaning environment is filled with various tiling patterns of the tetriamond-based robot, and each tiling pattern is addressed by a waypoint. The objective is to minimize the amount of shape-shifting needed to fill the workspace. The energy cost function is formulated based on the travel distance between waypoints, which considers the platform locomotion inside the workspace. The objective function is optimized based on evolutionary algorithms such as the genetic algorithm (GA) and ant colony optimization (ACO) of the traveling salesman problem (TSP) and estimates the shortest path that connects all waypoints. The proposed path planning technique can be extended to other polyamond-based reconfigurable robots.

Single-Implant Treatment.

The replacement of one tooth using one implant involves a set of unique criteria for long-term success. Successful therapy should be based on long-term function and health of the adjacent tissues. Sections of this article examine these critical criteria that when working together can result in successful long-term tooth replacement.

Does ventrolateral prefrontal cortex help in searching for the lost key? Evidence from an fNIRS study.

The Key Search Task (KST) is a neuropsychological test that requires strategies for searching a lost key in an imaginary field. This request may involve different cognitive processes as mental imagery and navigation planning. This study was aimed at investigating, by a twenty-channel functional near-infrared spectroscopy (fNIRS) system, the hemodynamic response (i.e., oxygenated-hemoglobin (O2Hb) and deoxygenated-hemoglobin (HHb) changes) of the prefrontal cortex in navigation planning. A right ventrolateral prefrontal cortex (rVLPFC) activation during the KST was hypothesized. Thirty-eight volunteers performed the KST and a Control Task (CT), the latter requiring the volunteers to mark the X letter. An activation (i.e., increase/decrease in O2Hb/HHb) of: 1) rVLPFC during the KST execution, and 2) bilateral dorsolateral prefrontal cortex (DLPFC) during the CT execution was found. The present study provides a contribution in localizing the rVLPFC as the critically active region, within the frontal lobes, that was found maximally activated during mental navigation in the mind's eye of healthy participants while performing the KST. Considering the contribution of rVLPFC in spatial navigation, its activation suggests that the KST could be adopted in the clinical routine for investigating navigation planning. Compared to other neuroimaging techniques, fNIRS (with its relatively low physical constraints) contributes to better clarifying the role of rVLPFC in some aspects of human navigation. Therefore, the combined use of the fNIRS and the KST could be considered as an innovative and valid tool to evaluate fundamental functions for everyday life, such as spatial navigation planning.

The Key of the Maze: The role of mental imagery and cognitive flexibility in navigational planning.

Spatial navigation planning ability relies on both mental imagery and cognitive flexibility. Considering the importance of planning ability in everyday life, several neuropsychological tests are used in clinical practice for its assessment, although some of these are not aimed at assessing the strategies of navigational planning. The Porteus Maze Test (PMT) and the Key Search Task (KST) require to plan a strategy in a maze and in an imagined space, respectively. To the best of our knowledge, although these two tests share some features, the relationship between them has never been explored. The purpose of the present study was to investigate, for the first time, the relationship between the PMT and the KST performances in 38 healthy subjects in order to understand the implications of this association for the assessment of spatial navigation ability. Subjects were subdivided in bad or good navigation planners on the basis of the their KST score. The results of the study have revealed a significant difference (t=2.35; p=0.03) in the number of errors made at the PMT by bad navigational planners (0.78±0.28) and good navigational planners (0.10±0.06). The first group (bad navigational planners) made more errors at the PMT than the good navigational planners (who made less errors at the PMT). This provides evidence of the possibility to use the KST and the PMT in a combined way as a new tool for the assessment of spatial navigational planning ability. Furthermore, this finding highlights the importance of mental imagery and cognitive flexibility in spatial navigation, suggesting that these functions could be the link between a good planning ability and a successful spatial navigation. In conclusion, this study suggests that an efficient navigation would not be possible without a good navigational planning ability.