Reinforcement Learning for Mobile Robotics Exploration: A Survey.

Efficient exploration of unknown environments is a fundamental precondition for modern autonomous mobile robot applications. Aiming to design robust and effective robotic exploration strategies, suitable to complex real-world scenarios, the academic community has increasingly investigated the integration of robotics with reinforcement learning (RL) techniques. This survey provides a comprehensive review of recent research works that use RL to design unknown environment exploration strategies for single and multirobots. The primary purpose of this study is to facilitate future research by compiling and analyzing the current state of works that link these two knowledge domains. This survey summarizes: what are the employed RL algorithms and how they compose the so far proposed mobile robot exploration strategies; how robotic exploration solutions are addressing typical RL problems like the exploration-exploitation dilemma, the curse of dimensionality, reward shaping, and slow learning convergence; and what are the performed experiments and software tools used for learning and testing. Achieved progress is described, and a discussion about remaining limitations and future perspectives is presented.

Proposal for an Embedded System Architecture Using a GNDVI Algorithm to Support UAV-Based Agrochemical Spraying.

An important area in precision agriculture is related to the efficient use of chemicals applied onto fields. Efforts have been made to diminish their use, aiming at cost reduction and fewer chemical residues in the final agricultural products. The use of unmanned aerial vehicles (UAVs) presents itself as an attractive and cheap alternative for spraying pesticides and fertilizers compared to conventional mass spraying performed by ordinary manned aircraft. Besides being cheaper than manned aircraft, small UAVs are capable of performing fine-grained instead of the mass spraying. Observing this improved method, this paper reports the design of an embedded real-time UAV spraying control system supported by onboard image processing. The proposal uses a normalized difference vegetation index (NDVI) algorithm to detect the exact locations in which the chemicals are needed. Using this information, the automated spraying control system performs punctual applications while the UAV navigates over the crops. The system architecture is designed to run on low-cost hardware, which demands an efficient NDVI algorithm. The experiments were conducted using Raspberry Pi 3 as the embedded hardware. First, experiments in a laboratory were conducted in which the algorithm was proved to be correct and efficient. Then, field tests in real conditions were conducted for validation purposes. These validation tests were performed in an agronomic research station with the Raspberry hardware integrated into a UAV flying over a field of crops. The average CPU usage was about 20% while memory consumption was about 70 MB for high definition images, with 4% CPU usage and 20.3 MB RAM being observed for low-resolution images. The average current measured to execute the proposed algorithm was 0.11 A. The obtained results prove that the proposed solution is efficient in terms of processing and energy consumption when used in embedded hardware and provides measurements which are coherent with the commercial GreenSeeker equipment.