A Novel Medium Access Policy Based on Reinforcement Learning in Energy-Harvesting Underwater Sensor Networks.

Underwater acoustic sensor networks (UASNs) are fundamental assets to enable discovery and utilization of sub-sea environments and have attracted both academia and industry to execute long-term underwater missions. Given the heightened significance of battery dependency in underwater wireless sensor networks, our objective is to maximize the amount of harvested energy underwater by adopting the TDMA time slot scheduling approach to prolong the operational lifetime of the sensors. In this study, we considered the spatial uncertainty of underwater ambient resources to improve the utilization of available energy and examine a stochastic model for piezoelectric energy harvesting. Considering a realistic channel and environment condition, a novel multi-agent reinforcement learning algorithm is proposed. Nodes observe and learn from their choice of transmission slots based on the available energy in the underwater medium and autonomously adapt their communication slots to their energy harvesting conditions instead of relying on the cluster head. In the numerical results, we present the impact of piezoelectric energy harvesting and harvesting awareness on three lifetime metrics. We observe that energy harvesting contributes to 4% improvement in first node dead (FND), 14% improvement in half node dead (HND), and 22% improvement in last node dead (LND). Additionally, the harvesting-aware TDMA-RL method further increases HND by 17% and LND by 38%. Our results show that the proposed method improves in-cluster communication time interval utilization and outperforms traditional time slot allocation methods in terms of throughput and energy harvesting efficiency.

Energy Harvesting and Task-Aware Multi-Robot Task Allocation in Robotic Wireless Sensor Networks.

In this work, we investigate an energy-aware multi-robot task-allocation (MRTA) problem in a cluster of the robot network that consists of a base station and several clusters of energy-harvesting (EH) robots. It is assumed that there are M+1 robots in the cluster and M tasks exist in each round. In the cluster, a robot is elected as the cluster head, which assigns one task to each robot in that round. Its responsibility (or task) is to collect the resultant data from the remaining M robots to aggregate and transmit directly to the BS. This paper aims to allocate the M tasks to the remaining M robots optimally or near optimally by considering the distance to be traveled by each node, the energy required for executing each task, the battery level at each node, and the energy-harvesting capabilities of the nodes. Then, this work presents three algorithms: Classical MRTA Approach, Task-aware MRTA Approach, EH and Task-aware MRTA Approach. The performances of the proposed MRTA algorithms are evaluated under both independent and identically distributed (i.i.d.) and Markovian energy-harvesting processes for different scenarios with five robots and 10 robots (with the same number of tasks). EH and Task-aware MRTA Approach shows the best performance among all MRTA approaches by keeping up to 100% more energy in the battery than the Classical MRTA Approach and keeping up to 20% more energy in the battery than the Task-aware MRTA Approach.

Heuristic Resource Reservation Policies for Public Clouds in the IoT Era.

With the advances in the IoT era, the number of wireless sensor devices has been growing rapidly. This increasing number gives rise to more complex networks where more complex tasks can be executed by utilizing more computational resources from the public clouds. Cloud service providers use various pricing models for their offered services. Some models are appropriate for the cloud service user's short-term requirements whereas the other models are appropriate for the long-term requirements of cloud service users. Reservation-based price models are suitable for long-term requirements of cloud service users. We used the pricing schemes with spot and reserved instances. Reserved instances support a hybrid cost model with fixed reservation costs that vary with contract duration and an hourly usage charge which is lower than the charge of the spot instances. Optimizing resources to be reserved requires sufficient research effort. Recent algorithms proposed for this problem are generally based on integer programming problems, so they do not have polynomial time complexity. In this work, heuristic-based polynomial time policies are proposed for this problem. It is exhibited that the cost for the cloud service user which uses our approach is comparable to optimal solutions, i.e., it is near-optimal.

Energy-Efficient Cluster-Based Data Collection by a UAV with a Limited-Capacity Battery in Robotic Wireless Sensor Networks.

In this work, our motivation focuses on an energy-efficient data collection problem by a mobile sink, an unmanned aerial vehicle (UAV) with limited battery capacity, in a robot network divided into several robot clusters. In each cluster, a cluster head (CH) robot allocates tasks to the remaining robots and collects data from them. Our contribution is to minimize the UAV total energy consumption coupled to minimum cost data collection from CH robots by visiting optimally a portion of the CH robots. The UAV decides the subset of CH robots to visit by considering not only the locations of all CH robots but also its battery capacity. If the UAV cannot visit all CH robots, then the CH robots not visited by the UAV transmit their data to another CH robot to forward it. The decision of transmission paths of transmitting robots is included in the cost optimization. Our contribution passes beyond the existing paradigms in the literature by considering the constant battery capacity for the UAV. We derive the optimal approach analytically for this problem. For various numbers of clusters, the performance of our strategy is compared with the approach in the close literature in terms of total energy consumed by CH robots, which affects network lifetime. Numerical results demonstrate that our strategy outperforms the approach in the close literature.

Average Throughput Performance of Myopic Policy in Energy Harvesting Wireless Sensor Networks.

This paper considers a single-hop wireless sensor network where a fusion center collects data from M energy harvesting wireless sensors. The harvested energy is stored losslessly in an infinite-capacity battery at each sensor. In each time slot, the fusion center schedules K sensors for data transmission over K orthogonal channels. The fusion center does not have direct knowledge on the battery states of sensors, or the statistics of their energy harvesting processes. The fusion center only has information of the outcomes of previous transmission attempts. It is assumed that the sensors are data backlogged, there is no battery leakage and the communication is error-free. An energy harvesting sensor can transmit data to the fusion center whenever being scheduled only if it has enough energy for data transmission. We investigate average throughput of Round-Robin type myopic policy both analytically and numerically under an average reward (throughput) criterion. We show that Round-Robin type myopic policy achieves optimality for some class of energy harvesting processes although it is suboptimal for a broad class of energy harvesting processes.