Reinforcement-Learning-Based Routing and Resource Management for Internet of Things Environments: Theoretical Perspective and Challenges.

Internet of Things (IoT) devices are increasingly popular due to their wide array of application domains. In IoT networks, sensor nodes are often connected in the form of a mesh topology and deployed in large numbers. Managing these resource-constrained small devices is complex and can lead to high system costs. A number of standardized protocols have been developed to handle the operation of these devices. For example, in the network layer, these small devices cannot run traditional routing mechanisms that require large computing powers and overheads. Instead, routing protocols specifically designed for IoT devices, such as the routing protocol for low-power and lossy networks, provide a more suitable and simple routing mechanism. However, they incur high overheads as the network expands. Meanwhile, reinforcement learning (RL) has proven to be one of the most effective solutions for decision making. RL holds significant potential for its application in IoT device's communication-related decision making, with the goal of improving performance. In this paper, we explore RL's potential in IoT devices and discuss a theoretical framework in the context of network layers to stimulate further research. The open issues and challenges are analyzed and discussed in the context of RL and IoT networks for further study.

Preparing Wi-Fi 7 for Healthcare Internet-of-Things.

The healthcare Internet of Things (H-IoT) is an interconnection of devices capable of sensing and transmitting information that conveys the status of an individual's health. The continuous monitoring of an individual's health for disease diagnosis and early detection is an important application of H-IoT. Ambient assisted living (AAL) entails monitoring a patient's health to ensure their well-being. However, ensuring a limit on transmission delays is an essential requirement of such monitoring systems. The uplink (UL) transmission during the orthogonal frequency division multiple access (OFDMA) in the wireless local area networks (WLANs) can incur a delay which may not be acceptable for delay-sensitive applications such as H-IoT due to their random nature. Therefore, we propose a UL OFDMA scheduler for the next Wireless Fidelity (Wi-Fi) standard, the IEEE 802.11be, that is compliant with the latency requirements for healthcare applications. The scheduler allocates the channel resources for UL transmission taking into consideration the traffic class or access category. The results demonstrate that the proposed scheduler can achieve the required latency for H-IoT applications. Additionally, the performance in terms of fairness and throughput is also superior to state-of-the-art schedulers.

Reinforcement Learning-Enabled Cross-Layer Optimization for Low-Power and Lossy Networks under Heterogeneous Traffic Patterns.

The next generation of the Internet of Things (IoT) networks is expected to handle a massive scale of sensor deployment with radically heterogeneous traffic applications, which leads to a congested network, calling for new mechanisms to improve network efficiency. Existing protocols are based on simple heuristics mechanisms, whereas the probability of collision is still one of the significant challenges of future IoT networks. The medium access control layer of IEEE 802.15.4 uses a distributed coordination function to determine the efficiency of accessing wireless channels in IoT networks. Similarly, the network layer uses a ranking mechanism to route the packets. The objective of this study was to intelligently utilize the cooperation of multiple communication layers in an IoT network. Recently, Q-learning (QL), a machine learning algorithm, has emerged to solve learning problems in energy and computational-constrained sensor devices. Therefore, we present a QL-based intelligent collision probability inference algorithm to optimize the performance of sensor nodes by utilizing channel collision probability and network layer ranking states with the help of an accumulated reward function. The simulation results showed that the proposed scheme achieved a higher packet reception ratio, produces significantly lower control overheads, and consumed less energy compared to current state-of-the-art mechanisms.