Model-Based Self-Advising for Multi-Agent Learning.

In multiagent learning, one of the main ways to improve learning performance is to ask for advice from another agent. Contemporary advising methods share a common limitation that a teacher agent can only advise a student agent if the teacher has experience with an identical state. However, in highly complex learning scenarios, such as autonomous driving, it is rare for two agents to experience exactly the same state, which makes the advice less of a learning aid and more of a one-time instruction. In these scenarios, with contemporary methods, agents do not really help each other learn, and the main outcome of their back and forth requests for advice is an exorbitant communications' overhead. In human interactions, teachers are often asked for advice on what to do in situations that students are personally unfamiliar with. In these, we generally draw from similar experiences to formulate advice. This inspired us to provide agents with the same ability when asked for advice on an unfamiliar state. Hence, we propose a model-based self-advising method that allows agents to train a model based on states similar to the state in question to inform its response. As a result, the advice given can not only be used to resolve the current dilemma but also many other similar situations that the student may come across in the future via self-advising. Compared with contemporary methods, our method brings a significant improvement in learning performance with much lower communication overheads.

Differential Advising in Multiagent Reinforcement Learning.

Agent advising is one of the main approaches to improve agent learning performance by enabling agents to share advice. Existing advising methods have a common limitation that an adviser agent can offer advice to an advisee agent only if the advice is created in the same state as the advisee's state. However, in complex environments, it is a very strong requirement that two states are the same, because a state may consist of multiple dimensions and two states being the same means that all these dimensions in the two states are correspondingly identical. Therefore, this requirement may limit the applicability of existing advising methods to complex environments. In this article, inspired by the differential privacy scheme, we propose a differential advising method that relaxes this requirement by enabling agents to use advice in a state even if the advice is created in a slightly different state. Compared with the existing methods, agents using the proposed method have more opportunity to take advice from others. This article is the first to adopt the concept of differential privacy on advising to improve agent learning performance instead of addressing security issues. The experimental results demonstrate that the proposed method is more efficient in complex environments than the existing methods.

Rehabilitation Training Combined with Jiaji Electroacupuncture Can Promote the Recovery of Muscle Group Function and Improve the Quality of Life in Patients with Upper Limb Peripheral Nerve Injury.

This study was designed to probe into the improvement of rehabilitation training combined with Jiaji electroacupuncture intervention on patients with upper limb peripheral nerve injury. A total of 114 patients with peripheral nerve injury of upper limbs in our hospital from August 2017 to November 2019 were collected as the research participants. Among them, 59 in the control group (CG) received rehabilitation training alone, while 65 in the observation group (OG) received rehabilitation training combined with Jiaji electroacupuncture intervention. The therapeutic efficacy, Barthel index, and Fugl-Meyer assessment score, motor nerve conduction velocity, sensory nerve conduction velocity and amplitude, and quality of life (score SF-36) were compared between the two groups before and after treatment. The total effective rate of the OG was markedly higher than that of the CG. After treatment, the Barthel index, Fugl-Meyer assessment score, motor nerve conduction velocity, and sensory nerve conduction velocity and amplitude of the OG were obviously higher than those of the CG, and the SF-36 scores of the OG were higher than those of the CG in 8 dimensions. Rehabilitation training combined with Jiaji electroacupuncture intervention can dramatically promote the recovery of muscle group function and improve the quality of life of patients with upper limb peripheral nerve injury.

Differentially Private Malicious Agent Avoidance in Multiagent Advising Learning.

Agent advising is one of the key approaches to improve agent learning performance by enabling agents to ask for advice between each other. Existing agent advising approaches have two limitations. The first limitation is that all the agents in a system are assumed to be friendly and cooperative. However, in the real world, malicious agents may exist and provide false advice to hinder the learning performance of other agents. The second limitation is that the analysis of communication overhead in these approaches is either overlooked or simplified. However, in communication-constrained environments, communication overhead has to be carefully considered. To overcome the two limitations, this paper proposes a novel differentially private agent advising approach. Our approach employs the Laplace mechanism to add noise on the rewards used by student agents to select teacher agents. By using the differential privacy technique, the proposed approach can reduce the impact of malicious agents without identifying them. Also, by adopting the privacy budget concept, the proposed approach can naturally control communication overhead. The experimental results demonstrate the effectiveness of the proposed approach.

A Self-Adaptive Sleep/Wake-Up Scheduling Approach for Wireless Sensor Networks.

Sleep/wake-up scheduling is one of the fundamental problems in wireless sensor networks, since the energy of sensor nodes is limited and they are usually unrechargeable. The purpose of sleep/wake-up scheduling is to save the energy of each node by keeping nodes in sleep mode as long as possible (without sacrificing packet delivery efficiency) and thereby maximizing their lifetime. In this paper, a self-adaptive sleep/wake-up scheduling approach is proposed. Unlike most existing studies that use the duty cycling technique, which incurs a tradeoff between packet delivery delay and energy saving, the proposed approach, which does not us duty cycling, avoids such a tradeoff. The proposed approach, based on the reinforcement learning technique, enables each node to autonomously decide its own operation mode (sleep, listen, or transmission) in each time slot in a decentralized manner. Simulation results demonstrate the good performance of the proposed approach in various circumstances.

Elevated level of nerve growth factor in the bladder pain syndrome/interstitial cystitis: a meta-analysis.

OBJECTIVES: To elucidate the association between nerve growth factor (NGF) level and bladder pain syndrome/interstitial cystitis (BPS/IC) by conducting a meta-analysis. METHODS: We conducted a systematic literature search to identify original studies of NGF level in BPS/IC before November 2015. Eligible studies were retrieved via both computer searches and manual review of references. The summary difference estimates between controlled group and BPS/IC group were calculated based on the weighted mean difference (WMD) with its 95 % confidence interval (CI). Sensitivity and publication analyses were performed after the pooled analysis. RESULTS: Meta-analysis of 10 original studies involving 295 cases and 290 normal controls showed an increased level of urinary NGF in BPS/IC patients (z = 3.08, P = 0.002). The combined WMD was 36.39 (95 % CI 13.27-59.51). There was significant difference between controlled group and BPS/IC patients in the term of NGF/Cr level (WMD = 0.96, 95 % CI 0.58-1.35; z = 4.89, P < 0.01). There was no significant publication bias in the included studies (P for Begg's test = 0.73, P for egger's test = 0.13). CONCLUSIONS: Our results demonstrated that there was an increased level of NGF in the BPS/IC patients.

A multi-agent framework for packet routing in wireless sensor networks.

Wireless sensor networks (WSNs) have been widely investigated in recent years. One of the fundamental issues in WSNs is packet routing, because in many application domains, packets have to be routed from source nodes to destination nodes as soon and as energy efficiently as possible. To address this issue, a large number of routing approaches have been proposed. Although every existing routing approach has advantages, they also have some disadvantages. In this paper, a multi-agent framework is proposed that can assist existing routing approaches to improve their routing performance. This framework enables each sensor node to build a cooperative neighbour set based on past routing experience. Such cooperative neighbours, in turn, can help the sensor to effectively relay packets in the future. This framework is independent of existing routing approaches and can be used to assist many existing routing approaches. Simulation results demonstrate the good performance of this framework in terms of four metrics: average delivery latency, successful delivery ratio, number of live nodes and total sensing coverage.