A review of reinforcement learning based hyper-heuristics.

The reinforcement learning based hyper-heuristics (RL-HH) is a popular trend in the field of optimization. RL-HH combines the global search ability of hyper-heuristics (HH) with the learning ability of reinforcement learning (RL). This synergy allows the agent to dynamically adjust its own strategy, leading to a gradual optimization of the solution. Existing researches have shown the effectiveness of RL-HH in solving complex real-world problems. However, a comprehensive introduction and summary of the RL-HH field is still blank. This research reviews currently existing RL-HHs and presents a general framework for RL-HHs. This article categorizes the type of algorithms into two categories: value-based reinforcement learning hyper-heuristics and policy-based reinforcement learning hyper-heuristics. Typical algorithms in each category are summarized and described in detail. Finally, the shortcomings in existing researches on RL-HH and future research directions are discussed.

Construction of Mn doped Cu7S4 nanozymes for synergistic tumor therapy in NIR-I/II bio-windows.

In photothermal therapy (PTT) and chemodynamic therapy (CDT) of cancer, poor performance of nanoagents severely impaired the therapeutic effect of cancer. To solve the problem, we proposed and constructed a novel Mn doped Cu7S4 phothermal nanoagent both in the first near-infrared (NIR-I) and the second near- infrared (NIR-II) windows in this work, which exhibited high photothermal conversion efficiency of 40.3% at 808 nm (NIR-I window) and 33.4% at 1064 nm (NIR-II window), as well as outstanding pH-sensitive catalytic performance (peroxidase-like catalytic activity and Fenton-like catalytic activities). The as-prepared Mn doped Cu7S4 could be used to load chemotherapy drug doxorubicin (DOX) after modified by folic acid. Both in vitro and in vivo studies indicated that it could be used as nanoagent for chemodynamic therapy (CDT)/photothermal therapy (PTT)/ chemotherapy of cervical carcinoma. This study thus provided an NIR-I/NIR-II/pH responsive nanoagent for potential synergistic therapy of deep-seated tumors.

A Multiphase Semistatic Training Method for Swarm Confrontation Using Multiagent Deep Reinforcement Learning.

In this paper, we propose a multiphase semistatic training method for swarm confrontation using multi-agent deep reinforcement learning. In particular, we build a swarm confrontation game, the 3V3 tank fight, based on the Unity platform and train the agents by a MDRL algorithm called MA-POCA, coming with the ML-Agent toolkit. By multiphase learning, we split the traditional single training phase into multiple consecutive training phases, where the performance level of the strong team for each phase increases in an incremental way. On the other hand, by semistatic learning, the strong team in all phases will stop learning when fighting against the weak team, which reduces the possibility that the weak team keeps being defeated and learns nothing at all. Comprehensive experiments prove that, in contrast to the traditional single-phase training method, the multiphase semistatic training method proposed in this paper can significantly increase the training efficiency, shedding lights on how the weak could learn from the strong with less time and computational cost.