Outracing champion Gran Turismo drivers with deep reinforcement learning.

Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block opponents while operating their vehicles at their traction limits1. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the non-linear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world's best e-sports drivers. We combine state-of-the-art, model-free, deep reinforcement learning algorithms with mixed-scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing's important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world's best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.

Identification of the causative gene of a transparent phenotype of juvenile red sea bream Pagrus major.

Deformities in cultured fish species may be genetic, and identifying causative genes is essential to expand production and maintain farmed animal welfare. We previously reported a genetic deformity in juvenile red sea bream, designated a transparent phenotype. To identify its causative gene, we conducted genome-wide linkage analysis and identified two single nucleotide polymorphisms (SNP) located on LG23 directly linked to the transparent phenotype. The scaffold on which the two SNPs were located contained two candidate genes, duox and duoxa, which are related to thyroid hormone synthesis. Four missense mutations were found in duox and one in duoxa, with that in duoxa showing perfect association with the transparent phenotype. The mutation of duoxa was suggested to affect the transmembrane structure and thyroid-related traits, including an enlarged thyroid gland and immature erythrocytes, and lower thyroxine (T4) concentrations were observed in the transparent phenotype. The transparent phenotype was rescued by T4 immersion. Loss-of-function of duoxa by CRISPR-Cas9 induced the transparent phenotype in zebrafish. Evidence suggests that the transparent phenotype of juvenile red sea bream is caused by the missense mutation of duoxa and that this mutation disrupts thyroid hormone synthesis. The newly identified missense mutation will contribute to effective selective breeding of red sea bream to purge the causative gene of the undesirable phenotype and improve seed production of red sea bream as well as provide basic information of the mechanisms of thyroid hormones and its related diseases in fish and humans.