Mastering the game of Go with deep neural networks and tree search.

The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses 'value networks' to evaluate board positions and 'policy networks' to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

Soft song during aggressive interactions: seasonal changes and endocrine correlates in song sparrows.

It is well known that songbirds produce high amplitude songs ("broadcast songs"). Songbirds also produce low amplitude songs ("soft songs") during courtship or territorial aggression in the breeding season. Soft songs are important social signals but have been studied far less than broadcast songs. To date, no studies have examined seasonal changes in soft song or its endocrine regulation. Here, in male song sparrows, we examined soft songs during a simulated territorial intrusion in the breeding season and non-breeding season. We also measured plasma testosterone and dehydroepiandrosterone (DHEA) levels in subjects immediately after the aggressive encounter. The total number of songs produced (broadcast+soft songs) did not vary between seasons. However, there was a dramatic increase in the percentage of soft song in the non-breeding season. Further, the percentage of soft song was negatively correlated with plasma testosterone levels in the non-breeding season. There were seasonal differences in the acoustic structure of two major elements of soft song, trills and buzzes. The minimum frequency of trills was lower in the non-breeding season, and the element repetition rate of buzzes was lower in the non-breeding season. To our knowledge, this is the first study to (1) examine soft songs outside of the breeding season and (2) to identify endocrine correlates of soft songs, which are important social signals in songbirds.

Rapid and widespread effects of 17ß-estradiol on intracellular signaling in the male songbird brain: a seasonal comparison.

Across vertebrate species, 17ß-estradiol (E(2)) acts on the brain via both genomic and nongenomic mechanisms to influence neuronal physiology and behavior. Nongenomic E(2) signaling is typically initiated by membrane-associated estrogen receptors that modulate intracellular signaling cascades, including rapid phosphorylation of ERK. Phosphorylated ERK (pERK) can, in turn, rapidly phosphorylate tyrosine hydroxylase (TH) and cAMP response element-binding protein (CREB). Recent data suggest that the rapid effects of E(2) on mouse aggressive behavior are more prominent during short photoperiods (winter) and that acute aromatase inhibition reduces songbird aggression in winter only. To date, seasonal plasticity in the rapid effects of E(2) on intracellular signaling has not been investigated. Here, we compared the effects of acute (15 min) E(2) treatment on pERK, pTH, and pCREB immunoreactivity in male song sparrows (Melospiza melodia) pretreated with the aromatase inhibitor fadrozole during the breeding and nonbreeding seasons. We examined immunoreactivity in 14 brain regions including portions of the song control system, social behavior network, and the hippocampus (Hp). In both seasons, E(2) significantly decreased pERK in nucleus taeniae of the amygdala, pTH in ventromedial hypothalamus, and pCREB in mesencephalic central gray, robust nucleus of the arcopallium, and caudomedial nidopallium. However, several effects were critically dependent upon season. E(2) decreased pERK in caudomedial nidopallium in the breeding season only and decreased pCREB in the medial preoptic nucleus in the nonbreeding season only. Remarkably, E(2) decreased pERK in Hp in the breeding season but increased pERK in Hp in the nonbreeding season. Together, these data demonstrate that E(2) has rapid effects on intracellular signaling in multiple regions of the male brain and also demonstrate that rapid effects of E(2) can be profoundly different across the seasons.