Emergence of belief-like representations through reinforcement learning.

To behave adaptively, animals must learn to predict future reward, or value. To do this, animals are thought to learn reward predictions using reinforcement learning. However, in contrast to classical models, animals must learn to estimate value using only incomplete state information. Previous work suggests that animals estimate value in partially observable tasks by first forming "beliefs"-optimal Bayesian estimates of the hidden states in the task. Although this is one way to solve the problem of partial observability, it is not the only way, nor is it the most computationally scalable solution in complex, real-world environments. Here we show that a recurrent neural network (RNN) can learn to estimate value directly from observations, generating reward prediction errors that resemble those observed experimentally, without any explicit objective of estimating beliefs. We integrate statistical, functional, and dynamical systems perspectives on beliefs to show that the RNN's learned representation encodes belief information, but only when the RNN's capacity is sufficiently large. These results illustrate how animals can estimate value in tasks without explicitly estimating beliefs, yielding a representation useful for systems with limited capacity.

Data-driven segmentation of audiometric phenotypes across a large clinical cohort.

Pure tone audiograms are used to assess the degree and underlying source of hearing loss. Audiograms are typically categorized into a few canonical types, each thought to reflect distinct pathologies of the ear. Here, we analyzed 116,400 patient records from our clinic collected over a 24-year period and found that standard categorization left 46% of patient records unclassified. To better account for the full spectrum of hearing loss profiles, we used a Gaussian Mixture Model (GMM) to segment audiograms without any assumptions about frequency relationships, interaural symmetry or etiology. The GMM converged on ten types, featuring varying degrees of high-frequency hearing loss, flat loss, mixed loss, and notched profiles, with predictable relationships to patient age and sex. A separate GMM clustering of 15,380 audiograms from the National Health and Nutrition Examination Survey (NHANES) identified six similar types, that only lacked the more extreme hearing loss configurations observed in our patient cohort. Whereas traditional approaches distill hearing loss configurations down to a few canonical types by disregarding much of the underlying variability, an objective probabilistic model that accounted for all of the data identified an organized, but more heterogenous set of audiogram types that was consistent across two large clinical databases.