Life-history of Palaeoloxodon antiquus reveals Middle Pleistocene glacial refugium in the Megalopolis basin, Greece.

The Balkans are considered a major glacial refugium where flora and fauna survived glacial periods and repopulated the rest of Europe during interglacials. While it is also thought to have harboured Pleistocene human populations, evidence linking human activity, paleoenvironmental indicators and a secure temporal placement to glacial periods is scant. Here, we present the first intra-tooth multi-isotope analysis for the European straight-tusked elephant Palaeoloxodon antiquus, on an adult male individual excavated in association with lithic artefacts at the MIS 12 site Marathousa 1 (Megalopolis basin, Greece). The studied find also exhibits anthropogenic modifications, providing direct evidence of hominin presence. We employed strontium, carbon and oxygen isotope analysis on enamel bioapatite to investigate its foraging and mobility behaviour, using a sequential sampling strategy along the tooth growth axis of the third upper molar, to assess ecological changes during the last decade of life. We found a geographically restricted range, in a C3-dominated open woodland environment, and relatively stable conditions over the examined timeframe. Our results show that, despite the severity of the MIS 12 glacial, the Megalopolis basin sustained a mesic habitat, sufficient plant cover and limited seasonal fluctuations in resource availability, pointing to its role as a glacial refugium for both fauna and hominins.

A domain-agnostic approach for characterization of lifelong learning systems.

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of (1) Continuous Learning, (2) Transfer and Adaptation, and (3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.

First record of Macaca (Cercopithecidae, Primates) in the Middle Pleistocene of Greece.

In this article, we describe an almost complete macaque mandible from the Middle Pleistocene locality Marathousa 1 in the Megalopolis Basin of southern Greece. The mandible belonged to a male individual of advanced ontogenetic age and of estimated body mass ∼13 kg. Comparative metric analysis of its teeth permits its attribution to the Barbary macaque Macaca sylvanus, a species that was geographically widely distributed in Western Eurasia during the Plio-Pleistocene. The dental dimensions of the Marathousa 1 macaque fit better within the variation of the Early Pleistocene M. s. florentina and the Middle to Late Pleistocene M. s. pliocena rather than with the extant representative M. s. sylvanus. Moreover, principal component analysis reveals a better match with M. s. pliocena. However, because no clear-cut diagnostic criteria have been defined to differentiate these European fossil subspecies, we attribute the Marathousa 1 specimen to M. s. cf. pliocena, in agreement with the chronology of the locality. Previously known only from the Early Pleistocene of Greece by some isolated teeth, this is the first record of Macaca in the Middle Pleistocene of the country and one of very few in the eastern sector of the peri-Mediterranean region. We discuss the presence of macaques in the paleolake environment of Marathousa 1, as well as their predation risks from both carnivores and hominins present at the locality.

Optical Coherence Tomography-Guided Robotic Ophthalmic Microsurgery via Reinforcement Learning from Demonstration.

Ophthalmic microsurgery is technically difficult because the scale of required surgical tool manipulations challenge the limits of the surgeon's visual acuity, sensory perception, and physical dexterity. Intraoperative optical coherence tomography (OCT) imaging with micrometer-scale resolution is increasingly being used to monitor and provide enhanced real-time visualization of ophthalmic surgical maneuvers, but surgeons still face physical limitations when manipulating instruments inside the eye. Autonomously controlled robots are one avenue for overcoming these physical limitations. We demonstrate the feasibility of using learning from demonstration and reinforcement learning with an industrial robot to perform OCT-guided corneal needle insertions in an ex vivo model of deep anterior lamellar keratoplasty (DALK) surgery. Our reinforcement learning agent trained on ex vivo human corneas, then outperformed surgical fellows in reaching a target needle insertion depth in mock corneal surgery trials. This work shows the combination of learning from demonstration and reinforcement learning is a viable option for performing OCT guided robotic ophthalmic surgery.

Modeling and Planning with Macro-Actions in Decentralized POMDPs.

Decentralized partially observable Markov decision processes (Dec-POMDPs) are general models for decentralized multi-agent decision making under uncertainty. However, they typically model a problem at a low level of granularity, where each agent's actions are primitive operations lasting exactly one time step. We address the case where each agent has macro-actions: temporally extended actions that may require different amounts of time to execute. We model macro-actions as options in a Dec-POMDP, focusing on actions that depend only on information directly available to the agent during execution. Therefore, we model systems where coordination decisions only occur at the level of deciding which macro-actions to execute. The core technical difficulty in this setting is that the options chosen by each agent no longer terminate at the same time. We extend three leading Dec-POMDP algorithms for policy generation to the macro-action case, and demonstrate their effectiveness in both standard benchmarks and a multi-robot coordination problem. The results show that our new algorithms retain agent coordination while allowing high-quality solutions to be generated for significantly longer horizons and larger state-spaces than previous Dec-POMDP methods. Furthermore, in the multi-robot domain, we show that, in contrast to most existing methods that are specialized to a particular problem class, our approach can synthesize control policies that exploit opportunities for coordination while balancing uncertainty, sensor information, and information about other agents.

On The Necessity of Abstraction.

A generally intelligent agent faces a dilemma: it requires a complex sensorimotor space to be capable of solving a wide range of problems, but many tasks are only feasible given the right problem-specific formulation. I argue that a necessary but understudied requirement for general intelligence is the ability to form task-specific abstract representations. I show that the reinforcement learning paradigm structures this question into how to learn action abstractions and how to learn state abstractions, and discuss the field's progress on these topics.

Handedness and Reach-to-Place Kinematics in Adults: Left-Handers Are Not Reversed Right-Handers.

The primary goal of this study was to examine the relations between limb control and handedness in adults. Participants were categorized as left or right handed for analyses using the Edinburgh Handedness Inventory. Three-dimensional recordings were made of each arm on two reach-to-place tasks: adults reached to a ball and placed it into the opening of a toy (fitting task), or reached to a Cheerio inside a cup, which they placed on a designated mark after each trial (cup task). We hypothesized that limb control and handedness were related, and we predicted that we would observe side differences favoring the dominant limb based on the dynamic dominance hypothesis of motor lateralization. Specifically, we predicted that the dominant limb would be straighter and smoother on both tasks compared with the nondominant limb (i.e., right arm in right-handers and left arm in left-handers). Our results only partially supported these predictions for right-handers, but not for left-handers. When differences between hands were observed, the right hand was favored regardless of handedness group. Our findings suggest that left-handers are not reversed right-handers when compared on interlimb kinematics for reach-to-place tasks, and reaffirm that task selection is critical when evaluating manual asymmetries.

Representing, learning, and controlling complex object interactions.

We present a framework for representing scenarios with complex object interactions, where a robot cannot directly interact with the object it wishes to control and must instead influence it via intermediate objects. For instance, a robot learning to drive a car can only change the car's pose indirectly via the steering wheel, and must represent and reason about the relationship between its own grippers and the steering wheel, and the relationship between the steering wheel and the car. We formalize these interactions as chains and graphs of Markov decision processes (MDPs) and show how such models can be learned from data. We also consider how they can be controlled given known or learned dynamics. We show that our complex model can be collapsed into a single MDP and solved to find an optimal policy for the combined system. Since the resulting MDP may be very large, we also introduce a planning algorithm that efficiently produces a potentially suboptimal policy. We apply these models to two systems in which a robot uses learning from demonstration to achieve indirect control: playing a computer game using a joystick, and using a hot water dispenser to heat a cup of water.

Active Exploration for Learning Symbolic Representations.

We introduce an online active exploration algorithm for data-efficiently learning an abstract symbolic model of an environment. Our algorithm is divided into two parts: the first part quickly generates an intermediate Bayesian symbolic model from the data that the agent has collected so far, which the agent can then use along with the second part to guide its future exploration towards regions of the state space that the model is uncertain about. We show that our algorithm outperforms random and greedy exploration policies on two different computer game domains. The first domain is an Asteroids-inspired game with complex dynamics but basic logical structure. The second is the Treasure Game, with simpler dynamics but more complex logical structure.

Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes.

We introduce a new formulation of the Hidden Parameter Markov Decision Process (HiP-MDP), a framework for modeling families of related tasks using low-dimensional latent embeddings. Our new framework correctly models the joint uncertainty in the latent parameters and the state space. We also replace the original Gaussian Process-based model with a Bayesian Neural Network, enabling more scalable inference. Thus, we expand the scope of the HiP-MDP to applications with higher dimensions and more complex dynamics.

Constructing Abstraction Hierarchies Using a Skill-Symbol Loop.

We describe a framework for building abstraction hierarchies whereby an agent alternates skill- and representation-construction phases to construct a sequence of increasingly abstract Markov decision processes. Our formulation builds on recent results showing that the appropriate abstract representation of a problem is specified by the agent's skills. We describe how such a hierarchy can be used for fast planning, and illustrate the construction of an appropriate hierarchy for the Taxi domain.

Representing and Learning Complex Object Interactions.

We present a framework for representing scenarios with complex object interactions, in which a robot cannot directly interact with the object it wishes to control, but must instead do so via intermediate objects. For example, a robot learning to drive a car can only indirectly change its pose, by rotating the steering wheel. We formalize such complex interactions as chains of Markov decision processes and show how they can be learned and used for control. We describe two systems in which a robot uses learning from demonstration to achieve indirect control: playing a computer game, and using a hot water dispenser to heat a cup of water.

Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizations.

Control applications often feature tasks with similar, but not identical, dynamics. We introduce the Hidden Parameter Markov Decision Process (HiP-MDP), a framework that parametrizes a family of related dynamical systems with a low-dimensional set of latent factors, and introduce a semiparametric regression approach for learning its structure from data. We show that a learned HiP-MDP rapidly identifies the dynamics of new task instances in several settings, flexibly adapting to task variation.

Hand preference status and reach kinematics in infants.

Infants show age-related improvements in reach straightness and smoothness over the first years of life as well as a decrease in average movement speed. This period of changing kinematics overlaps the emergence of handedness. We examined whether infant hand preference status is related to the development of motor control in 53 infants ranging from 11 to 14 months old. Hand preference status was assessed from reaching to a set of 5 objects presented individually at the infant's midline; infants were classified into 'right preference' or 'no preference' groups. Three-dimensional (3-D) recordings were made of each arm for reaches under two distinct conditions: pick up a ball and fit it into the opening of a toy (grasp-to-place task) or pick up a Cheerio® and consume it (grasp-to-eat task). Contrary to expectations, there was no effect of hand preference status on reach smoothness or straightness for either task. On the grasp-to-eat task only, average speed of the left hand differed as a function of hand preference status. Infants in the no preference group exhibited higher left hand average speeds than infants in the right preference group. Our results suggest that while behavioral differences in the use of the two hands may be present in some infants, these differences do not appear to be systematically linked to biases in motor control of the arms early in development.

Kinematics of reaching and implications for handedness in rhesus monkey infants.

Kinematic studies of reaching in human infants using two-dimensional (2-D) and three-dimensional (3-D) recordings have complemented behavioral studies of infant handedness by providing additional evidence of early right asymmetries. Right hand reaches have been reported to be straighter and smoother than left hand reaches during the first year. Although reaching has been a popular measure of handedness in primates, there has been no systematic comparison of left and right hand reach kinematics. We investigated reaching in infant rhesus monkeys using the 2-D motion analysis software MaxTRAQ Lite+ (Innovision Systems). Linear mixed-effects models revealed that left hand reaches were smoother, but not straighter, than right hand reaches. An early left bias matches previous findings of a left hand preference for reaching in adult rhesus monkeys. Additional work using this kind of kinematic approach will extend our understanding of primate handedness beyond traditional studies measuring only frequency or bouts of hand use.