Oculocentric frames of reference for limb movement.

We have reviewed evidence that suggests that the target for limb motion is encoded in a retinocentric frame of reference. Errors in pointing that are elicited by an illusion that distorts the perceived motion of a target are strongly correlated with errors in gaze position. The modulations in the direction and speed of ocular smooth pursuit and of the hand show remarkable similarities, even though the inertia of the arm is much larger than that of the eye. We have suggested that ocular motion is constrained so that gaze provides an appropriate target signal for the hand. Finally, ocular and manual tracking deficits in patients with cerebellar ataxia are very similar. These deficits are also consistent with the idea that a gaze signal provides the target for hand motion; in some cases limb ataxia would be a consequence of optic ataxia rather than reflecting a deficit in the control of limb motion per se. These results, as well as neurophysiological data summarized here, have led us to revise a hypothesis we have previously put forth to account for the initial stages of sensorimotor transformations underlying targeted limb motions. In the original hypothesis, target location and initial arm posture were ultimately encoded in a common frame of reference tied to somatosensation, i.e. a body-centered frame of reference, and a desired change in posture was derived from the difference between the two. In our new scheme, a movement vector is derived from the difference between variables encoded in a retinocentric frame of reference. Accordingly, gaze, with its exquisite ability to stabilize a target image even under dynamic conditions, would be used as a reference signal. Consequently, this scheme would facilitate the processing of information under conditions in which the body and the target are moving relative to each other.

Capturing the frame of reference of shoulder muscle forces.

We have adjusted and validated models of the lines of action of six human shoulder muscles. Compared to other models of the shoulder mechanism (e.g., 9, 11, 17, 24) ours is greatly simplified in that the scapula and its many muscles are largely neglected, and the action of each of six broad muscles is summarized by a single line segment. The close correspondence between measured and predicted moment direction over a wide range of postures suggests that this latter simplification was reasonable. We found that assignment of two via points (one fixed to the origin and one fixed to the insertion) was adequate to represent the measured actions, and our sensitivity analysis highlights the importance of via point placement. Because our model predicts the action of each muscle over a very large range of arm postures, it should be useful for future investigations of the control of muscle forces.

Two virtual fingers in the control of the tripod grasp.

To investigate the organization of multi-fingered grasping, we asked subjects to grasp an object using three digits: the thumb, the index finger, and the middle or ring finger. The object had three coarse flat contact surfaces, whose locations and orientations were varied systematically. Subjects were asked to grasp and lift the object and then to hold it statically. We analyzed the grasp forces in the horizontal plane that were recorded during the static hold period. Static equilibrium requires that the forces exerted by the three digits intersect at a common point, the force focus. The directions of the forces exerted by the two fingers opposing the thumb depended on the orientation of the contact surfaces of both fingers but not on the orientation of the contact surface of the thumb. The direction of the thumb's force did not depend on the orientation of the contact surfaces of the two fingers and depended only weakly on the orientation of the thumb's contact surface. In general, the thumb's force was directed to a point midway between the two fingers. The results are consistent with a hierarchical model of the control of a tripod grasp. At the first level, an opposition space is created between the thumb and a virtual finger located approximately midway between the two actual fingers. The directions of the forces exerted by the two fingers are constrained to be mirror symmetric about the opposition axis. The actual directions of finger force are elaborated at the next level on the basis of stability considerations.

The Duncker illusion and eye-hand coordination.

A moving background alters the perceived direction of target motion (the Duncker illusion). To test whether this illusion also affects pointing movements to remembered/extrapolated target locations, we constructed a display in which a target moved in a straight line and disappeared behind a band of moving random dots. Subjects were required to touch the spot where the target would emerge from the occlusion. The four directions of random-dot motion induced pointing errors that were predictable from the Duncker illusion. Because it has been previously established that saccadic direction is influenced by this illusion, gaze was subsequently recorded in a second series of experiments while subjects performed the pointing task and a similar task with eye-tracking only. In the pointing task, subjects typically saccaded to the lower border of the occlusion zone as soon as the target disappeared and then tried to maintain fixation at that spot. However, it was particularly obvious in the eye-tracking-only condition that horizontally moving random dots generally evoked an appreciable ocular following response, altering the gaze direction. Hand-pointing errors were related to the saccadic gaze error but were more highly correlated with final gaze errors (resulting from the initial saccade and the subsequent ocular following response). The results suggest a model of limb control in which gaze position can provide the target signal for limb movement.

Force synergies for multifingered grasping.

Compared with the control of precision grips involving the thumb and one or two fingers, the control of grasping using the entire hand involves a larger number of degrees of freedom that has to be controlled simultaneously, and it introduces indeterminacies in the distribution of grip forces suitable for holding an object. We studied the control of five-digit grasping by measuring contact forces when subjects lifted, held, and replaced a manipulandum. This study focused primarily on the patterns of coordination among the normal forces exerted by each of the digits, assessed by varying the center of mass of the manipulandum. The force patterns during the lift and hold phases were modulated as a function of the location of the center of mass. A frequency domain analysis revealed a consistent temporal synergy by which digits tended to exert normal forces in phase with each other across all experimental conditions. This tendency for in-phase covariations by the normal forces exerted by the digits extended over the entire functional frequency range (up to 10 Hz). When the effect of thumb force was removed, a second synergy was revealed in which forces in two fingers could be modulated 180 degrees out of phase (also prevailing throughout the range of frequencies studied). The first synergy suggests the presence of a "common drive" to all of the extrinsic finger muscles, whereas the second one suggests another input, ultimately resulting in a reciprocally organized pattern of activity of some of these muscles.

Similarity in the response of smooth pursuit and manual tracking to a change in the direction of target motion.

Subjects were asked to track, with their eyes or their hand, the movement of a target that maintained a constant speed and made a single, abrupt change in direction. The tracking speed and direction of motion after the step change in target direction were compared for the eyes and the hand. After removal of the saccades from the eye movement records, it was found that in both cases, there was a slow rotation from the initial direction to the new direction. For the eyes and the hand, it was found that this change in direction of movement occurred at a similar rate that was proportional to the magnitude of the abrupt change in target direction. This was further described by comparing the direction of pursuit tracking with the response of a second-order system to a step input. In addition, it was found that the speed of manual and pursuit tracking was modulated in a similar manner, with a reduction in tracking speed occurring before the change in tracking direction. This reduction in speed following the change in the direction of target motion was very similar for the hand and the eye, despite the large difference in the inertias of the two systems. Taken together, these data suggest that the neural mechanisms for smooth pursuit and manual tracking have common functional elements and that musculoskeletal dynamics do not appear to be a rate-limiting factor.

Manual tracking in two dimensions.

Manual tracking was studied by asking subjects to follow, with their finger, a target moving on a touch-sensitive video monitor. The target initially moved in a straight line at a constant speed and then, at a random point in time, made one abrupt change in direction. The results were approximated with a simple model according to which, after a reaction time, the hand moved in a straight line to intercept the target. Both the direction of hand motion and its peak speed could be predicted by assuming a constant time to intercept. This simple model was able to account for results obtained over a broad range of target speeds as well as the results of experiments in which both the speed and the direction of the target changed simultaneously. The results of an experiment in which the target acceleration was nonzero suggested that the error signals used during tracking are related to both speed and direction but poorly (if at all) to target acceleration. Finally, in an experiment in which target velocity remained constant along one axis but the perpendicular component underwent a step change, tracking along both axes was perturbed. This last finding demonstrates that tracking in two dimensions cannot be decomposed into its Cartesian components. However, an analytical model in a hand-centered frame of reference in which speed and direction are the controlled variables could account for much of the data.

Oculomotor tracking in two dimensions.

Results from studies of oculomotor tracking in one dimension have indicated that saccades are driven primarily by errors in position, whereas smooth pursuit movements are driven primarily by errors in velocity. To test whether this result generalizes to two-dimensional tracking, we asked subjects to track a target that moved initially in a straight line then changed direction. We found that the general premise does indeed hold true; however, the study of oculomotor tracking in two dimensions provides additional insight. The first saccade was directed slightly in advance of target location at saccade onset. Thus its direction was related primarily to angular positional error. The direction of the smooth pursuit movement after the saccade was related linearly to the direction of target motion with an average slope of 0.8. Furthermore the magnitude and direction of smooth pursuit velocity did not change abruptly; consequently the direction of smooth pursuit appeared to rotate smoothly over time.

Postural hand synergies for tool use.

Subjects were asked to shape the right hand as if to grasp and use a large number of familiar objects. The chosen objects typically are held with a variety of grips, including "precision" and "power" grips. Static hand posture was measured by recording the angular position of 15 joint angles of the fingers and of the thumb. Although subjects adopted distinct hand shapes for the various objects, the joint angles of the digits did not vary independently. Principal components analysis showed that the first two components could account for >80% of the variance, implying a substantial reduction from the 15 degrees of freedom that were recorded. However, even though they were small, higher-order (more than three) principal components did not represent random variability but instead provided additional information about the object. These results suggest that the control of hand posture involves a few postural synergies, regulating the general shape of the hand, coupled with a finer control mechanism providing for small, subtle adjustments. Because the postural synergies did not coincide with grip taxonomies, the results suggest that hand posture may be regulated independently from the control of the contact forces that are used to grasp an object.

Gradual molding of the hand to object contours.

Subjects were asked to reach to and to grasp 15 similarly sized objects with the four fingers opposed to the thumb. The objects' contours differed: some presented a concave surface to the fingers, others a flat one, and yet others a convex surface. Flexion/extension at the metacarpal-phalangeal and proximal interphalangeal joints of the fingers was recorded during the reaching movement. We used discriminant analysis, cluster analysis, and information theory to determine the extent to which the shape of the hand was affected by the objects' shapes along a convexity/concavity gradient. Maximum aperture of the hand was reached about midway in the reaching movement. At that time, the hand's posture was influenced by the shape of the object to be grasped but imperfectly. The information transmitted by hand posture about object shape increased gradually and monotonically as the hand approached the object, reaching a maximum at the time the object was in the grasp of the hand. We also asked subjects to shape the hand so as to grasp the object without moving the arm. Their performance was poorer on this task in the sense that hand shape discriminated among fewer objects and that trial-to-trial variability was greater than when the distal and proximal components of the motion were linked. The results indicate that the hand is molded only gradually to the contours of an object to be grasped. Because other parameters of the motion, such as movement direction, for example, already are specified fully early on in a movement, the results also suggest that the specification of diverse aspects of a movement does not evolve at a uniform rate.

Postural dependence of muscle actions: implications for neural control.

The neural control of reaching entails the specification of a precise pattern of muscle activation distributed across the many muscles of the arm. Musculoskeletal geometry limits the possible solutions to this problem. Insight into the nature of this constraint was obtained by quantifying the postural variation in the mechanical actions of six human shoulder muscles. Estimates of muscle mechanical actions were obtained by electrically stimulating muscles to the point of contraction and recording the resulting forces and torques with a six-degree-of-freedom force-torque transducer. In a given experiment, data were obtained for up to 29 different arm postures. The mechanical actions of each muscle varied systematically with arm posture, regardless of the frame of reference used to define these actions. The nature of this dependence suggests that a relatively simple strategy can be used by the nervous system to account for the changing mechanical actions of arm muscles.

Anticipatory and sequential motor control in piano playing.

Pianists were asked to play short excerpts from several pieces on an electronic keyboard. In each piece, there were two phrases whose first few notes were played identically with the right hand. Thereafter, the two phrases were played differently. The aim of the investigation was to ascertain whether or not hand and finger kinematics diverged prior to the depression of the last common note. Such a divergence would imply an anticipatory modification of sequential movements of the hand, akin to the phenomenon of coarticulation in speech. The lack of such a divergence would imply a strictly serial organization of movement sequences with one hand, as was found previously to be the case for typing. The time at which each key was depressed and released and the speed with which the key was depressed was recorded via a MIDI interface to a laboratory computer. The motion of the right wrist and of the fingers of the right hand was recorded optoelectronically. Piano playing can invoke anticipatory modifications of hand and finger kinematics. The time at which two patterns of movements diverged varied considerably from piece to piece. Playing an ascending scale with the requirement of a "thumb-under" maneuver could evoke an anticipatory modification as much as 500 ms in advance of the last common note. In another piece, keypresses appeared to be executed in a strict serial ordering and a third piece gave results intermediate between these two extremes. We interpret the results to suggest that a strict serial execution of a movement sequence is favored as long as this is compatible with the demands of the task.

Evaluating an integrated musculoskeletal model of the human arm.

A simplified model of the mechanical properties of muscle and of the musculoskeletal geometry was used to predict torques at the shoulder and elbow during arm movements in the sagittal plane. Subjects made movements to 20 targets spaced on the diameter of a circle centered on the initial location of the hand. Movement kinematics and the electromyographic (EMG) activity of nine shoulder and elbow muscles were recorded. Muscle force was predicted using rectified EMG activity as an input to a Hill-type model of muscle dynamics. The model also made simplifying assumptions about muscle geometry. Muscle force was then converted to torque and the individual muscle torques were weighted to provide the best fit to the joint torque computed from the kinematic data. The overall fit of the model was reasonably good, but the goodness of fit was not uniform over all movement directions. The results suggest that the assumptions about the musculo-skeletal geometry, the model of muscle dynamics, and muscles not included in the analysis all contributed to the error.

Flexibility and repeatability of finger movements during typing: analysis of multiple degrees of freedom.

The kinematics of the hand and fingers were studied during various keystrokes in typing. These movements were defined by 17 degrees of freedom of motion, and methods were developed to identify simplifying strategies in the execution of the task. Most of the analysis was restricted to the 11 degrees of freedom of the fingers, neglecting thumb and wrist motion. Temporal characteristics of the motion were defined by computing principal components, and it was found that only a few (two to four) principal components were needed to characterize motion of each of the degrees of freedom. Hierarchical relationships among patterns within and between different degrees of freedom were identified using cluster analysis. There was a considerable amount of consistency each time a given keystroke was executed by a subject, and this repeatability may imply a reduction in the number of degrees of freedom independently controlled by the nervous system. However, there also appears to be considerable flexibility in the coordination of the many joints of the hand when examined across different keys and across different subjects.

Matching object size by controlling finger span and hand shape.

The ability of human subjects to accurately control finger span (distance between thumb and one finger) was studied. The experiments were performed without visual feedback of the hand and were designed to study the dependence of accuracy on object size, shape, distance, orientation and finger configuration. The effects of finger combination and sensory modality used to perceive object size (vision and haptics) were also studied. Subjects were quite proficient at this task; the small errors tended to be predominantly negative, i.e., finger span < object size. The thumb-little finger combination was less accurate than the other finger combinations, irrespective of the sensory modality used. Subjects made larger under-estimating errors when matching the size of cylinders than when matching cubes and parallelepipeds. No effect of viewing distance, object orientation and finger configuration was found. Accuracy in matching object size was not dependent on the sensory modality used. The question of how the individual degrees of freedom of the fingers and thumb contributed to the control of finger span was also addressed. Principal components analysis showed that two components could characterize the hand postures used, irrespective of object size. The amplitude of the first principal component was constant, and the amplitude of the second scaled linearly with object size. This finding suggests that all of the degrees of freedom of the hand are controlled as a unit. This result is discussed in relation to the 'virtual finger' hypothesis for grasping.

Somatosensory cortical activity in relation to arm posture: nonuniform spatial tuning.

1. Single unitary activity in primate somatosensory cortex (SI) was recorded while monkeys maintained a range of static arm postures. Unit discharge was related to parameters defining the posture of the arm by multiple linear regression techniques. 2. Two monkeys were trained to grasp a manipulandum presented at locations distributed throughout their workspace. The discharge of single units in SI was recorded for 3 s while the monkeys maintained contact with the manipulandum and the mean discharge rate over this hold time was related to the location of the hand and to the shoulder and elbow joint angles of the arm. 3. Unitary activity of 171 neurons in the proximal arm region of areas 3, 1, and 2 was recorded during the task. Of the total, 78 neurons had activity that varied with the location of the hand in space. Neuronal discharge typically varied monotonically with the target location, reaching a maximum at the borders of the work-space. The discharge rate in most of these neurons varied with both shoulder and elbow angles. 4. Discharge rate was related to the hand's location along three axes by means of a polynomial fit. In approximately half of the neurons, activity varied significantly only for displacements along a single axis in space. However, many neurons exhibited nonlinear relations between hand location along this preferred axis and discharge rate. Discharge rate did not vary for displacements of the hand in the plane perpendicular to this preferred axis (null plane). 5. In other neurons, discharge rate varied for hand displacements in a plane, i.e., along two perpendicular axes. Displacements of the hand along the axis perpendicular to this plane (null axis) did not affect the discharge rate. In only a small minority of neurons did discharge rate vary for hand displacements along all three axes in space. 6. The distribution of the sensitivity of the neural population to hand displacements along arbitrary directions in space was not uniform. On average, hand displacement along a vertical axis led to the smallest modulation of neural discharge, and displacement of the hand along the anteroposterior direction led to the largest modulation of activity.

Psychophysical approaches to motor control.

A variety of experimental approaches have recently helped identify the reference frames and coordinate systems that describe the control of eye and limb movements. These descriptions apply at the behavioral level and also, despite the distributed nature of neural processing, to the population responses of different neural structures. Studies on the process of adaptation to altered environments have also provided new insights into the controlled variables for movements: although handpaths can be adapted to extrinsic demands, the adaptation is, in some cases, in an intrinsic frame of reference.

Moving effortlessly in three dimensions: does Donders' law apply to arm movement?

Donders' law, as applied to the arm, predicts that to every location of the hand in space there corresponds a unique posture of the arm as defined by shoulder and elbow angles. This prediction was tested experimentally by asking human subjects to make pointing movements to a select number of target locations starting from a wide range of initial hand locations. The posture of the arm was measured at the start and end of every movement by means of video cameras. It was found that, in general, the posture of the arm at a given hand location does depend on the starting location of the movement and that, consequently, Donders' law is violated in this experimental condition. Kinematic and kinetic factors that could account for the variations in arm posture were investigated. It proved impossible to predict the final posture of the arm purely from kinematics, based on the initial posture of the arm. One hypothesis was successful in predicting final arm postures, namely that the final posture minimizes the amount of work that must be done to transport the arm from the starting location.

Use of tactile afferent information in sequential finger movements.

We have investigated how tactile afferent information contributes to the generation of sequences of skilled finger movements by anesthetizing the right index fingers of experienced typists. Subjects were asked to type phrases in which the right index finger was used only once every seven to 12 keypresses. The time at which each key was depressed was recorded with a digital timer, and the translational and rotational motion of the fingers and wrist of the right hand were recorded optoelectronically from the location of reflective markers placed on the fingers. Midway through the experiment, a local anesthetic was injected at the base of the distal phalange of the right index finger. Following digital anesthesia, error rates increased considerably, mainly due to the diminished accuracy of movements of the anesthetized finger. The typing intervals following keypresses with the anesthetized fingertip were unaffected by the removal of tactile information. When errors occurred during control trials, the intervals immediately following the errors were greatly prolonged. However, errors produced with the anesthetized right index finger did not influence the timing of subsequent keypresses, implying that lack of tactile cues affected error recognition. The movement patterns during keypresses were similar before and after digital anesthesia for some subjects, while a less pronounced flexion-extension movement was seen in other subjects. The results suggest that tactile afferent information is not essential for initiating movement segments in a sequence. Rather, they emphasize the importance of this information for ensuring movement accuracy and for detecting errors.

On the form of the internal model for reaching.

We investigated, by using simulations, possible mechanisms responsible for the errors in the direction of arm movements exhibited by deafferented patients. Two aspects of altered feedforward control were evaluated: the inability to sense initial conditions and the degradation of an internal model. A simulation which assumed no compensation for variations in initial arm configuration failed to reproduce the characteristic pattern of errors. In contrast, a simulation that assumed random variability in the generation of joint torque resulted in a distribution of handpaths which resembled some aspects of the pattern of errors exhibited by deafferented patients.