Looking at the Ebbinghaus illusion: differences in neurocomputational requirements, not gaze-mediated attention, explain a classic perception-action dissociation.

Perceiving and grasping an object present an animal with different sets of computational problems. The solution in primates entails the specialization of separate neural networks for visual processing with different object representations. This explains why the Ebbinghaus illusion minimally affects the grasping hand's in-flight aperture, which normally scales with target size, even though the size of the target disc remains misperceived. An attractive alternative account, however, posits that grasps are refractory to the illusion because participants fixate on the target and fail to attend to the surrounding context. To test this account, we tracked both limb and gaze while participants made forced-choice judgments of relative disc size in the Ebbinghaus illusion or did so in combination with grasping or manually estimating the size of one of the discs. We replicated the classic dissociation: grasp aperture was refractory to the measured illusory effect on perceived size, while judgments and manual estimates of disc size were not. Importantly, the number of display-wide saccades per second and the percentage of total fixation time or fixations directed at the selected disc failed to explain the dissociation. Our findings support the contention that object perception and goal-directed action rely on distinct visual representations. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

The Ties that Bind: Agnosia, Neglect and Selective Attention to Visual Scale.

PURPOSE OF REVIEW: Historical and contemporary treatments of visual agnosia and neglect regard these disorders as largely unrelated. It is thought that damage to different neural processes leads directly to one or the other condition, yet apperceptive variants of agnosia and object-centered variants of neglect share remarkably similar deficits in the quality of conscious experience. Here we argue for a closer association between "apperceptive" variants of visual agnosia and "object-centered" variants of visual neglect. We introduce a theoretical framework for understanding these conditions based on "scale attention", which refers to selecting boundary and surface information at different levels of the structural hierarchy in the visual array. RECENT FINDINGS: We review work on visual agnosia, the cortical structures and cortico-cortical pathways that underlie visual perception, visuospatial neglect and object-centered neglect, and attention to scale. We highlight direct and indirect pathways involved in these disorders and in attention to scale. The direct pathway involves the posterior vertical segments of the superior longitudinal fasciculus that are positioned to link the established dorsal and ventral attentional centers in the parietal cortex with structures in the inferior occipitotemporal cortex associated with visual apperceptive agnosia. The connections in the right hemisphere appear to be more important for visual conscious experience, whereas those in the left hemisphere appear to be more strongly associated with the planning and execution of visually guided grasps directed at multi-part objects such as tools. In the latter case, semantic and functional information must drive the selection of the appropriate hand posture and grasp points on the object. This view is supported by studies of grasping in patients with agnosia and in patients with neglect that show that the selection of grasp points when picking up a tool involves both scale attention and semantic contributions from inferotemporal cortex. The indirect pathways, which include the inferior fronto-occipital and horizontal components of the superior longitudinal fasciculi, involve the frontal lobe, working memory and the "multiple demands" network, which can shape the content of visual awareness through the maintenance of goal- and task-based abstractions and their influence on scale attention. Recent studies of human cortico-cortical pathways necessitate revisions to long-standing theoretical views on visual perception, visually guided action and their integrations. We highlight findings from a broad sample of seemingly disparate areas of research to support the proposal that attention to scale is necessary for typical conscious visual experience and for goal-directed actions that depend on functional and semantic information. Furthermore, we suggest that vertical pathways between the parietal and occipitotemporal cortex, along with indirect pathways that involve the premotor and prefrontal cortex, facilitate the operations of scale attention.

Grasping of Real-World Objects Is Not Biased by Ensemble Perception.

The visual system is known to extract summary representations of visually similar objects which bias the perception of individual objects toward the ensemble average. Although vision plays a large role in guiding action, less is known about whether ensemble representation is informative for action. Motor behavior is tuned to the veridical dimensions of objects and generally considered resistant to perceptual biases. However, when the relevant grasp dimension is not available or is unconstrained, ensemble perception may be informative to behavior by providing gist information about surrounding objects. In the present study, we examined if summary representations of a surrounding ensemble display influenced grip aperture and orientation when participants reached-to-grasp a central circular target which had an explicit size but importantly no explicit orientation that the visuomotor system could selectively attend to. Maximum grip aperture and grip orientation were not biased by ensemble statistics during grasping, although participants were able to perceive and provide manual estimations of the average size and orientation of the ensemble display. Support vector machine classification of ensemble statistics achieved above-chance classification accuracy when trained on kinematic and electromyography data of the perceptual but not grasping conditions, supporting our univariate findings. These results suggest that even along unconstrained grasping dimensions, visually-guided behaviors toward real-world objects are not biased by ensemble processing.

Grip Constancy but Not Perceptual Size Constancy Survives Lesions of Early Visual Cortex.

Object constancies are central constructs in theories of visual phenomenology. A powerful example is "size constancy," in which the perceived size of an object remains stable despite changes in viewing distance [1-4]. Evidence from neuropsychology [5], neuroimaging [6-11], transcranial magnetic stimulation [12, 13], single-unit and lesion studies in monkey [14-20], and computational modeling [21] suggests that re-entrant processes involving reciprocal interactions between primary visual cortex (V1) and extrastriate visual areas [22-26] play an essential role in mediating size constancy. It is seldom appreciated, however, that object constancies must also operate for the visual guidance of goal-directed action. For example, when reaching out to pick up an object, the hand's in-flight aperture scales with size of the goal object [27-30] and is refractory to the decrease in retinal-image size with increased viewing distance [31-41] (Figure 1), a phenomenon we call "grip constancy." Does grip constancy, like perceptual constancy, depend on V1 or can it be mediated by pathways that bypass it altogether? We tested these possibilities in an individual, M.C., who has bilateral lesions encompassing V1 and much of the ventral visual stream. We show that her perceptual estimates of object size co-vary with retinal-image size rather than real-world size as viewing distance varies. In contrast, M.C. shows near-normal scaling of in-flight grasp aperture to object size despite changes in viewing distance. Thus, although early visual cortex is necessary for perceptual object constancy, it is unnecessary for grip constancy, which is mediated instead by separate visual inputs to dorsal-stream visuomotor areas [42-48].

The Role of Haptic Expectations in Reaching to Grasp: From Pantomime to Natural Grasps and Back Again.

When we reach to pick up an object, our actions are effortlessly informed by the object's spatial information, the position of our limbs, stored knowledge of the object's material properties, and what we want to do with the object. A substantial body of evidence suggests that grasps are under the control of "automatic, unconscious" sensorimotor modules housed in the "dorsal stream" of the posterior parietal cortex. Visual online feedback has a strong effect on the hand's in-flight grasp aperture. Previous work of ours exploited this effect to show that grasps are refractory to cued expectations for visual feedback. Nonetheless, when we reach out to pretend to grasp an object (pantomime grasp), our actions are performed with greater cognitive effort and they engage structures outside of the dorsal stream, including the ventral stream. Here we ask whether our previous finding would extend to cued expectations for haptic feedback. Our method involved a mirror apparatus that allowed participants to see a "virtual" target cylinder as a reflection in the mirror at the start of all trials. On "haptic feedback" trials, participants reached behind the mirror to grasp a size-matched cylinder, spatially coincident with the virtual one. On "no-haptic feedback" trials, participants reached behind the mirror and grasped into "thin air" because no cylinder was present. To manipulate haptic expectation, we organized the haptic conditions into blocked, alternating, and randomized schedules with and without verbal cues about the availability of haptic feedback. Replicating earlier work, we found the strongest haptic effects with the blocked schedules and the weakest effects in the randomized uncued schedule. Crucially, the haptic effects in the cued randomized schedule was intermediate. An analysis of the influence of the upcoming and immediately preceding haptic feedback condition in the cued and uncued random schedules showed that cuing the upcoming haptic condition shifted the haptic influence on grip aperture from the immediately preceding trial to the upcoming trial. These findings indicate that, unlike cues to the availability of visual feedback, participants take advantage of cues to the availability of haptic feedback, flexibly engaging pantomime, and natural modes of grasping to optimize the movement.

Visuomotor adaptation in the absence of input from early visual cortex.

Prism adaptation is a time-honored tool for studying how the motor system adapts to sensory perturbations. Past research on the neural substrates of prism adaptation has implicated the posterior parietal cortex (PPC) and the cerebellum, under the assumption that these structures gain their visual input from the dominant retinogeniculate pathway to V1. Here we question whether this pathway is even required for visuomotor adaptation to occur. To investigate this, we examined prism adaptation in 'MC', someone who is blind to static stimuli following bilateral lesions that encompass much of her occipital cortex and the caudal-most areas of ventrotemporal cortex. Remarkably, MC shows evidence of prism adaptation that is similar to healthy control participants. First, when pointing with either the left or the right hand, MC shows spatial realignment; the classical after-effect exhibited by most people when adapting to displacing prisms. Second, MC demonstrates strategic recalibration - a reduction in her pointing error over time - that is similar in magnitude to healthy controls. These findings suggest that the geniculostriate pathway is not necessary for visuomotor adaptation to take place. Alternatively, we suggest that an extrageniculostriate pathway which provides visual inputs to the cerebellum from area MT and the PPC via the dorsolateral pons plays a significant and appreciable role in the guidance of unconscious automatic visuomotor adaptation.

Affective blindsight in the absence of input from face processing regions in occipital-temporal cortex.

Previous research suggests that the implicit recognition of emotional expressions may be carried out by pathways that bypass primary visual cortex (V1) and project to the amygdala. Some of the strongest evidence supporting this claim comes from case studies of "affective blindsight" in which patients with V1 damage can correctly guess whether an unseen face was depicting a fearful or happy expression. In the current study, we report a new case of affective blindsight in patient MC who is cortically blind following extensive bilateral lesions to V1, as well as face and object processing regions in her ventral visual stream. Despite her large lesions, MC has preserved motion perception which is related to sparing of the motion sensitive region MT+ in both hemispheres. To examine affective blindsight in MC we asked her to perform gender and emotion discrimination tasks in which she had to guess, using a two-alternative forced-choice procedure, whether the face presented was male or female, happy or fearful, or happy or angry. In addition, we also tested MC in a four-alternative forced-choice target localization task. Results indicated that MC was not able to determine the gender of the faces (53% accuracy), or localize targets in a forced-choice task. However, she was able to determine, at above chance levels, whether the face presented was depicting a happy or fearful (67%, p = .006), or a happy or angry (64%, p = .025) expression. Interestingly, although MC was better than chance at discriminating between emotions in faces when asked to make rapid judgments, her performance fell to chance when she was asked to provide subjective confidence ratings about her performance. These data lend further support to the idea that there is a non-conscious visual pathway that bypasses V1 which is capable of processing affective signals from facial expressions without input from higher-order face and object processing regions in the ventral visual stream.

The left hand disrupts subsequent right hand grasping when their actions overlap.

Adaptive motor control is premised on the principle of movement minimization, which in turn is premised on a form of sensorimotor memory. But what is the nature of this memory and under what conditions does it operate? Here, we test the limits of sensorimotor memory in an intermanual context by testing the effect that the action performed by the left hand has on subsequent right hand grasps. Target feature-overlap predicts that sensorimotor memory is engaged when task-relevant sensory features of the target are similar across actions; partial effector-overlap predicts that sensorimotor memory is engaged when there is similarity in the task-relevant effectors used to perform an action; and the action-goal conjunction hypotheses predicts that sensorimotor memories are engaged when the action goal and the action type overlap. In three experiments, participants used their left hand to reach out and pick up an object, manually estimate its size, pinch it, look at it, or merely rest the left hand before reaching out to pick up a second object with their right hand. The in-flight anticipatory grip aperture of right-hand grasps was only influenced when it was preceded by grasps performed by the left-hand. Overlap in the sizes of the objects, partial overlap in the effectors used, and in the availability of haptic feedback bore no influence on this metric. These results support the hypothesis that intermanual transfer of sensorimotor memory on grasp execution is dependent on a conjunction of action type and goal.

Predictive joint-action model: A hierarchical predictive approach to human cooperation.

Research in a number of related fields has recently begun to focus on the perceptual, cognitive, and motor workings of cooperative behavior. There appears to be enough coherence in these efforts to refer to the study of the mechanisms underlying human cooperative behavior as the field of joint-action (Knoblich, Butterfill, & Sebanz, 2011; Sebanz, Bekkering, & Knoblich, 2006). Yet, the development of theory in this field has not kept pace with the proliferation of research findings. We propose a hierarchical predictive framework for the study of joint-action that we call the predictive joint-action model (PJAM). The presentation of this theoretical framework is organized into three sections. In the first section, we summarize hierarchical predictive principles and discuss their application to joint-action. In the second section, we juxtapose PJAM's assumptions with empirical evidence from the current literature on joint-action. In the third section, we discuss the overall success of the hierarchical predictive approach to account for the burgeoning empirical literature on joint-action research. Finally, we consider the model's capacity to generate novel and testable hypotheses about joint-action. This is done with the larger goal of uncovering the empirical and theoretical pieces that are still missing in a comprehensive understanding of joint action.

The Sander parallelogram illusion dissociates action and perception despite control for the litany of past confounds.

The two visual systems hypothesis proposes that human vision is supported by an occipito-temporal network for the conscious visual perception of the world and a fronto-parietal network for visually-guided, object-directed actions. Two specific claims about the fronto-parietal network's role in sensorimotor control have generated much data and controversy: (1) the network relies primarily on the absolute metrics of target objects, which it rapidly transforms into effector-specific frames of reference to guide the fingers, hands, and limbs, and (2) the network is largely unaffected by scene-based information extracted by the occipito-temporal network for those same targets. These two claims lead to the counter-intuitive prediction that in-flight anticipatory configuration of the fingers during object-directed grasping will resist the influence of pictorial illusions. The research confirming this prediction has been criticized for confounding the difference between grasping and explicit estimates of object size with differences in attention, sensory feedback, obstacle avoidance, metric sensitivity, and priming. Here, we address and eliminate each of these confounds. We asked participants to reach out and pick up 3D target bars resting on a picture of the Sander Parallelogram illusion and to make explicit estimates of the length of those bars. Participants performed their grasps without visual feedback, and were permitted to grasp the targets after making their size-estimates to afford them an opportunity to reduce illusory error with haptic feedback. The results show unequivocally that the effect of the illusion is stronger on perceptual judgments than on grasping. Our findings from the normally-sighted population provide strong support for the proposal that human vision is comprised of functionally and anatomically dissociable systems.

Attention in action and perception: Unitary or separate mechanisms of selectivity?

What is the relation between the two visual stream hypothesis and selective visual attention? In this chapter, we first consider this question at a theoretical level before presenting an example of work from our lab that examines the question: Under what conditions does the emotional content of a visual object influence visually guided action? Previous research has demonstrated that fear can influence perception, both consciously and unconsciously, but it is unclear when fear influences visually guided action. The study tested participants with varying degrees of spiderphobia on two visually guided pointing tasks, while manipulating the emotional valence of the target (positive and negative) and the cognitive load of the participant (single vs dual task). Participants rapidly moved their finger from a home position to a suddenly appearing target image on a touch screen. The images were emotionally negative (e.g., spiders and scorpions) or positive (e.g., flowers and food). In order to test the effect of emotional valence on the online control of the reach, the target either remained static or jumped to a new location. In both the single and dual tasks, a stream of digits were presented on the screen near the finger's starting location, but only in the dual task were participants asked to identify a letter somewhere in the stream. In the single task, increased fear of spiders reduced the speed and accuracy of the movement. In the dual task, increased fear impaired letter identification, but pointing actions were now equally efficient for low- and high-fear participants. These results imply that the finger's autopilot is influenced by emotional content only when attention can be fully devoted to the identification of the emotion-evoking images. As such, the results support the view that the mechanisms of selection are not the same in the two visual streams.

A TMS Investigation on the Role of Lateral Occipital Complex and Caudal Intraparietal Sulcus in the Perception of Object Form and Orientation.

We used TMS to assess the causal roles of the lateral occipital (LO) and caudal intraparietal sulcus (cIPS) areas in the perceptual discrimination of object features. All participants underwent fMRI to localize these areas using a protocol in which they passively viewed images of objects that varied in both form and orientation. fMRI identified six significant brain regions: LO, cIPS, and the fusiform gyrus, bilaterally. In a separate experimental session, we applied TMS to LO or cIPS while the same participants performed match-to-sample form or orientation discrimination tasks. Compared with sham stimulation, TMS to either the left or right LO increased RTs for form but not orientation discrimination, supporting a critical role for LO in form processing for perception- and judgment-based tasks. In contrast, we did not observe any effects when we applied TMS to cIPS. Thus, despite the clear functional evidence of engagement for both LO and cIPS during the passive viewing of objects in the fMRI experiment, the TMS experiment revealed that cIPS is not critical for making perceptual judgments about their form or orientation.

Biomechanical constraints do not influence pantomime-grasping adherence to Weber's law: A reply to Utz et al. (2015).

Work by our group and others employed the within-participants variability in peak grip aperture as a 'just-noticeable-difference' (JND) in grasping. Notably, our group reported that grasping responses with decoupled spatial relations between stimulus and response (i.e., pantomime-grasping) produced JNDs that increased linearly with increasing target object size (i.e., adherence to Weber's law) and interpreted that result as law-based evidence of aperture shaping via relative visual information. In contrast, Utz et al. (2015) reported that pantomime-grasping elicits an inverse JND/object size relationship and proposed that JNDs in grasping do not reflect the sensory properties of a target object but rather reflect range effects in the biomechanical limits of aperture opening (i.e., the biomechanical hypothesis). Thus, the biomechanical hypothesis asserts that small objects have a larger range of possible aperture values than larger objects due to reduced biomechanical freedom associated with the hand's effective range of motion. To test the biomechanical hypothesis we measured participants' maximal thumb and forefinger separation and custom-built target objects with widths that matched decile increments (i.e., 10 through 80%) of each participant's effective range of motion. Results showed that JNDs increased linearly with increasing target object size - a result incompatible with the biomechanical hypothesis. Instead, the JND/object size relationship observed here supports convergent evidence that pantomime-grasping is a perception-based task mediated via relative visual information.

Unusual hand postures but not familiar tools show motor equivalence with precision grasping.

A central question in sensorimotor control is whether or not actions performed with the hands and corresponding actions performed with tools share a common underlying motor plan, even though different muscles and effectors are engaged. There is certainly evidence that tools used to extend the reach of the limb can be incorporated into the body schema after training. But even so, it is not clear whether or not actions such as grasping with tools and grasping with the fingers share the same programming network, i.e. show 'motor equivalence'. Here we first show that feedback-appropriate motor programming for grasps with atypical hand postures readily transfers to stereotypical precision grasps. In stark contrast, however, we find no evidence for an analogous transfer of the programming for grasps using tools to the same stereotypical precision grasps. These findings have important implications for our understanding of body schema. Although the extension of the limb that is afforded by tool use may be incorporated into the body schema, the programming of a grasping movement made with tools appears to resist such incorporation. It could be the case that the proprioceptive signals from the limb can be easily updated to reflect the end of a tool held in the hand, but the motor programs and sensory signals associated with grasping with the thumb and finger cannot be easily adapted to control the opening and closing of a tool. Instead, new but well-practiced motor programs are put in place for tool use that do not exhibit motor equivalence with manual grasping.

Rapid decrement in the effects of the Ponzo display dissociates action and perception.

It has been demonstrated that pictorial illusions have a smaller influence on grasping than they do on perceptual judgments. Yet to date this work has not considered the reduced influence of an illusion as it is measured repeatedly. Here we studied this decrement in the context of a Ponzo illusion to further characterize the dissociation between vision for perception and for action. Participants first manually estimated the lengths of single targets in a Ponzo display with their thumb and index finger, then actually grasped these targets in another series of trials, and then manually estimated the target lengths again in a final set of trials. The results showed that although the perceptual estimates and grasp apertures were equally sensitive to real differences in target length on the initial trials, only the perceptual estimates remained biased by the illusion over repeated measurements. In contrast, the illusion's effect on the grasps decreased rapidly, vanishing entirely after only a few trials. Interestingly, a closer examination of the grasp data revealed that this initial effect was driven largely by undersizing the grip aperture for the display configuration in which the target was positioned between the diverging background lines (i.e., when the targets appeared to be shorter than they really were). This asymmetry between grasping apparently shorter and longer targets suggests that the sensorimotor system may initially treat the edges of the configuration as obstacles to be avoided. This finding highlights the sensorimotor system's ability to rapidly update motor programs through error feedback, manifesting as an immunity to the effects of illusion displays even after only a few trials.

Real-time vision, tactile cues, and visual form agnosia: removing haptic feedback from a "natural" grasping task induces pantomime-like grasps.

Investigators study the kinematics of grasping movements (prehension) under a variety of conditions to probe visuomotor function in normal and brain-damaged individuals. "Natural" prehensile acts are directed at the goal object and are executed using real-time vision. Typically, they also entail the use of tactile, proprioceptive, and kinesthetic sources of haptic feedback about the object ("haptics-based object information") once contact with the object has been made. Natural and simulated (pantomimed) forms of prehension are thought to recruit different cortical structures: patient DF, who has visual form agnosia following bilateral damage to her temporal-occipital cortex, loses her ability to scale her grasp aperture to the size of targets ("grip scaling") when her prehensile movements are based on a memory of a target previewed 2 s before the cue to respond or when her grasps are directed towards a visible virtual target but she is denied haptics-based information about the target. In the first of two experiments, we show that when DF performs real-time pantomimed grasps towards a 7.5 cm displaced imagined copy of a visible object such that her fingers make contact with the surface of the table, her grip scaling is in fact quite normal. This finding suggests that real-time vision and terminal tactile feedback are sufficient to preserve DF's grip scaling slopes. In the second experiment, we examined an "unnatural" grasping task variant in which a tangible target (along with any proxy such as the surface of the table) is denied (i.e., no terminal tactile feedback). To do this, we used a mirror-apparatus to present virtual targets with and without a spatially coincident copy for the participants to grasp. We compared the grasp kinematics from trials with and without terminal tactile feedback to a real-time-pantomimed grasping task (one without tactile feedback) in which participants visualized a copy of the visible target as instructed in our laboratory in the past. Compared to natural grasps, removing tactile feedback increased RT, slowed the velocity of the reach, reduced in-flight grip aperture, increased the slopes relating grip aperture to target width, and reduced the final grip aperture (FGA). All of these effects were also observed in the real time-pantomime grasping task. These effects seem to be independent of those that arise from using the mirror in general as we also compared grasps directed towards virtual targets to those directed at real ones viewed directly through a pane of glass. These comparisons showed that the grasps directed at virtual targets increased grip aperture, slowed the velocity of the reach, and reduced the slopes relating grip aperture to the widths of the target. Thus, using the mirror has real consequences on grasp kinematics, reflecting the importance of task-relevant sources of online visual information for the programming and updating of natural prehensile movements. Taken together, these results provide compelling support for the view that removing terminal tactile feedback, even when the grasps are target-directed, induces a switch from real-time visual control towards one that depends more on visual perception and cognitive supervision. Providing terminal tactile feedback and real-time visual information can evidently keep the dorsal visuomotor system operating normally for prehensile acts.

The two-visual-systems hypothesis and the perspectival features of visual experience.

Some critics of the two-visual-systems hypothesis (TVSH) argue that it is incompatible with the fundamentally egocentric nature of visual experience (what we call the 'perspectival account'). The TVSH proposes that the ventral stream, which delivers up our visual experience of the world, works in an allocentric frame of reference, whereas the dorsal stream, which mediates the visual control of action, uses egocentric frames of reference. Given that the TVSH is also committed to the claim that dorsal-stream processing does not contribute to the contents of visual experience, it has been argued that the TVSH cannot account for the egocentric features of our visual experience. This argument, however, rests on a misunderstanding about how the operations mediating action and the operations mediating perception are specified in the TVSH. In this article, we emphasize the importance of the 'outputs' of the two-systems to the specification of their respective operations. We argue that once this point is appreciated, it becomes evident that the TVSH is entirely compatible with a perspectival account of visual experience.