Multisensory convergence of visual and haptic object preference across development.
ABSTRACT: Visuohaptic inputs offer redundant and complementary information regarding an object?s geometrical structure. The integration of these inputs facilitates object recognition in adults. While the ability to recognize objects in the environment both visually and haptically develops early on, the development of the neural mechanisms for integrating visual and haptic object shape information remains unknown. In the present study, we used functional Magnetic Resonance Imaging (fMRI) in three groups of participants, 4 to 5.5 year olds, 7 to 8.5 year olds, and adults. Participants were tested in a block design involving visual exploration of two-dimensional images of common objects and real textures, and haptic exploration of their three-dimensional counterparts. As in previous studies, object preference was defined as a greater BOLD response for objects than textures. The analyses specifically target two sites of known visuohaptic convergence in adults: the lateral occipital tactile-visual region (LOtv) and intraparietal sulcus (IPS). Results indicated that the LOtv is involved in visuohaptic object recognition early on. More importantly, object preference in the LOtv became increasingly visually dominant with development. Despite previous reports that the lateral occipital complex (LOC) is adult-like by 8 years, these findings indicate that at least part of the LOC is not. Whole-brain maps showed overlap between adults and both groups of children in the LOC. However, the overlap did not build incrementally from the younger to the older group, suggesting that visuohaptic object preference does not develop in an additive manner. Taken together, the results show that the development of neural substrates for visuohaptic recognition is protracted compared to substrates that are primarily visual or haptic.
Project description:Research has provided strong evidence of multisensory convergence of visual and haptic information within the visual cortex. These studies implement crossmodal matching paradigms to examine how systems use information from different sensory modalities for object recognition. Developmentally, behavioral evidence of visuohaptic crossmodal processing has suggested that communication within sensory systems develops earlier than across systems; nonetheless, it is unknown how the neural mechanisms driving these behavioral effects develop. To address this gap in knowledge, BOLD functional Magnetic Resonance Imaging (fMRI) was measured during delayed match-to-sample tasks that examined intramodal (visual-to-visual, haptic-to-haptic) and crossmodal (visual-to-haptic, haptic-to-visual) novel object recognition in children aged 7-8.5 years and adults. Tasks were further divided into sample encoding and test matching phases to dissociate the relative contributions of each. Results of crossmodal and intramodal object recognition revealed the network of known visuohaptic multisensory substrates, including the lateral occipital complex (LOC) and the intraparietal sulcus (IPS). Critically, both adults and children showed crossmodal enhancement within the LOC, suggesting a sensitivity to changes in sensory modality during recognition. These groups showed similar regions of activation, although children generally exhibited more widespread activity during sample encoding and weaker BOLD signal change during test matching than adults. Results further provided evidence of a bilateral region in the occipitotemporal cortex that was haptic-preferring in both age groups. This region abutted the bimodal LOtv, and was consistent with a medial to lateral organization that transitioned from a visual to haptic bias within the LOC. These findings converge with existing evidence of visuohaptic processing in the LOC in adults, and extend our knowledge of crossmodal processing in adults and children.
Project description:Object interaction requires knowledge of the weight of an object, as well as its shape. The lateral occipital complex (LOC), an area within the ventral visual pathway, is well known to be critically involved in processing visual shape information. Recently, however, LOC has also been implicated in coding object weight before grasping-a result that is surprising because weight is a nonvisual object property that is more relevant for motor interaction than visual perception. Here, we examined the causal role of LOC in perceiving heaviness and in determining appropriate fingertip forces during object lifting. We studied perceptions of heaviness and lifting behavior in a neuropsychological patient (M.C.) who has large bilateral occipitotemporal lesions that include LOC. We compared the patient's performance to a group of 18 neurologically healthy age-matched controls. Participants were asked to lift and report the perceived heaviness of a set of equally weighted spherical objects of various sizes-stimuli which typically induce the size-weight illusion, in which the smaller objects feel heavier than the larger objects despite having identical mass. Despite her ventral stream lesions, M.C. experienced a robust size-weight illusion induced by visual cues to object volume, and the magnitude of the illusion in M.C. was comparable to age-matched controls. Similarly, M.C. evinced predictive fingertip force scaling to visual size cues during her initial lifts of the objects that were well within the normal range. These single-case neuropsychological findings suggest that LOC is unlikely to play a causal role in computing object weight.
Project description:Functional magnetic resonance imaging (fMRI) studies have provided ample evidence for the involvement of the lateral occipital cortex (LO), fusiform gyrus (FG), and intraparietal sulcus (IPS) in visuo-haptic object integration. Here we applied 30 min of sham (non-effective) or real offline 1 Hz repetitive transcranial magnetic stimulation (rTMS) to perturb neural processing in left LO immediately before subjects performed a visuo-haptic delayed-match-to-sample task during fMRI. In this task, subjects had to match sample (S1) and target (S2) objects presented sequentially within or across vision and/or haptics in both directions (visual-haptic or haptic-visual) and decide whether or not S1 and S2 were the same objects. Real rTMS transiently decreased activity at the site of stimulation and remote regions such as the right LO and bilateral FG during haptic S1 processing. Without affecting behavior, the same stimulation gave rise to relative increases in activation during S2 processing in the right LO, left FG, bilateral IPS, and other regions previously associated with object recognition. Critically, the modality of S2 determined which regions were recruited after rTMS. Relative to sham rTMS, real rTMS induced increased activations during crossmodal congruent matching in the left FG for haptic S2 and the temporal pole for visual S2. In addition, we found stronger activations for incongruent than congruent matching in the right anterior parahippocampus and middle frontal gyrus for crossmodal matching of haptic S2 and in the left FG and bilateral IPS for unimodal matching of visual S2, only after real but not sham rTMS. The results imply that a focal perturbation of the left LO triggers modality-specific interactions between the stimulated left LO and other key regions of object processing possibly to maintain unimpaired object recognition. This suggests that visual and haptic processing engage partially distinct brain networks during visuo-haptic object matching.
Project description:Previous work using functional magnetic resonance imaging has shown that the identities of isolated objects viewed by human subjects can be extracted from distributed patterns of brain activity. Outside the laboratory, however, objects almost never appear in isolation; thus it is important to understand how multiple simultaneously occurring objects are encoded by the visual system. We used multivoxel pattern analysis to examine this issue, testing whether activity patterns in the lateral occipital complex (LOC) evoked by object pairs showed an ordered relationship to patterns evoked by their constituent objects. Applying a searchlight analysis to identify voxels with the highest signal-to-noise ratios, we found that responses to object pairs within these informative voxels were well predicted by the averages of responses to their constituent objects. Consistent with this result, we were able to classify object pairs by using synthetic patterns created by averaging single-object patterns. These results indicate that the representation of multiple objects in LOC is governed by a response normalization mechanism similar to that reported in visual areas of several nonhuman species. They also suggest a population coding scheme that preserves information about multiple objects under conditions of distributed attention, facilitating fast object and scene recognition during natural vision.
Project description:High-level visual cortex in humans includes functionally defined regions that preferentially respond to objects, faces and places. It is unknown how these regions develop and whether their development relates to recognition memory. We used functional magnetic resonance imaging to examine the development of several functionally defined regions including object (lateral occipital complex, LOC)-, face ('fusiform face area', FFA; superior temporal sulcus, STS)- and place ('parahippocampal place area', PPA)-selective cortices in children (ages 7-11), adolescents (12-16) and adults. Right FFA and left PPA volumes were substantially larger in adults than in children. This development occurred by expansion of FFA and PPA into surrounding cortex and was correlated with improved recognition memory for faces and places, respectively. In contrast, LOC and STS volumes and object-recognition memory remained constant across ages. Thus, the ventral stream undergoes a prolonged maturation that varies temporally across functional regions, is determined by brain region rather than stimulus category, and is correlated with the development of category-specific recognition memory.
Project description:Abstract The lateral occipital complex (LOC) is a set of areas in the human occipito-temporal cortex responding to objects as opposed to low-level control stimuli. Conventional functional magnetic resonance imaging (fMRI) analysis methods based on regional averages could not detect signals discriminative of different types of objects in this region. Here, we examined fMRI signals using multivariate pattern recognition (support vector classification) to systematically explore the nature of object-related information available in fine-grained activity patterns in the LOC. Distributed fMRI signals from the LOC allowed for above-chance discrimination not only of the category but also of within-category exemplars of everyday man-made objects, and such exemplar-specific information generalized across changes in stimulus size and viewpoint, particularly in posterior subregions. Object identity could also be predicted from responses of the early visual cortex, even significantly across the changes in size and viewpoint used here. However, a dissociation was observed between these two regions of interest in the degree of discrimination for objects relative to size: In the early visual cortex, two different sizes of the same object were even better discriminated than two different objects (in accordance with measures of pixelwise stimulus similarity), whereas the opposite was true in the LOC. These findings provide the first evidence that direct evoked fMRI activity patterns in the LOC can be different for individual object exemplars (within a single category). We propose that pattern recognition methods as used here may provide an alternative approach to study mechanisms of neuronal representation based on aspects of the fMRI response independent of those assessed in adaptation paradigms.
Project description:The human lateral occipital complex (LOC) is more strongly activated by images of objects compared to scrambled controls, but detailed information at the neuronal level is currently lacking. We recorded with microelectrode arrays in the LOC of 2 patients and obtained highly selective single-unit, multi-unit, and high-gamma responses to images of objects. Contrary to predictions derived from functional imaging studies, all neuronal properties indicated that the posterior subsector of LOC we recorded from occupies an unexpectedly high position in the hierarchy of visual areas. Notably, the response latencies of LOC neurons were long, the shape selectivity was spatially clustered, LOC receptive fields (RFs) were large and bilateral, and a number of LOC neurons exhibited three-dimensional (3D)-structure selectivity (a preference for convex or concave stimuli), which are all properties typical of end-stage ventral stream areas. Thus, our results challenge prevailing ideas about the position of the more posterior subsector of LOC in the hierarchy of visual areas.
Project description:Although visual cortical engagement in haptic shape perception is well established, its relationship with visual imagery remains controversial. We addressed this using functional magnetic resonance imaging during separate visual object imagery and haptic shape perception tasks. Two experiments were conducted. In the first experiment, the haptic shape task employed unfamiliar, meaningless objects, whereas familiar objects were used in the second experiment. The activations evoked by visual object imagery overlapped more extensively, and their magnitudes were more correlated, with those evoked during haptic shape perception of familiar, compared to unfamiliar, objects. In the companion paper (Deshpande et al., this issue), we used task-specific functional and effective connectivity analyses to provide convergent evidence: these analyses showed that the neural networks underlying visual imagery were similar to those underlying haptic shape perception of familiar, but not unfamiliar, objects. We conclude that visual object imagery is more closely linked to haptic shape perception when objects are familiar, compared to when they are unfamiliar.
Project description:In this paper, we assess the importance of visual and haptic information about materials for scaling the grasping force when picking up an object. We asked 12 participants to pick up and lift objects with six different textures, either blindfolded or with visual information present. We measured the grip force and estimated the load force from the object's weight and vertical acceleration. The coefficient of friction of the materials was measured separately. Already at an early phase in the grasp (before lift-off), the grip force correlated highly with the textures' static coefficient of friction. However, no strong influence on the presence of visual information was found. We conclude that the main mechanism for modulation of grip force in the early phase of grasping is the real-time sensation of the texture's friction.
Project description:Real-world visual scenes are complex cluttered, and heterogeneous stimuli engaging scene- and object-selective cortical regions including parahippocampal place area (PPA), retrosplenial complex (RSC), and lateral occipital complex (LOC). To understand the unique contribution of each region to distributed scene representations, we generated predictions based on a neuroanatomical framework adapted from monkey and tested them using minimal scenes in which we independently manipulated both spatial layout (open, closed, and gradient) and object content (furniture, e.g., bed, dresser). Commensurate with its strong connectivity with posterior parietal cortex, RSC evidenced strong spatial layout information but no object information, and its response was not even modulated by object presence. In contrast, LOC, which lies within the ventral visual pathway, contained strong object information but no background information. Finally, PPA, which is connected with both the dorsal and the ventral visual pathway, showed information about both objects and spatial backgrounds and was sensitive to the presence or absence of either. These results suggest that 1) LOC, PPA, and RSC have distinct representations, emphasizing different aspects of scenes, 2) the specific representations in each region are predictable from their patterns of connectivity, and 3) PPA combines both spatial layout and object information as predicted by connectivity.