by
|
by |
|
| This article will appear in a forthcoming issue of Trends in Cognitive Sciences. | |
The relation between vision and touch is an old chestnut, going back to the earliest days of psychology. The famous "Molyneux's question" asks whether a fomerly blind person, to whom sight were suddenly restored by some unspecified miraculous medical intervention, would immediately be able to discriminate a sphere from a cube on visual presentation alone. The empiricist's counter-argument is that it would be necessary to touch the objects first, in order to learn the relation between the objects' retinal projection and their volumetric shape [1]. The answer to this question would set a framework for interactions between sensory modalities: does touch lead vision or not?
Those who have revisited the question experimentally have generally been concerned with integration of visual and proprioceptive information about body position, rather than about external objects. Such studies have used tasks in which vision and somatic sensation give conflicting information [2], and have generally concluded that vision wins in such conflicts. To give just one example, vibrating the biceps tendon of the arm causes a compelling proprioceptive illusion in blindfolded subjects, in which the subject feels their elbow to be much more flexed, and their forearm much closer to the body than it really is. When subjects see their forearm during this procedure, there is no illusion of change in position [3]. Several behavioural studies have suggested that our normal body position sense is a weighted combination of visual and proprioceptive inputs [4,5] and have discovered the rules of sensor fusion that the brain uses to combine them [6].
The recent discovery of bimodal cells in multiple brain areas, including the putamen, premotor and parietal cortices, has resulted in how such multi-sensory information about body position is integrated in the brain. Interestingly, two classes of bimodal neurons were found in a recent study of monkey parietal cortex [7]. When the monkey's static arm was occluded from view, and a stuffed taxidermic arm moved across the workspace, some neurons showed visual receptive fields that shifted with the visually observed position of the fake arm. These neurons are clearly visually led, in that their response properties change when the visual inputs from the body change while the proprioceptive ones do not. Other neurons displayed the opposite behaviour: when the monkey moved its real arm unseen beneath the worksurface, but the fake arm remained statically in the same visible position above the surface, the visual receptive field shifted to coincide with the new felt position of the fake arm. These neurons are clearly somatically led, in that their response properties change when the proprioceptive inputs from the body change while the visual ones do not. From this evidence, it seems that the brain maintains multiple representations of body position, each generated by a different sensor fusion weighting.
What has been missing, up to now, has been the link between such multimodal neural representations and behavioural performance. One recent study suggests neuropsychological studies of brain damaged patients may provide that link. Newport, Hindle and Jackson [8] tested a patient, "CT," who had impaired somatosensation in her right limb following a thalamic lesion. In a matching task, the experimenters placed the index finger of one of the patient's hands (the "target" hand) unseen below the tabletop, and asked the patient to reach out to place the index finger of her other hand (the "reaching" hand) directly above the index finger of the target hand. The paper focuses on how the matching error (the distance between the two fingertips) varies as a result of the sensory information available during the task.
In one condition, CT was blindfold, and thus no visual information about either target or reaching limb was available. She made large matching errors when her impaired hand was the target hand, but minimal errors when her normal hand was the target hand. The reduced somatosensory function made her impaired hand feel much closer to the body than it really was. Most interestingly, these large errors disappeared when CT was allowed to have vision of her unimpaired reaching hand. The conundrum is why did vision of the reaching hand help: the impaired target hand remained hidden from view beneath the worksurface, and vision of the reaching hand and of the top of the surface gave no clue to the target hand location underneath The authors suggest that merely viewing the worksurface adjacent to the target limb can improve the proprioceptive representation of the target.
This finding requires a change in our thinking about sensor fusion. Previous studies using similar matching tasks proposed that the CNS integrates proprioceptive information with visual information about the position of the target limb [5,6]. In the present case, however, degraded somatic information about limb position seems to be integrated and improved by some visual process that does not directly contribute information about the target limb position. What might this process be? Two possibilities exist in the literature. The first would be a process of initial visual calibration of position sense. Gordon et al. have shown that patients with proprioceptive loss show big improvements in aimed movement if they are allowed to glimpse briefly the starting position of the hand before movement [9]. Proprioceptive information about limb position is known to drift over time [10]. The initial visual signal presumably realigns the felt position of the limb with its seen position. However, the patient in the present study was blindfolded throughout the critical trials in which she made proprioceptively-guided reaches to the felt location of the target hand.
A second possibility would be that the act of moving the eyes towards the felt position of the impaired target hand provides some additional non-retinal information which can enhance the residual proprioceptive signal from the target hand. Gaze signals are known to be important in the guidance of eye and hand movement [11].
The key question regarding the possible use of gaze information in this study seems to be where is CT looking? Does she look at the actual position of her target hand? Because neither proprioception nor vision can tell her accurately where the target hand is, it is hard to see how she could do so. Does she look at the felt position of the impaired hand, closer to the body than its actual position? If she does, it is hard to see how adding gaze information about the same incorrect position can so dramatically reduce her error. Nevertheless, a follow-up experiment suggests this does occur: CT had smaller matching errors when she could fixate the area of the workspace above the target hand than when she was required to fixate at the far edge of the workspace [8].
In conclusion, the mixture of somatic and visual information about body configuration appears to be more complex than previously thought. Non-retinal signals might also contribute to our perception of our body's arrangement in space. There is increasing evidence that the brain carries representations of our volumetric body in three distinct ways. First, we have a first-person representation based on proprioceptive sensation. Second, we have a largely visual representation of the location of our body parts in egocentric space. Thirdly, we have a hybrid resulting from one or more interactions of the first two. In this respect of sensor fusion, then, the representation of the body appears to be a very special case of the general process of sensory integration. In this special case, sensory fusion for the control of reaching has the interesting dual property of being proprioceptively led, and visually enhanced.
