The primary visual cortex (V1) has traditionally been considered to be a piece of neural machinery that is relatively hardwired to perform and edge detection. In this talk, I will present some recent evidence from primate electrophysiological experiments that demonstrate that V1 neurons are in fact quite dynamic and plastic in what they are encoding and in the way they are computing. For example, they are sensitive to the statistics of the stimulus environment, changing their tuning properties on the fly, and that their processing of perceptual information is a function of higher order stimulus properties, behavioral experience and task demands. Taken together, we conclude that V1 is not simply the first stage of visual processing, but an integral part of the reciprocally connected system that participates in many levels of visual perceptual inference. I will also sketch a computational framework on how V1 can serve as a high-resolution buffer for integrating information and coordinating computation across multiple visual modules.

A widely held view in the field of multisensory science is that sensory information from a single object (e.g. a barking dog) is first processed extensively through the respective unisensory processing streams before the information is combined in higher-order multisensory regions of cortex. Under this view, multisensory modulations which have recently been found in early sensory cortices during hemodynamic imaging studies have been interpreted as reflecting feedback modulations that occur subsequent to multisensory processing in the higher-order areas. Of course, hemodynamic imaging cannot address the relative timing of such activations. In this talk, I will focus on recent evidence from both human and monkey electrophysiological investigations as well as hemodynamic imaging studies that challenge an exclusively feedback model. Specifically, these studies show that multisensory integration effects are seen at very early latencies relative to the well-established timecourse of sensory activation within the respective unisensory processing streams. In fact, intracranial recordings show that multisensory inputs found in early sensory cortices display a characteristic feedforward pattern of inputs. For many years, it was believed that there was no direct anatomic connections between early sensory cortices of the different sensory modalities but recent anatomic tracer studies have begun to uncover direct connections between early visual and auditory sensory regions, providing the necessary anatomic substrate for such early interactions. In light of these recent findings, it is becoming evident that models of multisensory cortical processing will need to incorporate early feedforward integration mechanisms as well as feedback. These data highlight the tremendous potential of "multimodality" imaging, whereby data from the macrospcopic noninvasive level (fMRI and high-density scalp recordings in humans) are integrated with equivalent data from the more microscopic/cellular level which we record intracranially in non-human primates.

For many years, our understanding of the brain relied largely on data from macaque monkeys and other animals, because it is usually not possible to study the human brain invasively. During the past decade, this information from animals has been increasingly supplemented by noninvasive studies of human brain, using functional magnetic resonance imaging (fMRI). However it still remained impossible to compare these two data sets directly , since the data from monkeys was generated using quite different techniques (e.g., single unit recording, neural tracing) compared to the techniques used to study human brain (e.g. fMRI, psychophysics, etc.). Thus one often could not say whether a given discrepancy arose from differences between the species tested (e.g. humans versus macaques) or in the techniques used to reveal those differences (e.g. single unit recording versus fMRI).

To resolve some of these issues, and to open up a wide range of potential future experiments, we began doing fMRI in visual cortex of macaque monkeys. Such studies were done at 3 T, in a horizontal bore scanner, using the BOLD signal, in awake behaving primates --- exactly as in parallel fMRI experiments on human subjects. To minimize the experimental differences even more, we showed both human and macaque subjects the same visual stimuli, and had them perform equivalent tasks in the scanner. Data analysis (in flattened cortical surface format) was also equivalent for both primate families. To compensate for the much-smaller size of the macaque brain, we scanned macaques at higher spatial resolution (to 1 mm3).

Initial studies included fMRI tests for retinotopy, color, motion, stereopsis and face selectivity. As one might expect, we found broad similarities in the organization of visual cortex in macaques and humans --- but distinctive differences were found as well. The distinctions include: 1) a difference in the location of a supplementary motion-selective area --- V3 in macaques, instead V3A in humans; 3) retinotopic differences (e.g. V3, V4d), 3) a somewhat different organization for the processing of stereopsis --- although V3A and caudal intraparietal areas are clearly specialized for stereopsis in both humans and macaques. Comforting likenesses were also found. For instance, the fMRI revealed a face-selective area in inferotemporal cortex of macaque monkey --- quite similar to previous descriptions of a face-selective area in human inferotemporal cortex ('FFA').

Such differences and similarities should be expected from members of the same order, which have been evolutionarily separated by ~ 30 million years. It is crucial to become informed about such differences and similarities, if the macaque is to serve optimally as a ``model system'' for the human brain.

Perceptual bi-stability occurs when the same stimulus leads to two (or more) different interpretations. Upon prolonged observation, the perception of the stimulus alternates haphazardly over time. Plaids are ambiguous stimuli that can be seen as a single pattern translating rigidly ("coherency") or split into two gratings sliding over each other ("transparency"). Although they cause bi-stability, nearly all the work using plaids to study motion integration and segmentation was with brief presentations, neglecting the dynamical aspect. We studied the dynamics of perceptual alternations in plaids by measuring the cummulative time spent perceiving each interpretation, as well as the mean duration of individual epochs. We show that the dynamics approach provides more sensitive measures of the relative strength of the two interpretations, and therefore can shed new light on motion integration/segmentation. We also show that a fundamental result about the dynamics of alternations in binocular rivalry (Levelt 1967) can be generalized to the case of plaids, suggesting there may be general rules governing the dynamics of bi-stable perception. Specifically, it suggests a suppressive/inhibitory coupling between rivaling stimulus interpretations as a general principle in the brain.

Joint work with Jean-Michel Hupe