Neurons in ventral premotor area F5 and inferior parietal areas AIP and PFG have been shown to activate when observing videos of monkey or human actions22,23,24. These neurons can help us to understand others’ actions and intentions25,36,37.
In the present study, we tested whether VLPF also contains neurons responding to action observation. We trained monkeys to watch different types of video stimuli, displaying biological movements (goal-directed actions performed by animals and non-goal-directed movements) and object motion.
Detection of motion
Activation of neurons in the monkey ventral premotor cortex (VLPF) occurs during observation of biological stimuli and object motion. We evaluated VLPF neuron activation by displaying videos (12 deg x 12 deg) showing several biological stimuli and object motion, while the monkey maintained fixation within a 6 deg x 6 deg fixation window centered on the video.
The videoclips showed a variety of visual stimuli: human actors grasping an object, and a monkey grasping food from different perspectives. Monkeys maintained fixation within the 6 deg x 6 deg fixation windows for a randomized time interval (500-900 ms). Trials were accepted as correct and rewarded. Discarded trials were repeated at the end of the sequence in order to collect at least 10 presentations for each stimulus.
Examples of highly selective VLPF neurons responding during observation of videos are shown in Fig. 2: a) BM (grasping of food from the first person perspective); b) HM (grasping of food from the third person perspective); c) HG (grasping of an object by a human actor). For BM, the discharge starts at video onset when movement has not yet started and discharges until hand-object interaction; for HM, the discharge begins before movement onset, unfolds during reaching movement and decreases during the lifting phase.
We then compared psychometric and neurometric curves of the VLPF neurons with those of a group of rhesus macaques trained to detect tactile stimuli. We also tested whether a short-time modulation of the firing rate elicited during the first pulse could explain the monkeys’ detection performance.
In addition, we examined motion direction representations in areas MT and V4. We found similar bimodality of motion direction information across the signal types: in LFP, single-unit data and multi-unit data, information about both direction and color was present in most areas apart from motion in IT (p
Our results show that motion-related information can be linked across the spatial scales of single units, multi-unit and MEG. These findings support the view that the thalamus may play an important role in perceptual and decision making processes.
Detection of facial expressions
A study conducted by researchers from Cedars-Sinai and Huntington Memorial Hospital has found that neurons in the amygdala, a part of the brain that plays an important role in facial recognition, are selectively activated when people recognize different emotions. The research team used a technique called fMRI to record the electrical activity of individual neurons while patients were shown pictures of faces with different emotions.
The fMRI images of these patients showed that certain neurons in the amygdala were more active when they were presented with images of fearful faces, while other neurons were more activated when they were presented with happy faces. This finding may help to explain why some people can be able to accurately recognize emotional expressions even when only a portion of the features on the face are visible.
Another study conducted by investigators at Ohio State University has found that specific muscles in the facial area are attuned to picking up key movements (such as those of the lips and chin) that combine to express emotion. These muscle movements (here labeled AU for action units) combine to express emotion by causing the facial expressions seen on the test subjects’ faces.
This activation is accompanied by an increased blood flow in the area of the brain responsible for face processing, the amygdala. The increased blood flow is a sign that the neural system is actively picking up signals from the surrounding environment, according to Ueli Rutishauser, assistant professor of neurosurgery at Cedars-Sinai and senior author on the study.
In addition to the activation that accompanies the onset of a facial expression, there is also activation during observation of the movement that makes the expression possible. Figures 2c and 2d show neurons that respond during the observation of the motion of a monkey grasping an object from first person perspective, with the strongest discharge occurring before the movement begins, but the response diminishes in each video epoch when the video is obscured.
A majority of the neurons responding selectively to identity and expression showed significant interactions between these two aspects of face interaction. These neurons included some that responded to face identity regardless of facial expression and some that responded to the threatening or neutral expressions of all five monkeys.
Detection of sounds
When a monkey watched a graduate student grasp and move an ice cream cone to his mouth, the brain cells that processed that movement lit up brightly. Those same neurons were also fired when the monkey watched human or other monkeys bring peanuts to their mouths.
The activated neurons detected the sounds of those events, which were presented in the form of random dot kinematograms. These kinematograms had a diameter of 6 degrees with a central annulus of 0.75 degrees and were presented in a continuous stream that ended when fixation was broken.
Detection performance was measured by calculating psychometric slopes and neurometric DA values for each individual stimulus. To ensure that the curves were symmetric and the amplitude modulation did not exceed 50% detection, only those curves with statistically significant goodness of fit were used.
A criterion was set as the mean power plus 3 SD at 20 Hz, and for all stimulus-absent trials, CRs were calculated as the proportion of trials in which the power did not reach this level. Compared to this, the DAp (difference between amplitude and threshold) was lower in the psychometric curves, probably because fewer VPL neurons had high detection performance when using periodicity to infer the presence of a stimulus.
This enables a broader range of stimulus amplitudes to be detected, which may be important for the receptive field of PV+ interneurons. In addition, this study demonstrates that PV+ interneurons are not suppressed during sound onset by network suppression of excitatory neurons, suggesting a clear role for them in processing contextual sounds (Kato et al., 2017).
During the movement of an object, SST+ interneurons increase their evoked response to sounds, which leads to a progressive increase in inhibitory modulation on excitatory neurons, thus leading to adaptation of evoked responses. This is likely to be in line with the fact that PV+ and SST+ interneurons are more active during task engagement with active listening than during passive hearing.
Moreover, the sensitivity of the monkeys to changes in stimulus amplitude was similar during passive stimulation and task performance, indicating that VPL neurons were sensitive to auditory signals when required to respond to them. This suggests that they play an essential role in classical aversive learning of auditory signals, and is consistent with their modulation by context-dependent processes in the cortical area involving the secondary motor cortex (M2).
Detection of objects
In a series of experiments, we investigated whether neurons in the monkey’s inferior temporal cortex respond to objects as well as they do to faces. First, we tested whether view-selective cells could be induced in response to wire or spheroidal stimuli.
The view-selective response of these neurons was specific to wire-like objects whose zero and 180deg views appeared as mirror-symmetrical images of each other, because of chance minimal self-occlusion. This type of response may be a special case of generalized pattern recognition. In such cases, the ‘views’ of the stimuli are encoded in a set of features representing regions of the object. In contrast, when the view-selective cells were induced in response to a reduced set of objects, as was done by Tanaka and his colleagues , they responded to a single view.
Moreover, when the neurons were manipulated to produce different shapes, such as spheroids, the responses varied from view to view. For example, one neuron showed a higher discharge rate for a Gouraud-shaded wire than for an amoeboid shape, while a second neuron was more selective for a view of a wire that had been scaled down to a more spheroidal form.
Furthermore, we investigated whether the neurons were responding to specific parts of the objects that appear in different views, as has been suggested by previous studies. For instance, for the wire-like object, we used 60 distractors to generate different views of the same object in order to test the view-selective response of the neurons.
These 60 distractors represented different geometric patterns, each with different translation and scaling, as well as different orientations. We compared the activity of the view-selective neuron to the response to each distractor object for different axes of rotation. Among the 61 view-selective neurons, only six cells (6.7 %) showed a view-selective response for rotations around one or two axes of rotation.
This is consistent with the finding that prefrontal neurons can code contextual information, even before they begin to code objects’ properties. These neurons would then be able to exploit this information in planning and guiding behaviors. They would also be able to predict the beginning and the end of an action.