The language is comprehension at single-cell resolution
Measurement of the Organization of Semantic Representations in the Neural Population Using Neuronal Activity and a Cross-validation Participant Dropping Procedure
We used the activity of every neuron to determine the organization of semantic representations within the neural population. The firing rate of each neuron is modeled as a linear combination of words.
Finally, to ensure that our results were not driven by any particular participant(s), we carried out a leave-one-out cross-validation participant-dropping procedure. Here we repeated several of the analyses described above but now sequentially removed individual participants (that is, participants 1–10) across 1,000 iterations. The removal of any group of participants who were disproportionally involved in the results would have a significant affect on them. A second 2 test was used to evaluate the distribution of neurons in participants.
To evaluate the degree to which semantic domains could be predicted from neuronal activity on a per-word level, we randomly sampled words from 60% of the sentences and then used the remaining 40% for validation across 1,000 iterations. Only candidate neuron that exhibited significant semantic selectivity and for which sufficient words and sentences were recorded were used for decoding purposes. For these we used the firing rates of all of the candidate neuron and predicted the semantic domain of words. The validation words were used to predict the semantic domain to which support nets were used. These SVCs were constructed to find the optimal hyperplanes that best separated the data by performing
The current word at position i is represented by P. Here, a neural network was used to estimate the P. Words that are more predictable on the basis of their preceding context would therefore have a low surprisal whereas words that are poorly predictable would have a high surprisal.
rmPa_rmactualLeft(d,,nright)-rmshuffle
Empirical analysis of the correlation between the degree of action of the neuron and a domain in relation to the classification of individual words and its associated cognitions
in which (y\in {\left{1,-1\right}}^{n}), corresponding to the classification of individual words, (x) is the neural activity, and ({{\rm{\zeta }}}{i}=\max \left(0,\,1-{y}{i}\left(w{x}_{i}-b\right)\right)). The regularization variable was set to 1. The inhomogeneously distributed words were accounted for by using a linearkernel and balanced class weight. Finally, after the SVCs were modelled on the bootstrapped training data, decoding accuracy for the models was determined by using words randomly sampled and bootstrapped from the validation data. We generated a null distribution by calculating the accuracy of the classifier after random shuffling the clusters on different permutations of the data. These models therefore together determine the most likely semantic domain from the combined activity patterns of all selective neurons. An empirical P value was then calculated as the percentage of permutations for which the decoding accuracy from the shuffled data was greater than the average score obtained using the original data. The significance was determined by the P value.
$$\mathop{min}\limits_{w,b,\zeta }\left(\frac{1}{2}{w}^{{\rm{T}}}{\rm{w}}+{\rm{C}}\mathop{\sum }\limits_{{\rm{i}}=1}^{{\rm{n}}}{\zeta }_{{\rm{i}}}\right)$$
Next we determined the SI of each neuron, which quantified the degree to which it responded to words within specific semantic domains compared to the others. SI was defined by the cells ability to differentiate words in a specific semantic domain, for example, food, compared to all others. Each neuron had its SI calculated.
The neuron’s firing rate is based on words in a domain and the other one. is the average firing rate in response to words outside the considered domain. The SI therefore reflects the magnitude of effect based on the absolute difference in activity for each neuron’s preferred semantic domain compared to others. Therefore, the output of the function is bounded by 0 and 1. There is no difference in activity across all of the semantic domains if an SI of 0 is used and a SI of 1.0 is used.
$${\rm{SI}}=\frac{\left|{{\rm{FR}}}{{\rm{domain}}}-{{\rm{FR}}}{{\rm{other}}}\right|}{\left|{{\rm{FR}}}{{\rm{domain}}}+{{\rm{FR}}}{{\rm{other}}}\right|}$$
Source: Semantic encoding during language comprehension at single-cell resolution
Probing the Semantic Domains of Elvis Presley using Purity Measures, d′ Analysis, and Word-List Control
Purity measures and d′ analysis were used to confirm the quality and separability of the semantic domains. To this end, we randomly sampled from 60% of the sentences across 100 iterations. We used the same spherical clustering procedure to group the words into clusters. The new clusters were compared to the original clusters using all possible matching arrangements and choosing an arrangement with the best word overlap. The percentage of the total number of words that were classified correctly was the main criterion for evaluating the clustering quality. This procedure is therefore a simple and transparent measure that varies between 0 (bad clustering) to 1 (perfect clustering; Fig. 1d, bottom). It is the total number of words in the new clusters that determines the accuracy of this assignment.
The number of words in the new clusters is n, while the number of clusters is k. We used a standard analysis to confirm the separability of the clusters. The d′ metric estimates the difference between vectoral cosine distances for all words assigned to a particular cluster compared to those assigned to all other clusters (Extended Data Fig. 2b).
Excerpts from a story narrative were introduced at the end of recordings to evaluate for the consistency of neuronal response. Here, instead of the eight-word-long sentences, the participants were given a brief story about the life and history of Elvis Presley (for example, “At ten years old, I could not figure out what it was that this Elvis Presley guy had that the rest of us boys did not have”; Extended Data Table 1). The naturalistic nature of this story made it a better choice than the previous sentences.
A word-list control was used to evaluate the effect that sentence context had on neuronal response. The word lists were the same as the ones used during the presentation of sentences, but they had a different order and no effect on lexico.
Homophone pairs were used to evaluate for meaning-specific changes in neural activity independently of phonetic content. All of the homophones came from sentence experiments in which homophones were available and in which the words within the homophone pairs came from different semantic domains. Homophones (for example, ‘sun’ and ‘son’; Extended Data Table 1), rather than homographs, were used as the word embeddings produce a unique vector for each unique token rather than for each token sense.
The participants were presented with eight-word-long sentences that included a broad sample of words across a wide variety of themes and contexts. The participants were asked if we could proceed with the next sentence every 10–15 sentences in order to confirm that they were paying attention.
The linguistic materials were given to the participants in audio format using a Python script utilizing the PyAudio library (version 0.2.11). Two microphones were used in the Alpha Omega rig for high-fidelity temporal alignment, as well as the sample of audio signals using 22 kHz. Audio recordings were annotated in a way that was semi-automated. The recordings were made at a 44 kHz sampling rate on the TasCAM DR-40 portable audio recorder and USB interface with a microphone. To further ensure granular time alignment for each word token with neuronal activity, the amplitude waveform of each session recording and the pre-recorded linguistic materials were cross-correlated to identify the time offset. The occurrence of each word token and its timing was verified manually, for additional confirmation. The measures allowed for the level of activity in the brain to be seen by the people doing the tasks.
For the tungsten microarray recordings, putative units were identified and sorted off-line through a Plexon workstation. To allow for consistency across recording techniques (that is, with the Neuropixels recordings), a semi-automated valley-seeking approach was used to classify the action potential activities of putative neurons and only well-isolated single units were used. The action potentials were sorted so that they could be used for isolation, unit selection and limit the inclusion of multi-unit activity. Candidate clusters of putative neurons needed to clearly separate from channel noise, display a voltage waveform consistent with that of a cortical neuron, and have 99% or more of action potentials separated by an inter-spike interval of at least 1 ms (Extended Data Fig. 1b,d). The units with clear instability were removed and any extended periods of little to no spiking activity were excluded from the analysis. In total, 18 sessions were completed for an average of 5.4 units per session. 1b, a.
The recordings were made using two main approaches. Altogether, ten participants underwent recordings using tungsten microarrays (Neuroprobe, Alpha Omega Engineering) and three underwent recordings using linear silicon microelectrode arrays (Neuropixels, IMEC). During the recording of the samples, a fibrin sealant was placed between the cortical surface and the inner table of the skull. Next, we incrementally advanced an array of up to five tungsten microelectrodes (500–1,500 kΩ; Alpha Omega Engineering) into the cortical ribbon at 10–100 µm increments to identify and isolate individual units. The microelectrodes were held in place for a few minutes to confirm signal stability after the units were identified. Here neuronal signals were recorded using a Neuro Omega system (Alpha Omega Engineering) that sampled the neuronal data at 44 kHz. Neuronal signals were amplified, band-pass-filtered (300 Hz and 6 kHz) and stored off-line. Most individuals underwent two recording sessions. After neural recordings from the cortex were completed, subcortical neuronal recordings and deep brain stimulator placement proceeded as planned.
Using the brain’s prefrontal cortex to perform deep brain stimulation targeting: A cohort study of the English language spoken by adults 18 years of age or older
Once and only after a patient was consented and scheduled for surgery, their candidacy for participation in the study was reviewed with respect to the following inclusion criteria: 18 years of age or older, right-hand dominant, capacity to provide informed consent for study participation and demonstration of English fluency. The participants were given randomly sample sentences and asked questions about them, in order to assess the capacity to participate in the study. There were questions that participants couldn’t answer that weren’t included in consideration. All participants consented to participate in the study, and were free to leave at any time. 13 people were Enrolled in the Data Table 1. No participant blinding or randomization was used.
The procedures were carried out in strict adherence to the guidelines of the Harvard Medical School. All participants included in the study were scheduled to undergo planned awake intraoperative neurophysiology and single-neuronal recordings for deep brain stimulation targeting. Consideration for surgery was made by a multidisciplinary team including neurologists, neurosurgeons and neuropsychologists18,19,55,56,57. It was the decision to carry out surgery that was made. Further, all microelectrode entry points and placements were based purely on planned clinical targeting and were made independently of any study consideration.
Recording from the brain’s cells is quicker than using an instrument and will be important for developing devices that restore speech to people who have lost it.
When participants listened to a number of long sentences containing 450 words, the scientists recorded which neurons fired at which time. The team recorded the activity of a small portion of the brain’s billions of neurons, but Williams said that only two or three distinct cells lit up for each word. The researchers then looked at the similarity between the words that activated the same neuronal activity.
The scientists were able to figure out what people were hearing by looking at the fire in their brains. Although they couldn’t recreate exact sentences, they could tell, for example, that a sentence contained an animal, an action and a food, in that order.
The categories that the brain assigns to words were similar between participants, Williams says, suggesting human brains all group meanings in the same way.
The sound of a word enters the ear through the brain area called the auditory cortex. The brain is in the prefrontal cortex, a region where higher-order brain activity takes place, and this is whereemantic meaning can be found.