How The Brain Processes Sentences

This Is Your Brain on Sentences

Researchers, for the first time, have decoded and predicted the brain activity patterns of word meanings within sentences, and successfully predicted what the brain patterns would be for new sentences

University of Rochester | August 15, 2016

Researchers at the University of Rochester have, for the first time, decoded and predicted the brain activity patterns of word meanings within sentences, and successfully predicted what the brain patterns would be for new sentences.
The study used functional magnetic resonance imaging (fMRI) to measure human brain activation. "Using fMRI data, we wanted to know if given a whole sentence, can we filter out what the brain's representation of a word is -- that is to say, can we break the sentence apart into its word components, then take the components and predict what they would look like in a new sentence," said Andrew Anderson, a research fellow who led the study as a member of the lab of Rajeev Raizada, assistant professor of brain and cognitive sciences at Rochester.

"These brain maps show how accurately it was possible to predict neural activation patterns for new, previously unseen sentences, in different regions of the brain. The brighter the area, the higher the accuracy. The most accurate area, which can be seen as the bright yellow strip, is a region in the left side of the brain known as the Superior Temporal Sulcus. This region achieved statistically significant sentence predictions in 11 out of the 14 people whose brains were scanned. Although that was the most accurate region, several other regions, broadly distributed across the brain, also produced significantly accurate sentence predictions." Source: https://www.sciencedaily.com/releases/2016/08/160815094623.htm

"Brain activation patterns for different sensory and emotional aspectsof the word 'play.' The numbers to the left of each brain pattern show how strongly the word is associate with each feature. For example, 'play' is positively associated with 'Biomotion', because playing often involves people moving their bodies. But it is negatively associated with 'Unpleasant', because play is rarely an unpleasant activity.'" Source: http://medicalxpress.com/news/2016-08-brain-sentences.html

<more at https://www.sciencedaily.com/releases/2016/08/160815094623.htm; related articles and links: http://www.terradaily.com/reports/Scientists_decode_sentence_signatures_among_brain_activity_patterns_999.html (Scientists decode sentence signatures among brain activity patterns. August 15, 2016) and http://cercor.oxfordjournals.org/content/early/2016/08/12/cercor.bhw240 (Predicting Neural Activity Patterns Associated with Sentences Using a Neurobiologically Motivated Model of Semantic Representation. Andrew James Anderson, Jeffrey R. Binder, Leonardo Fernandino, Colin J. Humphries, Lisa L. Conant, Mario Aguilar, Xixi Wang, Donias Doko and Rajeev D. S. Raizada. Cerebral Cortex (2016). doi: 10.1093/cercor/bhw240. First published online: August 12, 2016. [Abstract: We introduce an approach that predicts neural representations of word meanings contained in sentences then superposes these to predict neural representations of new sentences. A neurobiological semantic model based on sensory, motor, social, emotional, and cognitive attributes was used as a foundation to define semantic content. Previous studies have predominantly predicted neural patterns for isolated words, using models that lack neurobiological interpretation. Fourteen participants read 240 sentences describing everyday situations while undergoing fMRI. To connect sentence-level fMRI activation patterns to the word-level semantic model, we devised methods to decompose the fMRI data into individual words. Activation patterns associated with each attribute in the model were then estimated using multiple-regression. This enabled synthesis of activation patterns for trained and new words, which were subsequently averaged to predict new sentences. Region-of-interest analyses revealed that prediction accuracy was highest using voxels in the left temporal and inferior parietal cortex, although a broad range of regions returned statistically significant results, showing that semantic information is widely distributed across the brain. The results show how a neurobiologically motivated semantic model can decompose sentence-level fMRI data into activation features for component words, which can be recombined to predict activation patterns for new sentences.])>