PhD Project 22
Encoding and decoding the neural signatures of natural language comprehension
Recent computational advances have made it possible to reconstruct (decode) naturalistic stimuli from neural responses. We propose to transfer this approach to reconstructing auditory and linguistic features from brain activity measured while subjects listen to narratives. Work by members of the team shows (a) the feasibility to describe neural responses by means of stimulus characterization with a computational language model, and (b) that conceptual representations can be decoded from brain activity. The current project joins and extends these studies, paving the way for the development of brain-computer interfaces driven by internal speech, and leading to a fuller understanding of the brain basis of language comprehension under naturalistic conditions.
This project combines state of the art computational linguistic techniques with advanced neuroimaging methodologies. Given the multidisciplinary nature of this endeavor, the project sees the active collaboration of researchers from different communities (computational linguistics, cognitive science and computational neuroscience). The coupling of computational features and patterns of brain activity will allow optimal probing of the spatial and temporal localization of how the linguistic information is encoded and processed in the human brain. This will provide a novel evaluation of different approaches in language representation and processing from a computational point of view using naturalistic stimuli.
Analyses of an fMRI dataset using stochastic language models have been concluded. Language comprehension requires the simultaneous processing of linguistic information at the phonological, syntactic and semantic level. These analyses used probabilistic measures derived from stochastic language models in order to characterize the processing of these distinct levels of linguistic information in the brain. The results show that lexical, syntactic and phonological stochastic models distinctively predict activity in largely separated cortical networks. This corroborates the hypothesis that language processing can be decomposed in different streams of processing corresponding to different subdivisions of the critical language network.