Work Package 4 - Language in action

Research lines initiated by coordinating postdoc


Postdoctoral research associate: Irina Simanova
Work package leaders: Harold Bekkering & Asli Ozyurek

Irina Simanova’s research concerns the functional relationship between language and other cognitive systems in the brain, in particular, sensory perception. Language is intervened with sensory perception in many ways. Fast integration of linguistic input and multi–modal sensory information is crucial for social communication. On the other hand, categorical structure provided by language could shape sensory perception. Simanova's current research within the consortium addresses these different sides of the interaction between multi-modal perception and language.

Project 1 - The representation of mammals in the human brain

Collaboration with: Julia Berezutskaya (WP6), Harold Bekkering (WP4), Marcel van Gerven (DCC, RU), Peter Hagoort (WP3)

In this project we set out to disentangle the role of low-level structural properties and high-level categorical distinctions in the organization of semantic representations in the brain. We explored fMRI responses to pictures of animals from a large set (over 300 unique animals), along with a set of semantic ratings obtained for each stimulus. We found a strong interaction between the semantic factors and low-level visual properties of the images.

Progress
The results of this project were defended as a Master thesis, re-analysed and submitted for publication.

Project 2 - Decoding of concepts within and across semantic categories

Collaboration with Evgeniya Bednaya (CNS master student), Harold Bekkering (WP4), Marcel van Gerven (DCC, RU), Peter Hagoort (WP3):

In the second project, we tested the hypothesis that there is a posterior-to-anterior gradient of specificity of semantic representation in the temporal lobe, where categorical information is processed in the posterior part of the middle temporal gyrus, while more detailed representations are computed in the anterior temporal lobe. In this fMRI study participants are presented with words that are either basic or subordinate names from the categories of animals and buildings (e.g. dog/house or retriever/bulldog). We use searchlight-based multivariate pattern analysis in order to locate the brain substrate that discriminates between categories, and between individual words within a category.

Progress
The results of this project were defended as a Master thesis, re-analysed and are currently in preparation for publication.

Project 3 - Developing theoretical framework on the interaction between language and perception

Collaboration with: Floris de Lange (DCCN, RU) and Jolien Francken (DCCN, RU) and Harold Bekkering (WP4)

This project looks at the interaction between language and perception from the predictive perspective. A theoretical article published in August 2015 reviews recent literature on the role of top-down feedback processes in semantic representations in the brain. The article discusses the present evidence to the proposal that linguistic constructs, in particular, words, could serve as effective priors, facilitating perception and integration of sensory information. This theoretical framework will be used as a foundation for future experimental projects.

Progress
The results of this project have been published in the paper:
Simanova, I., Francken, J. C., de Lange, F. P., & Bekkering, H. (2016). Linguistic priors shape categorical perception. Language, Cognition and Neuroscience, 31(1), 159-165.

Project 4 - Language and the visual shape bias

Collaboration with: Samuel Noorman (UvA Master Student), David Neville (WP2), Harold Bekkering (WP4)

This project is based on the theoretical exploration (Project 3) of the interaction between language and perception. Developmental studies suggest a crucial role of the attention to the global object shape in learning object names in young infants. Here, using an adjusted label advantage paradigm, we investigate the interaction between visual shape perception and object labels in adults.

Progress
The pilot behavioural experiments were started in 2016; an EEG experiment is planned for 2017.