Work Package 2 - Semantic and conceptual basis of language

Research lines initiated by coordinating postdoc


Postdoctoral research associate: David Neville
Work package leaders: Guillén Fernández & Robert van Rooij


Projects 1 and 2, described above, provide an experimental and modelling basis for integrating other domains of research (e.g. multi-modal learning) thus promoting generalization and derivation of fundamental principles regarding the semantic and conceptual basis of knowledge formation.

Project 1 - Geometry of conceptual spaces

Collaboration with: Guillén Fernández (WP2), Christian Döller (WP2), Peter Hagoort (WP3) , Julia Uddén (MPI and RU), Ruud Berkers (DCCN, RU), Nils Müller (DCCN, RU)

Research on concept learning very often focuses on explaining how we form concepts (i.e. the meaning of chair) on the basis of statistical regularities we perceive in the environment (i.e. chair always means an object with four legs, something you sit on, etc.). A crucial question for this research is whether linguistic similarity can lead to the formation of new abstract representations (i.e. categories) by modulating the perceived similarity of associated (non-linguistic) percepts. To investigate this hypothesis, we are conducting a study over multiple days where subjects learn to map a set of novel artificial words to a set of artificial perceptual stimuli (i.e. ‘creatures’). This study aims at investigating how brain regions that are necessary for the processing of conceptual information can support the formation of novel categories by means of a modulation of perceptual information (e.g. changes in similarity) driven by the linguistic information. Specifically, two hypotheses will be tested. First, the grouping of rule-like words with perceptual stimuli will provide support for the mnemonic structures resulting in faster learning curves for the paired associative learning and higher performance for the final associative test for the grammatically valid words compared to non-grammatical ones. Second, the neural similarity of the perceptual creatures will change as a function of the paired associative learning leading to the emergence in representational space of a ‘grammatical and a ‘non grammatical’ cluster after learning (i.e. creatures mapped to grammatical labels will tend to be more similar to each other than to creatures mapped to non-grammatical labels). This project can provide important insights on how linguistic information can influence perceptual categorization in an artificial context, thus increasing our understanding of the interactions between linguistic and mnemonic structures during knowledge formation.

In 2016 a pilot study of 12 subjects (only behavioural) was conducted and fully analysed. In line with our hypotheses, results indicated: 1) a marked advantage during paired associative learning for stimuli (i.e. creatures) that were paired with grammatically names (i.e. words) and 2) an increase in the representational similarity for the stimuli (i.e. creatures) as a function of the linguistic similarity of their associated names. These results have been summarized and presented in poster format during the LiI summer school and Donders Session. Data collection of the fMRI data has started in 2016 and will be completed by March 2017.

Project 2 - Neural dynamics of updating and accumulating conceptual knowledge

Collaboration with: Ruud Berkers (DCCN, RU), Jaap Murre (WP2), and Guillén Fernández (WP2)

This fMRI study focused on tracking the brain networks supporting the formation and accumulation of knowledge for novel abstract concepts (e.g. higher-order associative regularities). Subjects learnt to associate artificial words composed of three artificial syllables (e.g. KO-HI-ME) with an abstract figure of a given colour, shape and movement (e.g. red circle moving to the right). Results showed that neocortical regions including the medial prefrontal cortex, angular gyrus, orbitofrontal cortex, mid-temporal gyrus and posterior cingulate cortex parametrically tracked the accumulation of knowledge. Trial-by-trial updating instead parametrically modulated activity in the caudate nucleus in the ventral striatum.  Furthermore, connectivity of the caudate nucleus fluctuated with the accumulation and updating of information in the angular gyrus. Overall these results suggest that the caudate nucleus provides a learning signal that drives the accumulation of conceptual knowledge in the angular gyrus, thus providing important insights on the underlying neural mechanisms.

Project 2 has provided important results on the brain networks supporting the acquisition of novel conceptual knowledge. Of particular relevance is the finding that knowledge accumulation was related with modulation of activity in mPFC and angular gyrus regions whereas updating of knowledge was reflected in modulation of activity in the caudate nucleus region. These findings provide novel insights on how knowledge information is processed in the brain by separating the relative contributions of distinct learning signals. The manuscript is completed and will be submitted.

Project 3 - A hierarchical Dynamic Bayesian Network for tracking knowledge acquisition and updating

Collaboration with: Andre Marquand (RUMC), Stephanie Theves (WP2), Christian Doeller (WP2) and Guillen Fernandez (WP2).

The project aims at deploying techniques from the field of machine learning in the study cognitive neuroscience. To this end, the project made of use of expertise from different research domains building up on the interdisciplinary infrastructure provided by the consortium.

Previous studies have shown that a network of brain areas encompassing mPFC, hippocampus and striatum, provides different learning signals supporting the acquisition of conceptual knowledge. Parametric analysis of behaviour with state-space modelling in combination with neuroimaging methods revealed a differential involvement of these brain areas in the acquisition conceptual knowledge. In this project we have expanded on previous findings and developed a generalized approach for modelling the acquisition and updating of conceptual knowledge with a hierarchical structure. The appeal of this modelling approach is the ability to separate the unique contribution of specific brain areas in the acquisition of conceptual knowledge across multiple levels of representation.

The previously applied state-space model was extended in the framework of Dynamic Bayesian Networks (DBNs) in order to incorporate multiple levels of representation. The newly developed model provides a state-of-the-art tool for the analysis on a trial-by-trial basis of hierarchical conceptual knowledge. To the best of our knowledge this is the first application of DBN approach in combination with neuroimaging.
The developed model was applied in the project of Stephanie Theves to separate the representational components of hierarchical conceptual knowledge during a rule-based categorization task. Model based results in combination with neuroimaging revealed a differential involvement of mPFC, hippocampus and striatum with the mPFC and striatum tracking the accumulation of knowledge and mPFC and hippocampus tracking the updating of conceptual knowledge (I.e. after an error).
The results have further expanded previous findings on the brain network supporting the acquisition and updating of conceptual knowledge. Of particular relevance is the identification of distinct neuronal signals and the comparison with the findings from previous studies.

Other collaborations

  • "Modeling local and global mechanisms of word-finding failures"
    Collaboration between David Neville, Guillen Fernandez and Meredith Shafto (CU).

    The tip-of-tongue (TOT) effect is a well-know phenomenon occurring in both in the young adults and elderly populations. One leading hypothesis is the transmission deficit (TD) hypothesis which states that TOTs are due to insufficient activation in the phonological system during production, due to the weakening of the connections from semantic to phonological representations which in turn lead to less signal transmission (less activation). The TD model has been developed as a cognitive/behavioural model and in this project we will develop a mechanistic account of the transmission deficit hypothesis which also has neurobiological plausibility. This proposal can provide fundamental insights for the overarching questions of LiI by examining failures in the mechanisms underlying word production and knowledge retrieval which are crucially depedent on both mnemonic and lingusitic structures.