Work Package 5 - Language evolution and diversity

Research lines initiated by coordinating postdoc


Postdoctoral research associate: Hans Rutger Bosker
Work package leaders: Pieter Muysken & Simon Fisher


Project 1 - What are the psychological mechanisms responsible for the variation we find within individuals in the production of speech rate?

Collaboration with: Joe Rodd (PhD #20), Antje Meyer (WP5), Mirjam Ernestus (WP1), and Louis ten Bosch (CLS, RU)

This project aims for a better understanding of lexical access in everyday communication by providing more insight into the diversity we find in speech production and by revealing the psychological mechanisms by which this diversity is maintained. Speech can be delivered at different rates, with great diversity between languages, between speakers, within a particular talker, and even within a single utterance. Project 1 aims to expose the psychological mechanisms behind this variation in speech rate production through the development of computational models of speech planning, and a series of speech rate elicitation experiments. The current investigation is innovative in its aim to provide a better understanding of lexical access in everyday communication, where speech rate varies, testing a very broad range of speakers (i.e., also including the non-student population). It has a broader scope than most psycholinguistic models of speech production, which tend to cover exclusively the planning processes. And finally, it builds upon and will contribute to psychological and phonetic theories of speaking and uses research techniques developed in psycholinguistics, phonetics, and computational modelling.

A first speech rate elicitation experiment has been run and analysed. Outcomes indicate that speakers produce shorter segments as well as increased co-articulation between segments in fast speech (vice versa in slow speech). This suggests that talkers combine both increases in gain in the speech production system, and earlier selection of the following speech gestures, in the production of fast speech. Outcomes up for submission in March 2017. Currently, different computational models are being explored in the extent to which they can account for these experimental findings.

Project 2 - How can neural mechanisms explain variation between individuals in speech rate normalization?

Collaboration with: Antje Meyer (WP5), Anne Kösem (DCCN, RU), Ole Jensen (WP6), and Peter Hagoort (WP3), Lars Riecke (UM), Oded Ghitza (BU)

This project aims to set up a framework, supported by experimental evidence, that bridges the field of neurobiology and the study of speech perception. Given the large-scale speech rate variation in production, listeners normalize the incoming speech signal for the rate at which it is produced. This project investigates a possible neural mechanism behind this normalization process involving neural theta oscillations phase-locking to the syllabic rate of speech. Innovations flowing forth from this project include (1) neural evidence of how listeners normalize for speech rate variation; and (2) both correlational and causal links between neural oscillatory entrainment and speech perception can be tested using MEG (brain imaging) and tACS (brain stimulation).

MEG data have been collected and analysed. Outcomes indicate that neural entrainment is the mechanism underlying behavioural rate normalization findings: the greater the neural tracking of the speech, the larger the behavioural rate normalization. These findings inspired a new experiment testing for causal links between neural oscillations and speech perception involving brain stimulation (tACS). This new experiment is currently being carried out in collaboration with Lars Riecke at UM.

Simultaneously, two behavioural investigations have been carried out. One showed that imposing rhythm on unintelligible speech enhances intelligibility and, consequently, also induces rate normalization. Another study, involving rate normalization induced by (fast and slow) tone sequences, demonstrated that it is the rate, not the duration, of preceding sounds that elicits rate normalization. Moreover, destroying the rhythm of a synchronous tone sequence removes rate normalization effects. These findings support neural entrainment accounts of rate normalization.

Bosker, H. R. (2016). Accounting for rate-dependent category boundary shifts in speech perception. Attention, Perception & Psychophysics. Advance online publication. doi:10.3758/s13414-016-1206-4.

Project 3 - Investigating the link between speech rate production and speech rate perception.

Collaboration with: Merel Maslowski (PhD; MPI matching funds), and Antje Meyer (WP5)

In everyday conversations, our own speech and that of other talkers follow each other in rapid succession. Despite the ubiquity of the sound of our own voice, it is unknown whether and how the production of our own speech influences the perception of the speech of others. This project investigates how the production of speech shapes the perception of the speech of others, for instance by studying rate normalization induced by one’s own speech rate. It is innovative in (1) providing insight into the interaction between production and perception, (2) its aim to provide a better understanding of communication in its most common form, namely in dialogue settings. Moreover, it has the potential to describe and explain individual differences in speech perception induced by individual differences in speech production.

A first study investigating effects of long-term tracking of speech rate has been completed. Results show that the speech rate of one talker can influence the perception of another talker, carrying implications for our understanding of dialogue. This study has been submitted to JEP:HPP. New investigations will focus on the task-dependency of these effects.
The original study on which this PhD project was built showed that one’s own speech rate can influence the perception of another talker.

Bosker, H. R. (2017). How our own speech rate influences our perception of others. Journal of Experimental Psychology: Learning, Memory, and Cognition. doi: 10.1037/xlm0000381.

Project 4 - How does speech rate perception interact with other cognitive systems?

Collaboration with: Eva Reinisch (LMU, Munich) and Matthias Sjerps (UC, Berkeley)

The literature suggests that the speech rate in a surrounding context shapes the perception of subsequent speech at a very early stage in perception. However, this body of literature has studied these rate normalization processes exclusively in the lab. How rate normalization operates in more natural situations – where listeners perform multiple tasks at the same time, receive multimodal rate cues, and are presented with phonologically deviant speech signals – is unknown. Speech perception and rate normalization in particular, are studied in situations of increased cognitive load (i.e., performing a dual task), in audiovisual perception, and in situations of listening to a foreign-accented or L2 talker.

The study on non-native speech perception has resulted in a proceedings paper (Bosker H.R., and Reinisch E. (2015) Normalization for speechrate in native and nonnative speech. Proceedings of the 18th International Congress of Phonetic Sciences 2015 [ICPhS XVIII], Glasgow).
The study on cognitive load effects in speech perception has also resulted in
(1) a proceedings paper (Bosker H.R., Reinisch E., and Sjerps M.J. (2016) Listening under cognitive load makes speech sound fast. Proceedings of the Speech Processing in Realistic Environments [SPIRE] workshop, Groningen), and
(2) a full paper: Bosker, H.R., Reinisch, E., & Sjerps, M.J. (2017). Cognitive load makes speech sound fast, but does not modulate acoustic context effects. Journal of Memory and Language, 94, 166-176. doi:10.1016/j.jml.2016.12.002.

Project 5 - Mapping out speech rate variation in the general population.

Collaboration with: Antje Meyer (WP5) and Caitlin DeCuyper (MPI)

Speech can be produced at different rates, and speech rate variation between individuals is rife. This project tries to map out the variation in speech rate production in the general public in under to try to understand the genetic, cognitive, and motor factors underlying speech rate variation. A large-scale study (N > 300) is performed at the national Science Center Nemo in Amsterdam. Here, whole families (primarily: father, mother, two siblings) visiting the museum are invited to take part in ‘real scientific experiments’. Measures of participants’ habitual and maximal speech rates will be correlated with measures of general perceptual processing speed and their performance on picture-word interference tasks. Thus, variability in speech rate in the general (i.e., non-student) population can be assessed, as well as associations between habitual and maximal speech rates, familial correlations (parent-sib, sib-sib), and also correlations between the different experimental tasks.

Outcomes of this project seem to suggest that, in an adult population, overall processing speed predicted individual’s speech rate. Also, parents’ speech rate predicted their children’s speech rate to some degree. Whether or not this is due to cultural or biological factors remains a topic for further investigation.

Project 6 - Can rate normalization be found in non-human species?

Collaboration with: Carel ten Cate (WP5)

This project is innovative in its aim to investigate whether non-human species show similar processing skills when it comes to prosody perception. It aims to test rate normalization in songbirds (zebra finches). There have been several studies investigating the perception of local prosodic cues in human speech by songbirds, such as zebra finches, of which several have come from Ten Cate's lab. This project is unique in its attempt to test how prosodic characteristics in the preceding distal context affect the categorization of speech stimuli. If it can be shown that zebra finches show the same perceptual effects of distal speech rate as found in human perception, this would provide evidence for convergent evolution of rate normalization in species of very different lineages. The project combines methods and insights from phonetics, psycholinguistics, and behavioural biology.

Due to unforeseen complications regarding the ethical approval of this project, this project is postponed until a later point in time.

Project 7 - Investigating the role of temporal amplitude modulations in speech-in-noise production and perception.

Collaboration with: Martin Cooke (BCBL, Spain)

If neural tracking of speech really is the backbone of speech comprehension (as suggested by the neurocognitive literature), then we would expect this neural mechanism to be especially important in challenging listening conditions, such as in speech-in-noise perception. It may even predict, for instance, that talkers in noisy environments produce speech that is ‘easier to track’ for the listening brain. If it can be shown that talkers produce ‘more rhythmic’ speech in noise vs. in quiet, this suggests that the speech signal is adjusted to allow listeners to better track the signal in demanding listening contexts. The project combines methods and insights from phonetics, psycholinguistics, and neurobiology.

Analyses of speech in four different corpora, comparing speech produced in noise vs. speech produced in quiet, demonstrated that talkers produce more pronounced temporal amplitude modulations in noisy listening environments.