Work Package 1

Speech perception and production in interaction

WP 1 addresses the primary mode of language for which our brain is evolutionarily adapted, namely producing and listening to spoken language. These adaptations have shaped our auditory and motor cortices in directions that allow us to perceive and produce the speech sounds which form, respectively, the entry point to or exit point from language-related knowledge stored in memory.

Successful interactions between speakers and listeners depend on rich sets of perceptual and motor processes. Speech perception requires multidimensional auditory processing, and sensorimotor and audiovisual integration, to deliver output at multiple levels; speech production requires integration of structural plans at different time scales to produce a finely timed coordination of motor output. Because of variability in their sound patterns, different languages require processing in different dimensions, different weightings of information, and different coordination of articulators. But many computational problems faced by speakers and listeners are the same the world over (e.g. how to encode messages by sequencing a limited number of phonological units; how to decode such sequences, how to organise turn taking in conversation). Understanding perceptual and motor processing in speech thus entails specifying the balance between what is language-universal and what is language-specific. Further, at all levels these perceptual and motor processes are intimately bound together—especially so in language acquisition.

WP 1’s goal is to understand this interplay. The key innovation in WP 1 is bridging from expertise in the neurobiological foundations of perceptual and motor processes to psycholinguistic expertise in the perception and production of speech, thus advancing understanding of how speech perception and production interact by linking neural, psychological and linguistic levels of analysis. This novel multi-disciplinary approach will generate new insights precisely because speech, the primary mode of language, has special characteristics. Specialised neurobiological infrastructure adapted to these characteristics, enabling infants to perceive and produce speech without explicit instruction, can thus have arisen by evolutionary selection. Genetic diversity, however, may also have led to variability in this infrastructure, and to variability across languages (e.g. tone vs non-tone languages) in the demand imposed on auditory and motor systems. Therefore it will be necessary to distinguish processes reflecting language-universal constraints from those reflecting linguistic diversity.

People involved

WP-leaders

Anne Cutler (MPI)
James McQueen (RU)

Principal Investigators

Mirjam Ernestus (RU)
Elia Formisano (UM)
Niels Schiller (LU)
Ivan Toni (RU)
Jean Vroomen (TiU)

Post Docs

Frank Eisner (RU)