MuSe

The multisensory linguistics lab (MuSe lab) complements the research infrastructure of Linguistics Labs at Konstanz. It studies language as a situated and embodied experience that is influenced by all of our senses.

We are surrounded by multitudes of sensory information channels in our daily lives, and the way we perceive and understand language can be influenced by all of them. One of the best-understood sources of such multisensory influences is vision. When we listen to speech, we subconsciously pay a lot of attention to our interlocutors’ facial expressions that help us understand them, which is one of the reasons why wearing facemasks during pandemic interventions was so challenging for our spoken interactions. But not only vision plays a role during language processing, all Aristotelian senses (i.e., auditory, visual, tactile, gustatory, and olfactory) shape the way we perceive and understand spoken and written messages, though the complexity of our situated and embodied experience of language is yet to be fully understood.

MuSe lab aims to study this complexity by using a range of methods including eye-tracking, pupillometry, sensorimotor synchronisation techniques, audio-visual recordings of human and avatar faces, as well as more traditional behavioural paradigms. 

Lab Director: Prof. Rathcke (website / Twitter)
Lab Co-director: Prof. Frassinelli (website)