Seminal
The magazine and atlas of EU-funded researchA new publication from the web and AI team at Scott Charnley

Atlas / Germany

Marie Skłodowska-Curie Action · Running

BrainDial Brain Dynamics of Audiovisual Dialogue

Coordinator
Max-Planck-Gesellschaft Zur Forderung Der Wissenschaften eV, Germany
Runs
1 September 2026 to 31 August 2028
Status
Running
EU funding
€217k
Total cost
—
Partners
1 organisations in 1 countries
Type
Marie Skłodowska-Curie Action
Grant agreement
101208333

What the project is doing

In the project's own words, from its summary on CORDIS.

Language comprehension naturally occurs in face-to-face dialogues. During dialogue interactions, language unfolds rapidly, yet comprehension remains possible because listeners anticipate what speakers will say next. Studies using short sentences and isolated visual cues (e.g., lip movements, hand gestures) suggest that these predictions rely on both linguistic content and the speaker’s visual behavior. However, little is known about how the brain processes multiple co-occurring audiovisual cues during natural dialogue, the primary context for language use.

Moreover, while dialogues are rich in pragmatic and prosodic audiovisual cues that support comprehension, their influence on word-level predictions remains unclear. The complexity of dialogues has traditionally made it challenging to study their neural basis, but recent methodological advances now allow for the naturalistic investigation of continuous speech processing. For the firts time, BrainDial (Brain Dynamics of Audiovisual Dialogue) will apply these advances in an ecologically valid audiovisual dialogue setting.

We will record natural dialogues and analyze their acoustic, linguistic, and visual cues. These annotated recordings will be used in a magnetoencephalography (MEG) study to reveal the fine-grained temporal dynamics and corticoanatomical details of how the brain generates linguisitc audiovisual predictions in real time. By integrating multimodal dialogue corpus annotations, computational linguisitc metrics derived from large-language models (LLMs), body-motion tracking, MEG, and neural computational modelling, BrainDial aims to provide groundbreaking insights into the neural mechanisms of dialogue comprehension.

Topics

Who is involved

OrganisationCityCountryRole
Max-Planck-Gesellschaft Zur Forderung Der Wissenschaften eVMunchenGermanyCoordinator

Source: CORDIS, European Commission. Data of 5 Nov 2026, reused under the Commission's reuse policy. View on CORDIS · Grant DOI