Seminal
The magazine and atlas of EU-funded researchA new publication from the web and AI team at Scott Charnley

Atlas / Netherlands

Marie Skłodowska-Curie Action · Running

PMRAG Dynamic Planning and Multimodal Faithful Generation in MRAG Systems for Complex Information Access

Coordinator
Universiteit Van Amsterdam, Netherlands
Runs
1 October 2026 to 30 September 2028
Status
Running
EU funding
€217k
Total cost
—
Partners
1 organisations in 1 countries
Type
Marie Skłodowska-Curie Action
Grant agreement
101279437

What the project is doing

In the project's own words, from its summary on CORDIS.

This project develops dynamic planning methods and faithful multimodal generation for Retrieval-Augmented Generation systems (MRAG). It addresses the limitation of the current MRAGs that primarily retrieve text and often produce ungrounded or hallucinated multimodal outputs when asked to combine images, tables, code, and video. I will design a hybrid multimodal retriever that indexes diverse content types, a learned planning module that decides which modalities and retrieval depth to use for each query, and constrained generation techniques that enforce grounding to retrieved evidence.

The work combines scalable representation learning, cross-modal alignment, and reinforcement learning for planning under budget and fidelity constraints. Benchmarks will be created by extending existing datasets with multimodal references and factuality labels; evaluation will measure retrieval precision, generation faithfulness, and utility on downstream tasks (QA, table reasoning, code synthesis, and multimodal summarisation).

The project includes research secondments with an industrial partner to transfer methods to real-world search and digital-assistant applications. Training will focus on multimodal ML, reproducible research, and ethics. Hosted by the University of Amsterdam under the supervision of Dr Mohammad Aliannejadi, this fellowship will advance methods for reliable multimodal information access and prepare me for a research career at the intersection of IR and generative AI.

Topics

Who is involved

OrganisationCityCountryRole
Universiteit Van AmsterdamAmsterdamNetherlandsCoordinator

Source: CORDIS, European Commission. Data of 5 Nov 2026, reused under the Commission's reuse policy. View on CORDIS · Grant DOI