Seminal
The magazine and atlas of EU-funded researchA new publication from the web and AI team at Scott Charnley

Atlas / United Kingdom

Marie Skłodowska-Curie Action · Running

VITAL-Rim VITAL-Rim: Vision-Tactile-Language Integration for Robotic Interactive Manipulation

Coordinator
The University of Liverpool, United Kingdom
Runs
1 July 2026 to 30 June 2028
Status
Running
EU funding
€276k
Total cost
—
Partners
3 organisations in 2 countries
Type
Marie Skłodowska-Curie Action
Grant agreement
101276178

What the project is doing

In the project's own words, from its summary on CORDIS.

The VITAL-Rim project, short for VIsion-TActile-Language integration for Robotic interactive manipulation, aims to enhance household robots' ability to assist with daily chores by endowing three key capabilities: scene perception through vision, semantic understanding via language, and real-time force control through touch. Currently, the development of household robots lags behind that of industrial robots, primarily due to two VITAL RIMs in homes: the barrier between humans and robots, and the complicated interaction between robots and their environments.

To address these challenges, this project will propose a novel slow-fast tactile-vision-language-action (TVLA) model that facilitates high-rate, dexterous control of robots in homes. To achieve this, the project will implement a three-tier hierarchical framework consisting of the VLA Semantic Layer, Slow Policy Layer, and Fast Action Layer. The VLA Semantic Layer will interpret language instructions and outline task sequences.

The Slow Policy Layer will generate scene-adaptive action blocks, bridging the operational frequency gap between semantic actions and real contact scenarios. The Fast Action Layer will refine these actions in real-time using a proposed lightweight vision-tactile fusion model, ensuring precise force control and rapid responsiveness. The project will design a portable handheld data collector that collects multimodal data from real household chores, thereby generating a comprehensive dataset and providing a benchmark for the TVLA model training.

Ultimately, this trained model will undergo thorough evaluation in both simulations and real-world robots. Overall, the VITAL-Rim project will empower home robots to effectively understand and respond to natural language instructions, significantly improving their interactive manipulation skills in the contact-rich scenarios. This aligns with the work programme's focus on advancing robotics for practical applications in domestic environments.

Topics

Who is involved

OrganisationCityCountryRole
The University of LiverpoolLiverpoolUnited KingdomCoordinator
Centre National De La Recherche Scientifique CNRSParisFranceAssociatedpartner
Rigmanic LtdSouthampton, UKUnited KingdomAssociatedpartner

Source: CORDIS, European Commission. Data of 5 Nov 2026, reused under the Commission's reuse policy. View on CORDIS · Grant DOI