AToM-FM Adaptive Tokenization and Memory in Foundation Models for Efficient and Long-Horizon AI
- Coordinator
- Imperial College of Science Technology and Medicine, United Kingdom
- Runs
- 1 November 2025 to 31 October 2030
- Status
- Running
- EU funding
- €1.5m
- Total cost
- €1.5m
- Partners
- 2 organisations in 1 countries
- Type
- European Research Council grant
- Grant agreement
- 101222956
What the project is doing
In the project's own words, from its summary on CORDIS.
The recent revolution in generative AI is powered by the ever-growing scale of Foundation Models (FMs). This, however, causes a series of harmful ramifications, such as their unsustainable energy demand and environmental pollution, which accelerate climate change. Moreover, the scale of FMs jeopardises data privacy, as it compels users to deploy them on third-party servers rather than edge devices. AToM-FM sets out to reverse this trend by remedying a fundamental source of inefficiency in FMs: the granularity of the ""atomic"" units for representing information in current FMs is fixed, as it entirely depends on how they update their memory and segment input data (a process known as tokenization).
Instead, AToM-FMs will couple their granularity with the complexity of each task, allocating only as much computing effort as needed. As its key technical breakthrough, AToM-FM will make memory and tokenization adaptive, elevating FMs to unprecedented levels of efficiency. To facilitate the swift adoption of this new technology, I will repurpose state-of-the-art FMs into adaptive architectures and release them to the public.
This will not only drastically cut the carbon footprint of FM deployment and uphold data privacy, but also foster the emergence of new FM capabilities. In fact, compressing the FM representations will effectively broaden their horizon, i.e., the amount of input and output they can perceive and generate, respectively, without exhausting their memory. This will unlock 1) permanent memories, which are a prerequisite for lifelong learning
Topics
Who is involved
| Organisation | City | Country | Role |
|---|---|---|---|
| Imperial College of Science Technology and Medicine | London | United Kingdom | Coordinator |
| The University of Edinburgh | Edinburgh | United Kingdom | Participant |
Source: CORDIS, European Commission. Data of 5 Nov 2026, reused under the Commission's reuse policy. View on CORDIS · Grant DOI