1-Bit GC-DL Distributed learning based on 1-bit gradient coding
- Coordinator
- Kungliga Tekniska Hoegskolan, Sweden
- Runs
- 6 November 2024 to 5 November 2026
- Status
- Running
- EU funding
- €223k
- Total cost
- —
- Partners
- 1 organisations in 1 countries
- Type
- Marie Skłodowska-Curie Action
- Grant agreement
- 101146669
What the project is doing
In the project's own words, from its summary on CORDIS.
In the framework of distributed learning, to mitigate the negative impact of the stragglers on the training time, the gradient coding (GC) technique has been adopted. On the other hand, to deal with high communication burden in distributed learning, 1-bit gradient vectors can be transmitted instead of real-valued ones. However, the existing distributed learning method based on 1-bit data does not take stragglers into account.
In addition, current GC techniques are only designed for the distributed learning scheme where real-valued encoded vectors are transmitted and it is difficult to apply them under the case where 1-bit vectors are transmitted. To overcome the above drawbacks and to reduce the communication overhead and the training time simultaneously, this project aims to propose novel distributed learning methods based on GC with 1-bit data.
First, this project will propose a distributed learning method named 1-Bit GC-DL, which develops a 1-bit GC strategy to encode the locally computed gradient vectors of the allocated subsets into 1-bit data. Based on that, the aggregation rule at the central server for the received 1-bit data will be designed, which guarantees that the central server computes an approximated version of the true gradient vector in the presence of a certain number of stragglers to.
Second, to further reduce the training time of 1-Bit GC-DL, this project will propose a lazily aggregated distributed learning method based on 1-bit GC, i.e., 1-Bit LA-GC-DL, by combining 1-Bit GC-DL with the lazily aggregated strategy. In 1-Bit LA-GC-DL, only a fraction of the workers participate in local training during each iteration and this project will provide the criterion for selecting the participating workers based on Age of Information.
The proposed methods will be compared with other state-of-the-art methods in the context of distributed learning on both simulated and realistic datasets under practical scenarios.
Who is involved
| Organisation | City | Country | Role |
|---|---|---|---|
| Kungliga Tekniska Hoegskolan | Stockholm | Sweden | Coordinator |
Source: CORDIS, European Commission. Data of 5 Nov 2026, reused under the Commission's reuse policy. View on CORDIS · Grant DOI