Communication is often the hidden cost of decentralized federated learning. A device may have enough compute to train locally but not enough bandwidth or battery to exchange a large model every round. The prototype experiments in the thesis make this trade-off concrete: sometimes the most important optimization is choosing a smaller object to communicate.
A simple cost model
Let be the number of model parameters and the number of bytes used to encode each parameter. A full-model message has an approximate payload:
If a client sends classes, modalities and -dimensional prototypes, the corresponding prototype payload can be approximated by:
where is the prototype precision and covers masks, labels, versions and integrity metadata. The approximation is intentionally simple. It is useful because it shows which design choices drive communication: representation size, number of classes, modalities and metadata.
Compression, sparsification and quantization can reduce either payload, but they do not change the architectural question. Are peers exchanging more information than they need for the collaboration objective?
What the Modalis resource table shows
In the reported AVMNIST resource configuration, the average client message for Modalis is approximately 0.11 MB. FedAvg and several full-model or communication-intensive baselines are around 4.75 to 4.85 MB per client message, with PmcmFL near 4.80 MB. In that comparison, the prototype message is more than 40 times smaller than the largest baselines.
The corresponding Modalis F1 is reported as 83.4%. The table also gives approximate resource figures of 44% CPU, 70% GPU and 31% RAM for the method under study. These values belong to one benchmark configuration. They should guide questions about the trade-off, not be presented as universal device requirements.
Bytes are not the only metric
A smaller message can still be a poor design if it carries unstable or misleading information. Communication efficiency should therefore be assessed with:
- payload per client and per round,
- number of rounds to reach a target quality,
- retransmissions and dropped messages,
- energy or resource cost of encoding and decoding,
- final quality for each modality and client type,
- sensitivity to stale or missing prototypes.
The total cost is better expressed as:
where is the number of communication rounds. A method with a tiny payload but twice as many rounds may not be cheaper end to end.
Why prototypes can be a good fit
In a multimodal setting, peers do not always need a copy of every layer. They may need a compact description of what a class looks like in a representation space, including information learned from a modality that is missing locally. Prototypes fit that semantic exchange better than a raw optimizer state.
There are caveats:
- A class with few examples can produce a noisy prototype.
- Prototypes may become stale when the local model changes.
- A prototype can still leak information about its source distribution.
- Metadata for masks and versions adds overhead that should be counted.
Communication savings should therefore be paired with support thresholds, versioning, integrity checks and leakage evaluation.
A practical reporting pattern
When comparing DFL protocols, report both the payload and the learning curve. A useful table includes message size, round count, cumulative bytes, F1 or loss, resource use and the configuration of clients and graph. This makes it possible to distinguish a genuinely efficient representation from a method that simply communicates less while learning less.
The thesis results suggest a productive direction: exchange compact, task-relevant representations when full models are unnecessarily expensive, then study how topology, missing modalities and robustness change the savings. Efficiency is not a cosmetic optimization. It can determine whether decentralized learning fits the network at all.
This note is an original synthesis of the communication analysis in the doctoral thesis. The reported payloads and resource values are tied to the cited AVMNIST experiment.
