Decentralized Federated Learning: Fundamentals and Applications

Funded by: Fundación Séneca (Science and Technology Agency of the Region of Murcia)

Grant: 21629/FPI/21

Decentralized Federated Learning (DFL)¹ is useful in settings where several participants need to train models together but raw data should remain local. Unlike centralized federated learning, DFL reduces the dependence on a single coordinator and moves part of the collaboration logic to the network participants.

DFL is a federated system in which communications are decentralized among the participants of a network. This can reduce the risk of a single point of failure, lower some communication costs, improve scalability, and reduce reliance on a central authority.

DFL also introduces challenges that need careful study. At its core is the question of how to train collaboratively while preserving privacy and keeping the system robust. By looking at i) DFL architectures, ii) components, iii) topologies, iv) communication protocols, and v) security methods, the main mechanisms and trade-offs become clearer.

The post also covers trust management and optimization choices, including algorithm selection and performance evaluation. It then looks at applications of DFL in sectors such as healthcare, manufacturing, mobile services, military systems and vehicles, where decentralization can be useful when raw data should not be moved to a central server.

Mathematical Foundations of DFL

Decentralized Aggregation

In DFL, the aggregation process is distributed across the network. A common decentralized formulation can be expressed as:

$$\theta_i^{(t+1)} = \sum_{j \in \mathcal{N}_i} w_{ij}\,\theta_j^{(t)} - \alpha \nabla f_i(\theta_i^{(t)})$$If the formula does not fit, focus it and use the left and right arrows, or scroll horizontally.

Where:

• $\theta_i^{(t)}$ is the model parameters of node i at iteration t
• $\mathcal{N}_i$ is the set of neighbors of node i
• $w_{ij}$ is the weight of the connection between nodes i and j
• $\alpha$ is the learning rate
• $\nabla f_i(\theta_i^{(t)})$ is the gradient of the local loss function

What makes a convergence claim meaningful

The equation above illustrates a mixing and descent step for loss minimization; the neighborhood includes the node itself and each row of weights sums to one. It is not a universal guarantee for neural networks. Convergence depends on loss smoothness, gradient noise, heterogeneity, learning rates and connectivity over time. A row-stochastic matrix preserves convex combinations, but not necessarily the uniform client average.

Koloskova and colleagues, ICML 2020 analyze changing topologies and local steps under explicit assumptions. For a concrete algorithm, check which objective converges, with which weights and under what network conditions.

DFL Architecture and Components

Network Topology

DFL networks can be organized in various topologies:

1. Ring Topology: Sequential communication pattern
2. Full Mesh Topology: All-to-all communication
3. Star Topology: Hub-and-spoke communication
4. Random Graph: Probabilistic connections

Security and Privacy in DFL

Privacy-Preserving Techniques

DFL can be combined with privacy-preserving mechanisms; these are not inherent features:

1. Differential Privacy: Adding calibrated noise to gradients
2. Secure Aggregation: Cryptographic protocols for model aggregation
3. Homomorphic Encryption: Computing on encrypted data
4. Zero-Knowledge Proofs: Verifying computations without revealing data

Cyberattack Detection with DFL

Another application scenario is in cyberattack detection. As cyberattacks grow more frequent and sophisticated, detection becomes increasingly challenging. DFL could empower a network of computers to train an ML model to identify suspicious behavior patterns with local data and decentralized coordination. Whether it improves detection or privacy depends on the protocol and evaluation; local data alone is insufficient.

Applications and design criteria

In healthcare, the challenge includes differences in populations, equipment and labeling practices. Keeping records local does not validate clinical use. A research evaluation needs cross-site transfer and subgroup quality measurements, with specific access and privacy controls.

In industry, facilities can collaborate on anomaly learning without centralizing every trace. Split training and test chronologically and distinguish authorized maintenance, faults and attacks. Local inference can continue through a disconnection even if collaborative training pauses.

For mobile devices and vehicles, participation and connectivity vary. A policy accepting only fast clients can bias the model toward better-resourced devices. Evaluate who is excluded and how a node recovers after an absence.

Count actual communication

Sending a dense tensor filled with zeros does not compress it. A sparse implementation must serialize indices and values, include metadata and measure transmitted bytes. Exchange frequency matters too: smaller messages can accompany more rounds.

Research on gossip with compressed communication illustrates why compression and consensus dynamics require joint analysis. For an experiment, first fix a quality target and compare cumulative time and bytes to reach it. If a method never reaches the target, retain that failure in the result.

Trust and participant admission

A moving-average reputation score can smooth signals, but does not establish honesty. A legitimate client with rare data can appear distant from its neighbors. Define authentication, influence limits, maximum update age and recovery, and measure false rejections without an attack.

The protocol also needs a policy when too few valid neighbors remain. Continuing locally, postponing mixing or requesting intervention are options that should remain visible. Do not silently renormalize an adversarial contribution into all available weight.

Research directions that can be tested

Three concrete questions connect the foundations to current experimentation: how admission and rejoining affect convergence; whether privacy mechanisms change a robust defense's false-rejection rate; and when exchanging prototypes actually reduces bytes to a quality target. Each question needs matched baselines and per-client measurements.

The privacy and robustness guide develops the second question, while the prototype communication note develops the third.

When should DFL replace server-based FL?

Choose according to the coordination requirement. A consortium with an accepted coordinator may obtain a simpler design with centralized FL. Autonomous peers with connectivity constraints may justify DFL, provided sufficient connectivity exists for the intended collaboration. A star topology still concentrates dependency at its center even when data remain distributed.

As a decision test, state what must survive coordinator loss: inference, exchange, training or member admission. These are different properties. Then inject that failure and check observed behavior rather than inferring it from the architecture name.

This guide draws on the 2023 DFL survey. Continue with the DFL system-design guide to turn those choices into a protocol, and NEBULA for an experimental starting point.

Conclusion

Decentralized Federated Learning is a practical direction for collaborative learning when privacy, security and decentralization matter. Its value depends on the details: topology, aggregation, communication costs, robustness and the threat model.

Key insights from this exploration:

• DFL enables collaborative learning with local data; privacy requires additional guarantees
• Traceability requires authenticated records and an explicit governance model
• Distributed observations can support cyberattack detection when their contribution is evaluated
• Healthcare and manufacturing are prime application domains
• Performance optimization is crucial for practical deployment

A useful DFL result states which dependency was reduced, which participants benefited and what costs or failure modes remain.

Footnotes

1. Martinez Beltrán, E. T., Quiles Pérez, M., Sánchez Sánchez, P., López Bernal, S., Bovet, G., Gil Pérez, M., Martínez Pérez, G., & Huertas Celdrán, A. (2023). Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges. IEEE Communications Surveys & Tutorials doi: 10.1109/COMST.2023.3315746 ↩