In this article
NEBULA is a federated learning platform for designing, running and observing experiments with centralized or peer-to-peer coordination. Its research value lies in connecting training with network conditions: topology, participation, resources and participant behavior.
This guide explains how to structure a reproducible experiment, what to retain and how to interpret results. Check version-specific capabilities against the documentation and code actually used.
Official resources and the connection to Fedstellar
The CyberDataLab repository identifies NEBULA as the evolution of Fedstellar and documents three main components: frontend, controller and core. It links the current installation guide. Use those instructions for the requirements and commands of your chosen version.
Related publications include Fedstellar in Expert Systems with Applications and the NEBULA demonstration at ACM SIGCOMM 2025. For access and updates, the repository links nebula-dfl.com and nebula-dfl.eu.
Architecture: separate experimentation from learning
The frontend supports scenario configuration and observation. The controller orchestrates execution. The core runs on each participant and handles the federated process. This separation matters: peer-to-peer training can coexist with a central service for launching experiments or visualizing results.
When evaluating availability, distinguish dashboard failure, orchestration failure and participant failure. They need not cause the same interruption. Define which process should continue and check it through a controlled test.
The DFL foundations guide explains why decentralizing aggregation does not eliminate every system dependency.
Before installation: define a testable question
An initial experiment should answer a small question: how does cost to a target quality change when replacing a star with a ring? Avoid changing dataset, model, topology and defense simultaneously, because attribution becomes impossible.
Document the dataset, license, preprocessing, client partition, seeds and validation policy. For time series, split before creating windows. For repeated devices or subjects, check that they do not appear in both training and test when the goal is to measure generalization.
A proposed experimental protocol
- Pin the version. Retain the commit or release, execution environment and relevant dependency versions.
- Check one client. Verify local learning and the correctness of labels and metrics.
- Establish a baseline. Run local learning and an unattacked federation under the same budget.
- Change one condition. Modify topology, heterogeneity or a failure while retaining the rest of the protocol.
- Repeat runs. Preserve all seeds and failed executions too.
- Export evidence. Keep configuration, logs, per-client curves and resource metrics under one run identifier.
This is a methodological proposal, not an API or an importable configuration. Exact installation steps belong to the official documentation.
What to measure beyond accuracy
| Dimension | Recommended record | Failure it helps detect |
|---|---|---|
| Local quality | F1 and recall by client and class | A strong mean hides a harmed client |
| Time | Round duration and time to target quality | An improvement requires many more rounds |
| Network | Sent, received and retransmitted bytes | Confusing payload with total cost |
| Resources | Peak memory and CPU/GPU usage | An average hides unsustainable peaks |
| Availability | Active participants and stale updates | Evaluating only fast nodes |
| Defense | Correct and false rejections | Penalizing legitimate heterogeneity |
Do not directly compare CPU percentages across machines without specifying hardware and measurement method. A utilization-based energy proxy must be labeled an estimate; it is not an electrical measurement.
Security and privacy require configuration-specific checks
Available security modules do not mean every control is enabled in a particular run. Record enabled mechanisms, their parameters and what each participant can observe. Keeping raw data local does not guarantee private updates.
The repository documents Opacus integration for differentially private training. Evaluating a configuration requires the protected unit, clipping, noise, sampling and privacy accounting. Adding tensor noise without these elements does not establish a guarantee. Similarly, an empty method named secure_aggregation would not implement a cryptographic protocol.
For defense studies, begin with laboratory scenarios and explicit boundaries. Compare clean and perturbed data, measure mechanism overhead and record legitimate clients that lose influence. The Byzantine aggregation guide explains those trade-offs.
Diagnosing a federation that fails to improve
If every client learns locally but federation hurts quality, check parameter compatibility, label semantics, normalization and weighting. If some clients do not advance, distinguish training failure, network delay and missing participation. If a defense rejects almost everything, evaluate its threshold on clean data before attributing the behavior to attacks.
Retain the event timeline. A quality drop after a timeout may reflect a stale update, but temporal correlation is insufficient: reproduce the case while changing only that condition.
A useful next research experiment
After reproducing the baseline, test disconnection and rejoining by a node with different data. Measure local recovery, effects on neighbors and additional bytes alongside final quality. This connects dynamic networking to learning quality without promising universal improvements.
The DFL design guide helps specify the protocol; prototype communication analysis helps examine whether a different exchanged object is worthwhile. NEBULA provides the experimental environment, while result validity depends on the question, controls and retained evidence.
