Federated Learning Without Sharing Raw Data

Many useful machine learning models require data from more than one organization or device. That raises a practical question: how can models be trained while respecting privacy, confidentiality and data sovereignty? Federated Learning (FL) is one answer to that problem.

What is Federated Learning?

Federated Learning is a machine learning paradigm that enables model training across multiple decentralized devices or servers holding local data samples, without exchanging the data samples themselves. Instead of centralizing data in one location, FL allows models to be trained collaboratively while keeping the raw data distributed.

The Core Principle

The fundamental idea behind federated learning is simple yet powerful:

1. Local Training: Each participant trains a model on their local data
2. Model Aggregation: Only the model updates (not the raw data) are shared
3. Global Model: A central server aggregates these updates to create an improved global model
4. Distribution: The improved model is sent back to participants for the next round

This process repeats iteratively until the model converges to a satisfactory performance level.

Why Federated Learning Matters

Data locality and privacy controls

Traditional machine learning approaches require data to be centralized, which poses significant privacy risks:

• Data Breaches: Centralized data repositories are attractive targets for cyberattacks
• Regulatory Compliance: GDPR, CCPA, and other privacy regulations make data sharing complex
• User Trust: Users are increasingly concerned about how their data is used

Federated learning addresses these concerns by keeping data local while still enabling collaborative learning.

Real-World Applications

Federated learning is already making waves across various industries:

Healthcare

• Medical Imaging: Hospitals can collaborate on diagnostic models without sharing patient data
• Drug Discovery: Pharmaceutical companies can pool insights while protecting proprietary research
• Clinical Trials: Multi-site trials can share learnings while retaining local control over patient data

Financial Services

• Fraud Detection: Banks can improve fraud detection models without sharing customer transaction data
• Credit Scoring: Financial institutions can collaborate on risk assessment while protecting customer privacy

Mobile Applications

• Predictive Text: Smartphone keyboards can learn from user typing patterns without uploading personal messages
• Recommendation Systems: Devices can contribute to collaborative recommendation models while retaining preference records locally; update privacy still needs protection

Technical Deep Dive

Federated Learning Architectures

There are several FL architectures, each suited for different scenarios:

1. Horizontal Federated Learning (HFL)

Also known as sample-based federated learning, HFL is used when participants have data with the same features but different samples.

Participant A: [user1_data, user2_data, user3_data]
Participant B: [user4_data, user5_data, user6_data]
Participant C: [user7_data, user8_data, user9_data]

Use Case: Multiple hospitals with similar patient data structures but different patients.

2. Vertical Federated Learning (VFL)

Also known as feature-based federated learning, VFL is used when participants have different features for the same samples.

Participant A: [user1_features_A, user2_features_A, user3_features_A]
Participant B: [user1_features_B, user2_features_B, user3_features_B]

Use Case: A bank and an e-commerce platform collaborating on user behavior analysis.

3. Federated Transfer Learning (FTL)

FTL uses transfer learning when participants share limited overlap in both samples and features. This differs from the non-IID distributions that can also arise in horizontal FL; a distribution shift alone does not define FTL. This taxonomy follows Yang and colleagues.

The Federated Averaging Algorithm

A foundational FL algorithm is FedAvg (Federated Averaging), proposed by McMahan et al. in 2017:

# Pseudocode for FedAvg
def federated_averaging(global_model, client_models, client_weights):
    """
    Aggregate client models using weighted averaging
    
    Args:
        global_model: Current global model parameters
        client_models: List of client model parameters
        client_weights: List of weights for each client (typically data size)
    """
    if not client_models or len(client_models) != len(client_weights):
        raise ValueError("Provide one weight per client model")
    if any(weight < 0 for weight in client_weights) or sum(client_weights) <= 0:
        raise ValueError("Weights must be nonnegative with a positive sum")
    aggregated_model = {}
    
    for param_name in global_model.keys():
        weighted_sum = 0
        total_weight = sum(client_weights)
        
        for i, client_model in enumerate(client_models):
            weighted_sum += client_weights[i] * client_model[param_name]
        
        aggregated_model[param_name] = weighted_sum / total_weight
    
    return aggregated_model

Challenges and Solutions

Communication Overhead

Challenge: FL requires frequent communication between participants and the central server, which can be expensive and slow.

Solutions:

• Model Compression: Techniques like quantization and pruning reduce model size
• Selective Communication: Only send significant model updates
• Asynchronous Updates: Allow participants to update at different frequencies

System Heterogeneity

Challenge: Participants may have different computational capabilities, network conditions, and data distributions.

Solutions:

• Adaptive Aggregation: Choose weights consistent with the learning objective; prioritizing faster devices can bias representation
• Robust Aggregation: Use techniques like median-based aggregation to handle outliers
• Personalized FL: Allow participants to maintain local model variations

Privacy Attacks

Challenge: Even model updates can reveal information about the underlying data.

Solutions:

• Differential Privacy: Add noise to model updates
• Secure Aggregation: Use cryptographic techniques to aggregate updates securely
• Homomorphic Encryption: Perform computations on encrypted data

My Research in Federated Learning

As a postdoctoral researcher specializing in AI, cybersecurity and federated learning, I continue working on several decentralized learning challenges:

Decentralized Federated Learning

Traditional FL relies on a central server for coordination. My work explores decentralized federated learning (DFL) architectures where participants communicate directly with each other, eliminating the need for a central coordinator.

Benefits:

• Fault tolerance: Reduced reliance on one aggregator; connectivity and supporting services still affect availability.
• Membership: Joining and leaving require an explicit protocol and convergence checks.
• Privacy: Updates need not concentrate at one entity, but neighboring participants can still observe them.

IoT Device Security

My research in the DEFENDIS project focuses on using federated learning for IoT device identification and security:

• Device Fingerprinting: Creating unique digital signatures for IoT devices
• Anomaly Detection: Identifying compromised or malfunctioning devices
• Distributed Security: Implementing security measures without central coordination

Privacy-Preserving Techniques

I'm developing novel approaches to enhance privacy in federated learning:

• Local Differential Privacy: Adding noise at the client level
• Secure Multi-Party Computation: Using cryptographic protocols for secure aggregation
• Federated Learning with Differential Privacy: Combining FL with DP guarantees

Future Directions

The field of federated learning is rapidly evolving, with several exciting directions:

1. Federated Learning at the Edge

As edge computing becomes more prevalent, FL will play a crucial role in training models on edge devices like smartphones, IoT sensors, and autonomous vehicles.

2. Cross-Silo Federated Learning

Large organizations will increasingly collaborate using FL to build better models while maintaining data sovereignty.

3. Federated Learning for Large Language Models

Training large language models using FL could democratize access to powerful AI capabilities while requiring explicit privacy controls.

4. Federated Learning with Foundation Models

Combining FL with foundation models could enable personalized AI assistants that learn from user interactions with explicit privacy controls.

Getting Started with Federated Learning

If you're interested in exploring federated learning, here are some resources to get you started:

Start with a reproducible local experiment. The NEBULA guide links to the official platform documentation and explains which configuration details to retain.

Learning Resources

• Papers: Start with the original FedAvg paper and recent surveys
• Tutorials: Many frameworks provide excellent tutorials and examples
• Conferences: Follow FL-related sessions at major ML conferences

A numerical FedAvg example and its limits

Suppose two clients with 20 and 80 examples return scalar parameters 1.0 and 3.0. Their sample-weighted average is 2.6, while equal client weighting yields 2.0. Neither number is an experimental result: they illustrate how the objective depends on weighting.

The pseudocode above covers aggregation only and assumes compatible parameter names. It does not implement client selection, authentication or cryptographic protection. FedAvg combines models after local training, which may contain multiple steps; it is not generally equivalent to averaging one global gradient. The original reference is McMahan and colleagues, AISTATS 2017.

Does keeping data local guarantee privacy?

No. Deep Leakage from Gradients demonstrates reconstruction from gradients under specific experimental conditions. This does not mean every update reveals an entire dataset; it establishes that update exchange needs a threat model.

Distinguish transport confidentiality, hiding individual contributions and limiting inferences about people or devices. Secure aggregation and differentially private training address different concerns. Architecture selection alone does not establish regulatory compliance.

How to tell whether collaboration helps

Hold out entire devices or institutions when evaluating transfer. Compare federated and local models under the same tuning budget. Report per-client results and identify who loses quality after collaboration. If labels mean different things across organizations, a global average can hide a task-definition problem.

Horizontal versus vertical describes how samples and features are partitioned; centralized versus decentralized describes coordination. They are different axes. Continue with DFL foundations and privacy versus robustness.

Conclusion

Federated learning is one practical way to train models collaboratively without moving raw data into a shared repository. That makes it relevant when privacy, regulation or organizational boundaries matter.

As FL algorithms and frameworks mature, the main questions are becoming more concrete: how to handle heterogeneous data, how to measure privacy, how to secure updates, and how to evaluate deployed systems.

For my work, the most interesting part is where FL meets security, decentralization and trustworthy machine learning.

If you work on federated learning, privacy-preserving systems or cybersecurity applications, I am always open to discussing related research problems.

This post forms part of my research on federated learning and privacy-preserving AI. Related methods and results are available in the publications section.