Advanced Federated Learning Techniques
Chapter 1: Federated Learning Foundations Revisited
Federated Learning Principles: A Recap
Challenges in Federated Environments
Mathematical Formulation of Federated Optimization
Synchronous vs. Asynchronous Federated Learning Models
Threat Models in Federated Learning
Evaluating Federated Learning Systems
Chapter 2: Advanced Aggregation Algorithms
Limitations of Federated Averaging (FedAvg)
FedProx: Addressing Statistical Heterogeneity
SCAFFOLD: Variance Reduction in Federated Optimization
FedNova: Normalized Averaging for Heterogeneous Systems
Byzantine-Robust Aggregation Methods
Adaptive Federated Optimization Techniques
Practice: Implementing Advanced Aggregation
Chapter 3: Enhancing Privacy in Federated Learning
Differential Privacy Mechanisms for FL
Applying DP to Gradient Updates
Composition Theorems and Privacy Budget Management
Secure Multi-Party Computation (SMC) Protocols for Aggregation
Homomorphic Encryption (HE) for Secure Aggregation
Comparing DP, SMC, and HE in FL Contexts
Privacy Attacks: Inference and Reconstruction
Hands-on Practical: Implementing DP-FedAvg
Chapter 4: Addressing Heterogeneity and Personalization
Sources of Heterogeneity: Statistical and System
Techniques for Handling Non-IID Data Distributions
Clustered Federated Learning Approaches
Meta-Learning for Federated Personalization
Multi-Task Learning in Federated Settings
Model Pruning and Adaptation for Device Constraints
Practice: Simulating Non-IID Data and Mitigation
Chapter 5: Communication Efficiency and System Optimization
Communication Bottlenecks in Federated Learning
Gradient Compression Techniques
Error Accumulation and Compensation Methods
Model Update Compression Strategies
Optimizing Local Computation
Asynchronous Federated Learning Optimizations
Hands-on Practical: Implementing Gradient Quantization
Chapter 6: Federated Learning System Design and Implementation
Architecture of Federated Learning Systems
Overview of FL Frameworks
Simulation vs. Deployment
Monitoring and Debugging Federated Systems
Cross-Silo vs. Cross-Device FL Implementations
Security Considerations in System Deployment
Practice: Setting up a Basic FL Simulation
Byzantine-Robust Aggregation Methods
Was this section helpful?
- Krum: Machine Learning with Byzantine Adversaries, Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, Julien Stainer, 2017 Advances in Neural Information Processing Systems (NeurIPS) 30, Vol. 30 (Curran Associates, Inc.) - Introduces the Krum and Multi-Krum aggregation methods, providing theoretical guarantees for robustness in the presence of Byzantine clients.
- Robust Federated Learning: A Survey, Mingxuan Li, Xiaoxiao Zheng, Yan Kang, Yang Liu, Zhaohui Wu, Qiang Yang, 2021 arXiv preprint arXiv:2101.07185 - Offers a broad overview of robust federated learning, covering various Byzantine-robust aggregation methods discussed in the section, including median, trimmed mean, and Krum.