Advanced Federated Learning Techniques
Chapter 1: Federated Learning Foundations Revisited
Federated Learning Principles: A Recap
Challenges in Federated Environments
Mathematical Formulation of Federated Optimization
Synchronous vs. Asynchronous Federated Learning Models
Threat Models in Federated Learning
Evaluating Federated Learning Systems
Chapter 2: Advanced Aggregation Algorithms
Limitations of Federated Averaging (FedAvg)
FedProx: Addressing Statistical Heterogeneity
SCAFFOLD: Variance Reduction in Federated Optimization
FedNova: Normalized Averaging for Heterogeneous Systems
Byzantine-Robust Aggregation Methods
Adaptive Federated Optimization Techniques
Practice: Implementing Advanced Aggregation
Chapter 3: Enhancing Privacy in Federated Learning
Differential Privacy Mechanisms for FL
Applying DP to Gradient Updates
Composition Theorems and Privacy Budget Management
Secure Multi-Party Computation (SMC) Protocols for Aggregation
Homomorphic Encryption (HE) for Secure Aggregation
Comparing DP, SMC, and HE in FL Contexts
Privacy Attacks: Inference and Reconstruction
Hands-on Practical: Implementing DP-FedAvg
Chapter 4: Addressing Heterogeneity and Personalization
Sources of Heterogeneity: Statistical and System
Techniques for Handling Non-IID Data Distributions
Clustered Federated Learning Approaches
Meta-Learning for Federated Personalization
Multi-Task Learning in Federated Settings
Model Pruning and Adaptation for Device Constraints
Practice: Simulating Non-IID Data and Mitigation
Chapter 5: Communication Efficiency and System Optimization
Communication Bottlenecks in Federated Learning
Gradient Compression Techniques
Error Accumulation and Compensation Methods
Model Update Compression Strategies
Optimizing Local Computation
Asynchronous Federated Learning Optimizations
Hands-on Practical: Implementing Gradient Quantization
Chapter 6: Federated Learning System Design and Implementation
Architecture of Federated Learning Systems
Overview of FL Frameworks
Simulation vs. Deployment
Monitoring and Debugging Federated Systems
Cross-Silo vs. Cross-Device FL Implementations
Security Considerations in System Deployment
Practice: Setting up a Basic FL Simulation
Limitations of Federated Averaging (FedAvg)
Was this section helpful?
- Communication-Efficient Learning of Deep Networks from Decentralized Data, H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Aguera y Arcas, Felix Hanzely, 2017 Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), Vol. 54 (Proceedings of Machine Learning Research) DOI: 10.55982/aistats.2017.96 - Introduces the Federated Averaging algorithm, which serves as the baseline whose limitations are discussed.
- Federated Optimization in Heterogeneous Networks, Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, Virginia Smith, 2020 Proceedings of Machine Learning Research, Vol. 119 - Analyzes the impact of statistical heterogeneity (Non-IID data) and client drift on FedAvg's convergence and performance.