Advanced Federated Learning Techniques
Chapter 1: Federated Learning Foundations Revisited
Federated Learning Principles: A Recap
Challenges in Federated Environments
Mathematical Formulation of Federated Optimization
Synchronous vs. Asynchronous Federated Learning Models
Threat Models in Federated Learning
Evaluating Federated Learning Systems
Chapter 2: Advanced Aggregation Algorithms
Limitations of Federated Averaging (FedAvg)
FedProx: Addressing Statistical Heterogeneity
SCAFFOLD: Variance Reduction in Federated Optimization
FedNova: Normalized Averaging for Heterogeneous Systems
Byzantine-Robust Aggregation Methods
Adaptive Federated Optimization Techniques
Practice: Implementing Advanced Aggregation
Chapter 3: Enhancing Privacy in Federated Learning
Differential Privacy Mechanisms for FL
Applying DP to Gradient Updates
Composition Theorems and Privacy Budget Management
Secure Multi-Party Computation (SMC) Protocols for Aggregation
Homomorphic Encryption (HE) for Secure Aggregation
Comparing DP, SMC, and HE in FL Contexts
Privacy Attacks: Inference and Reconstruction
Hands-on Practical: Implementing DP-FedAvg
Chapter 4: Addressing Heterogeneity and Personalization
Sources of Heterogeneity: Statistical and System
Techniques for Handling Non-IID Data Distributions
Clustered Federated Learning Approaches
Meta-Learning for Federated Personalization
Multi-Task Learning in Federated Settings
Model Pruning and Adaptation for Device Constraints
Practice: Simulating Non-IID Data and Mitigation
Chapter 5: Communication Efficiency and System Optimization
Communication Bottlenecks in Federated Learning
Gradient Compression Techniques
Error Accumulation and Compensation Methods
Model Update Compression Strategies
Optimizing Local Computation
Asynchronous Federated Learning Optimizations
Hands-on Practical: Implementing Gradient Quantization
Chapter 6: Federated Learning System Design and Implementation
Architecture of Federated Learning Systems
Overview of FL Frameworks
Simulation vs. Deployment
Monitoring and Debugging Federated Systems
Cross-Silo vs. Cross-Device FL Implementations
Security Considerations in System Deployment
Practice: Setting up a Basic FL Simulation
Federated Learning Principles: A Recap
Was this section helpful?
- Communication-Efficient Learning of Deep Networks from Decentralized Data, H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Agüera y Arcas, 2017 Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Vol. 54 (PMLR) - Introduces Federated Learning and the Federated Averaging (FedAvg) algorithm, establishing the fundamental approach for the field.
- Advances and Open Problems in Federated Learning, Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Martin Bober, Marc Buděšínský, Aaron Chow, Francisco Corella-Sáez, Heather Davis, K. Addison Debs, Mason R. Greenberg, Clement H. C. Guo, Ram Herlands, Paul K. Ireland, Maryann Y. Kamoun, Felix X. J. Kerschbaumer, Sanmi Koyejo, Tong Li, Sanja L. Fidler, Vladan Markovic, Dara Mirza, Alireza Narimani, Daniel Ogawa, José Augusto Pinto, Nicolas E. Ryffel, Sara Scalia, Lawrence I. Smith, Koray S. Tugan, William Wei, Galen Andrew, Sergey Arbuzov, Hugo Bagpipe, Blakeley H. B. Bauman, Zachary Charles, Geoffrey M. Hinton, Thomas P. M. Furlan, Jan Krcmar, Alon Z. G. Ravid, Andreas Terzis, Andrew C. T. Yao, 2021 Foundations and Trends® in Machine Learning, Vol. 14 DOI: 10.1561/2200000083 - A comprehensive survey of Federated Learning, discussing its principles, challenges (e.g., heterogeneity, privacy), and future research directions.
- Federated Learning: Challenges, Methods, and Future Directions, Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, Virginia Smith, 2020 IEEE Signal Processing Magazine, Vol. 37 (IEEE) DOI: 10.1109/MSP.2020.2975572 - Offers a structured overview of Federated Learning's motivations, core workflow, system and data heterogeneity challenges, and various methods.
- Federated Learning: An Overview of Concepts and Applications, Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Martin Bober, Marc Buděšínský, Aaron Chow, Francisco Corella-Sáez, Heather Davis, K. Addison Debs, Mason R. Greenberg, Clement H. C. Guo, Ram Herlands, Paul K. Ireland, Maryann Y. Kamoun, Felix X. J. Kerschbaumer, Sanmi Koyejo, Tong Li, Sanja L. Fidler, Vladan Markovic, Dara Mirza, Alireza Narimani, Daniel Ogawa, José Augusto Pinto, Nicolas E. Ryffel, Sara Scalia, Lawrence I. Smith, Koray S. Tugan, William Wei, Galen Andrew, Sergey Arbuzov, Hugo Bagpipe, Blakeley H. B. Bauman, Zachary Charles, Geoffrey M. Hinton, Thomas P. M. Furlan, Jan Krcmar, Alon Z. G. Ravid, Andreas Terzis, Andrew C. T. Yao, 2021 (Morgan & Claypool Publishers) DOI: 10.2200/S01095ED1V01Y202104MLT082 - An authoritative book by many of the field's creators, providing an in-depth, structured explanation of federated learning principles and applications.