QSGD: Communication-Efficient SGD via Gradient Quantization, Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, Milan Vojnovic, 2017Advances in Neural Information Processing Systems, Vol. 30 (NeurIPS) - 本文介绍了量化SGD(QSGD),这是一种用于通信高效分布式训练的基础性随机量化方法,与梯度量化直接相关。
Sparsified SGD with Memory, Stich, Sebastian U., Cordonnier, Jean-Baptiste, Jaggi, Martin, 2018Advances in Neural Information Processing Systems, Vol. 31 (NeurIPS) - 这项研究分析了稀疏化SGD,包括Top-k选择,并讨论了减少信息损失的记忆机制,直接针对梯度稀疏化。