GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism, Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, Zhifeng Chen, 2019Advances in Neural Information Processing Systems, Vol. 32 (NeurIPS) - 介绍流水线并行和微批处理技术,以提高大型模型训练中的硬件利用率。