硬件选择的权衡 (成本、性能、可用性)

这部分内容有帮助吗？

参考文献

A Domain-Specific Architecture for Training Deep Neural Networks, Norman P. Jouppi, Zhifeng Chen, David Dellweg, George N. Garland, Mark P. Herlihy, Gerard N. John, Nguyet Johnson, Liam K. Kavanagh, Adam Lake, Tibor Lindholm, Matthew R. Markidis, Andrew Myatt, Kevin R. Patuto, Katherine E. Polley, Jason Rolfe, Daniel Smith, Shengqi Wang, Richard J. Ward, Mark White, Martin Wicke, Anna You, Peng Zhao, 2021 Proceedings of the 47th Annual International Symposium on Computer Architecture (ISCA '20) (ACM) DOI: 10.1145/3400302.3400309 - 介绍了谷歌张量处理单元（TPU）的架构和性能，为专用AI硬件提供了参考。
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism, Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, Bryan Catanzaro, 2019 arXiv preprint arXiv:1909.08053 DOI: 10.48550/arXiv.1909.08053 - 一篇基础性论文，展示了如何将语言模型训练扩展到数十亿参数，详细介绍了高度依赖GPU间通信和内存管理的策略。