规范化层放置位置（前置LN vs. 后置LN）

这部分内容有帮助吗？

参考文献

Attention Is All You Need, Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin, 2017 Advances in Neural Information Processing Systems, Vol. 30 (Curran Associates, Inc.) DOI: 10.5555/3295222.3295349 - 介绍了原始的Transformer架构，其中使用了Post-Layer Normalization。
Layer Normalization, Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton, 2016 arXiv preprint arXiv:1607.06450 DOI: 10.48550/arXiv.1607.06450 - 介绍了Layer Normalization的概念，它是Transformer架构中的基本组成部分。
Language Models are Unsupervised Multitask Learners, Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019 (OpenAI) - 一篇重要论文，推广了Pre-Layer Normalization在训练超深Transformer模型时保持稳定性的应用。
On Layer Normalization in the Transformer Architecture, Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, Tie-Yan Liu, 2020 ICML 2020 DOI: 10.48550/arXiv.2002.04745 - 深入分析了Pre-LN和Post-LN Transformer配置在训练动态和稳定性方面的差异。