LoRA: Low-Rank Adaptation of Large Language Models, Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen, 2021International Conference on Learning Representations (ICLR 2022)DOI: 10.48550/arXiv.2106.09685 - Introduces Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning technique foundational to the discussed PEFT strategies.
torch.nn.parallel.DistributedDataParallel (DDP), PyTorch Documentation Team, 2024 (PyTorch) - Official documentation for PyTorch's DistributedDataParallel, which is central to data parallelism strategies discussed.
Parameter-Efficient Fine-tuning (PEFT) with 🤗PEFT, Hugging Face, 2024 (Hugging Face) - Official documentation for the Hugging Face PEFT library, providing practical implementation details for PEFT methods.