LoRA: Low-Rank Adaptation of Large Language Models, Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, 2021International Conference on Learning Representations (ICLR)DOI: 10.48550/arXiv.2106.09685 - Introduces Low-Rank Adaptation (LoRA), a parameter-efficient method for large language models, reducing trainable parameters and computational costs.
Parameter-Efficient Transfer Learning for NLP, Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, Sylvain Gelly, 2019International Conference on Machine Learning (ICML)DOI: 10.48550/arXiv.1902.00751 - Presents adapter modules, a method adding small, task-specific neural network layers to pre-trained models for efficient fine-tuning without modifying original model weights.
PEFT Library Documentation, Hugging Face, 2024 (Hugging Face) - Official documentation for the Hugging Face PEFT library, providing guides and API references for implementing various parameter-efficient fine-tuning techniques.