LoRA: Low-Rank Adaptation of Large Language Models, Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, 2021arXiv preprint arXiv:2106.09685DOI: 10.48550/arXiv.2106.09685 - Introduces LoRA, a method that substantially reduces trainable parameters and memory for fine-tuning LLMs by adding low-rank matrices.
Parameter-Efficient Fine-tuning (PEFT) documentation, Hugging Face, 2024 - Official documentation for the Hugging Face PEFT library, offering practical implementation specifics and use guides for various PEFT methods.
Parameter-Efficient Transfer Learning for NLP, Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, Sylvain Gelly, 2019International Conference on Machine Learning (ICML)DOI: 10.48550/arXiv.1902.00751 - Introduces adapter layers as a parameter-efficient approach for adapting pre-trained models to downstream tasks.