Other PEFT Methods: A Brief Survey

Parameter-Efficient Fine-Tuning (PEFT) offers various innovative approaches for efficient model tuning. Low-Rank Adaptation (LoRA) is a highly effective and popular PEFT technique, but several other methods are available. These techniques differ in their modification strategies: some, like LoRA, alter existing weight matrices, while others add new components to the model or manipulate the input. Understanding these alternatives expands your range of tools for adapting Large Language Models (LLMs) to specific constraints and tasks.

This section provides a brief survey of two significant alternative PEFT families: Adapter-based tuning and methods that learn continuous prompts, like Prefix-Tuning and Prompt-Tuning.

Adapter Tuning: Injecting New Layers

One of the earliest and most intuitive PEFT methods is Adapter Tuning. The core idea is simple: leave all the pre-trained model weights completely frozen and inject new, small, trainable neural network modules inside each transformer block. These modules are called "adapters."

An adapter typically consists of a bottleneck architecture:

1. A feed-forward down-projection layer that reduces the high-dimensional output from a transformer sub-layer (like the attention or feed-forward network) to a much smaller dimension.
2. A non-linear activation function, such as GELU or ReLU.
3. A feed-forward up-projection layer that restores the feature to its original dimension.
4. A residual connection that adds the adapter's output to the original, unmodified output from the transformer sub-layer.

During fine-tuning, only the parameters of these newly added adapter modules are updated. Since the bottleneck dimension is very small (e.g., 32 or 64) compared to the model's hidden dimension (e.g., 4096), the number of trainable parameters is a tiny fraction of the total.

An adapter module inserted after the feed-forward network in a transformer block. The original model weights (blue) are frozen, while only the adapter (orange) is trained.

The primary advantage of adapters is their modularity. You can train separate adapters for different tasks and simply "plug in" the relevant one at inference time without needing to modify the base model. This is highly efficient for serving multiple customized models. However, this approach can introduce a small amount of inference latency because it adds extra network layers that must be traversed.

Prompt and Prefix-Tuning: Modifying the Input

Instead of modifying the model's architecture, another family of methods focuses on manipulating the model's input. Prompt-Tuning and Prefix-Tuning are two leading examples that learn a "soft prompt" to steer the model's behavior.

Unlike the discrete text prompts you use with a model like ChatGPT, a soft prompt is a sequence of continuous numerical vectors. These vectors are prepended to the input embeddings and are trained via backpropagation to optimize the model's output for a specific task. The entire language model remains frozen.

• Prompt-Tuning is the simplest form. It adds a sequence of trainable vectors only to the input embedding layer. These vectors are treated like virtual tokens that provide task-specific context. It's extremely parameter-efficient, sometimes requiring only a few thousand trainable parameters even for a multi-billion parameter model.

• Prefix-Tuning is a more expressive variant. Instead of just adding a trainable prefix to the input embeddings, it inserts a unique, trainable prefix at the beginning of the key and value vectors in every attention layer of the transformer. This gives the model more direct, layer-by-layer control over its internal representations and generation process.

Comparison of input modification techniques. Prompt-Tuning adds a trainable soft prompt to the input embeddings, while Prefix-Tuning inserts trainable vectors into each attention block.

These methods are exceptionally lightweight. Since you are only training and storing the small soft prompt vectors for each task, you can customize a single frozen base model for hundreds of different tasks with minimal storage overhead. The main challenge is that their performance can sometimes lag behind more invasive methods like LoRA or full fine-tuning, as they have less influence over the model's internal computations.

Choosing the Right PEFT Method

The best PEFT method depends on your specific goals, balancing performance requirements with computational and storage constraints. LoRA often provides a strong balance of performance and efficiency, which contributes to its popularity. However, adapters offer superior modularity for multi-task deployments, and prompt-based methods are unmatched in their parameter efficiency.

Here is a summary comparing the techniques discussed:

Method	Core Idea	Trainable Parameters	Inference Latency	Primary Advantage
LoRA	Decomposes weight update matrices into low-rank factors.	Very Low	None (can be merged)	Excellent performance with high parameter efficiency.
Adapter Tuning	Injects small, trainable modules between frozen layers.	Very Low	Minor Increase	High modularity; easily swap adapters for different tasks.
Prompt-Tuning	Learns a continuous "soft prompt" prepended to the input.	Extremely Low	Negligible	Minimal storage; ideal for customizing one model for many tasks.
Prefix-Tuning	Learns continuous prefixes for keys/values in each attention block.	Extremely Low	Negligible	More expressive than Prompt-Tuning while remaining highly efficient.