Parameter-Efficient Fine-Tuning (PEFT)
Learn how adapters make model adaptation cheaper and more practical.
The adapter idea
Parameter-efficient fine-tuning keeps the base model largely frozen and learns a smaller set of additional parameters. LoRA is a common method: it represents a weight update through low-rank matrices that are much smaller than the original layer weights. Other methods tune prompts, selected layers, or quantized models. The practical result is a compact adapter artifact that can be loaded with a compatible base model. PEFT changes the cost profile, but it still needs good data, evaluation, licensing review, and production controls.
Why memory falls
Full fine-tuning stores gradients and optimizer state for many base parameters, which can require substantial GPU memory and checkpoint storage. PEFT reduces the trainable parameter count, so the optimizer and gradients are smaller. Quantization-aware approaches can reduce the base model footprint further. These savings make experiments feasible on fewer or smaller GPUs, but activations, sequence length, batch size, and optimizer choices still consume memory. Measure peak usage rather than estimating from parameter count alone.
from peft import LoraConfig
config = LoraConfig(
r=16,
lora_alpha=32,
lora_dropout=0.05,
target_modules=["q_proj", "v_proj"],
task_type="CAUSAL_LM",
)Choose adapter settings
Adapter rank, target modules, scaling, dropout, learning rate, and sequence length influence capacity and stability. A very small adapter may underfit a complex transformation, while a large adapter can raise cost and overfit. Start with documented defaults for the model family, then change one important variable at a time. Confirm that target module names actually match the architecture; a configuration that silently attaches to nothing can produce a convincing but useless training log.
Quantized adapter training
Quantized training can keep frozen base weights in lower precision while adapter parameters train at a higher precision. This can make a large model fit on limited hardware, but compatibility depends on the model, kernel support, hardware, and training library. Watch for numerical instability, unsupported operations, and quality loss. Keep the exact quantization configuration in the experiment record. Always compare the resulting adapter against an unquantized or stronger baseline on the same evaluation set.
Adapter composition
Multiple adapters can represent different domains, languages, tasks, or customers, but combining them is not automatically additive. Their assumptions may conflict, loading order can matter, and inference memory or latency can increase. Define ownership and compatibility rules before allowing runtime composition. If adapters are merged into the base model, preserve the original artifacts and test that the merged output matches expectations. A modular adapter strategy is useful only when versioning and selection remain understandable.
Serving and packaging
A deployable PEFT model includes the exact base-model revision, tokenizer, adapter weights, configuration, runtime version, prompt template, and license information. Decide whether to load adapters per request, keep a warm pool, or merge them for simpler serving. Measure cold-start time, memory, concurrency, and quality under realistic workloads. Reject a mismatched base model rather than attempting a best-effort load. Artifact metadata should make it possible to reproduce the model a user actually received.
Evaluate the trade-offs
Compare PEFT with full fine-tuning and prompt-only approaches on target quality, generalization, safety, latency, storage, memory, and operational complexity. Test hard negatives, domain shifts, longer inputs, and adapter switching if the product supports it. Look for catastrophic forgetting, overconfident errors, and behavior that appears only under certain phrasing. PEFT is a cost-efficient training technique, not a quality guarantee. Keep a base-model fallback and a rollback path when deploying a new adapter.
Walkthrough: LoRA experiment
Start with a frozen base checkpoint and attach LoRA adapters to the attention projections used by the model architecture. Train a small pilot with a pinned seed and save the adapter configuration alongside the weights. Load the adapter with the exact base revision, run the same held-out prompts through base and adapted models, and compare both target behavior and general responses. If the target module names do not match, fail the run rather than accepting a zero-effect adapter.
Trade-offs of adapters
PEFT lowers trainable memory and makes it practical to keep several task-specific artifacts, but quality depends on adapter capacity and base-model compatibility. Loading adapters per request can save storage while increasing cold-start and concurrency costs. Merging simplifies serving but reduces modularity and can complicate rollback. Quantization saves memory but can affect numerical stability. Keep configuration and compatibility metadata inseparable from every adapter.
Practical exercise: adapter release check
Train or simulate two adapters for different tasks. Test each with the correct base model, the wrong base revision, and a missing tokenizer. Measure load time, memory, output quality, and behavior when switching adapters between requests. Create a release checklist requiring checksums, license information, evaluation results, and a clean rollback to the base model.
Sources and further reading
These primary or specialist references informed the concepts in this guide. Product details can change, so verify current documentation before implementation.