Adapters are small files that cause big confusion

LoRA and QLoRA made fine-tuning cheap, but they moved the pain from training time to loading and serving time. A LoRA checkpoint is a handful of megabytes, not a multi-gigabyte model โ€” which is wonderful until you try to load it, merge it, and discover the outputs are identical to the base model, or that merging a 4-bit QLoRA adapter throws a dtype error you've never seen before. This guide is the practical map: what a PEFT adapter actually is, how to load one (or several), how to merge correctly, and the dtype pitfall that trips up almost everyone doing QLoRA.

If you'd rather have someone walk through your specific adapter on a call, our Hugging Face proxy job support team does exactly that. For the training side of the story, the PEFT fine-tuning job support page pairs with everything below.

What a PEFT adapter actually is

A LoRA adapter does not contain your model. It contains two small low-rank matrices (A and B) per targeted layer, plus a tiny adapter_config.json that records the base model name, the rank r, lora_alpha, and the target_modules they attach to. At inference the adapter computes ฮ”W = (alpha / r) ยท BยทA and adds it to the frozen base weight. That is the entire trick โ€” and it explains every downstream gotcha: the adapter is meaningless without the exact base model it was trained against.

When you save with PEFT you get something like:

adapter_model.safetensors   # the A/B matrices, in safetensors format
adapter_config.json         # base_model_name_or_path, r, alpha, target_modules
README.md                   # optional model card

Note what's not there: no model.safetensors with billions of parameters, no tokenizer necessarily. That is the point โ€” you version and ship kilobytes, not gigabytes.

Saving adapters vs saving a full model

There are two fundamentally different artifacts you can produce after training, and confusing them is the root of most "my adapter does nothing" tickets.

# 1) Save ONLY the adapter (recommended default) โ€” small, portable
trainer.model.save_pretrained("out/my-lora")   # writes adapter_model.safetensors

# 2) Merge into the base and save a standalone full model โ€” large, self-contained
merged = trainer.model.merge_and_unload()
merged.save_pretrained("out/merged-model", safe_serialization=True)

Option 1 keeps the adapter separate and requires the base at load time. Option 2 produces a normal Transformers model with the adapter baked in โ€” no PEFT needed to serve it. Which you want depends on how you serve, which we'll get to.

Loading an adapter with PeftModel.from_pretrained

To use a saved adapter you load the base model first, then wrap it. The base you load must match the base_model_name_or_path in the adapter config โ€” same repo, same revision, ideally the same dtype family.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "meta-llama/Llama-3.1-8B"
base = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto"
)
tok = AutoTokenizer.from_pretrained(base_id)

model = PeftModel.from_pretrained(base, "out/my-lora")   # attaches A/B matrices
model.eval()

At this point the adapter is active but not merged โ€” the forward pass adds ฮ”W on the fly. This is perfect for inference when you want to keep the base shared across many adapters. The interview-style reasoning behind "why load base then wrap" comes up often; see our Hugging Face proxy interview support if you're prepping for that.

Loading and switching multiple adapters

One base can host several adapters โ€” a support-tone adapter, a summarization adapter, a JSON-mode adapter โ€” and you switch between them without reloading gigabytes. Give each a name when you load it:

model = PeftModel.from_pretrained(base, "out/support-lora", adapter_name="support")
model.load_adapter("out/summarize-lora", adapter_name="summarize")

model.set_adapter("support")     # route the next calls through the support adapter
# ... generate ...
model.set_adapter("summarize")   # hot-swap, no base reload

You can also combine adapters with weighted merges using add_weighted_adapter for ensembling, or disable them entirely with the model.disable_adapter() context manager to get raw base outputs โ€” a handy A/B test when you suspect an adapter is a no-op.

Merging with merge_and_unload()

Merging folds ฮ”W into the base weights and returns a plain model with no PEFT wrapper and no runtime add overhead. This is what you do before exporting for high-throughput serving.

from peft import PeftModel

merged = model.merge_and_unload()          # W := W + (alpha/r)ยทBยทA
merged.save_pretrained("out/merged", safe_serialization=True)
tok.save_pretrained("out/merged")

After merge_and_unload() the object is a standard AutoModelForCausalLM โ€” it serializes to model.safetensors like any other model and can be loaded without PEFT installed. There is a subtlety, though, and it is the number-one QLoRA support ticket.

The dtype pitfall: you cannot merge into a 4-bit base

Here is the trap. You trained with QLoRA, so your base is loaded in 4-bit NF4. You call merge_and_unload() and either get a dtype error, a warning about merging into a quantized module, or โ€” worse โ€” a silent, low-quality merge. The reason is fundamental: LoRA math is fp16/bf16, but the base weights are packed 4-bit integers. You cannot add a bf16 ฮ”W into a 4-bit quantized weight and expect a faithful result. bitsandbytes quantized layers are not designed to absorb a merge cleanly.

The correct procedure is to dequantize by reloading the base in fp16/bf16, re-attach the adapter, then merge:

import torch
from transformers import AutoModelForCausalLM
from peft import PeftModel

# WRONG: base is still 4-bit here
# base_4bit = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb_config)
# PeftModel.from_pretrained(base_4bit, adapter_dir).merge_and_unload()  # dtype pitfall

# RIGHT: reload base WITHOUT quantization, in bf16, then merge
base_fp16 = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,     # NOT load_in_4bit
    device_map="auto",
)
model = PeftModel.from_pretrained(base_fp16, "out/my-qlora")
merged = model.merge_and_unload()   # now A/B and W share bf16 โ€” clean merge
merged.save_pretrained("out/merged-bf16", safe_serialization=True)

The adapter itself was trained against the 4-bit base, but the LoRA A/B matrices are stored in higher precision, so merging into the bf16 base reproduces the trained behavior faithfully. The merged model is a bf16 model; if you want it quantized for serving, quantize after merging (GPTQ/AWQ), not before. Details on that post-merge quantization path are in our Hugging Face quantization job support guide.

Choosing target_modules

Which layers you adapt affects both quality and whether a merge even changes anything meaningful. The common choices:

  • target_modules="all-linear" โ€” adapts every linear layer (attention projections + MLP). Highest quality, most parameters. A good default in 2026.
  • Attention only โ€” ["q_proj", "k_proj", "v_proj", "o_proj"]. Lighter, often sufficient for style/format tasks.
  • Explicit list โ€” needed for architectures where module names differ (e.g. fused c_attn, or MoE experts). Inspect with print(model) and match names exactly.

A frequent bug: the adapter's target_modules names don't exist in the base you loaded (wrong architecture or a renamed layer), so PEFT attaches nothing and your adapter is a silent no-op. Always confirm the config's targets match the base's actual module names. The training-side selection of rank, alpha, and targets is covered in our QLoRA fine-tuning job support resource.

"My adapter isn't affecting outputs"

When a loaded adapter appears to do nothing, it is almost always one of these:

  • Wrong base model or revision. The adapter was trained on Llama-3.1-8B but you loaded Llama-3.1-8B-Instruct. The weights don't line up, so ฮ”W lands on the wrong values.
  • Adapter never attached. target_modules didn't match any layer names โ€” check for the "no trainable/attached modules" condition.
  • You saved the base, not the adapter. If you called save_pretrained on an unwrapped model, or merged before saving expecting a small file, you may be loading the wrong artifact.
  • Adapter disabled. A stray disable_adapter() context, or you forgot set_adapter() after loading multiple.
  • Chat template mismatch. The adapter learned a specific prompt format; feeding raw text bypasses the behavior it was trained to trigger.

A fast diagnostic: generate with the adapter, then inside with model.disable_adapter(): generate again on the same prompt. If the two outputs are identical, the adapter is not engaged โ€” work down the list above.

Serving: merged model vs adapter on the fly

Two valid strategies, chosen by your traffic pattern:

  • Merge, then serve the standalone model. Best for a single fine-tune in production. No PEFT at runtime, no per-request add overhead, and engines like vLLM/TGI/SGLang consume it as an ordinary Transformers v5 model. Quantize after merging if you need it.
  • Keep the base loaded and apply adapters dynamically. Best when you serve many fine-tunes of the same base โ€” vLLM's multi-LoRA support and TGI can hot-swap adapters per request, sharing one copy of the base in VRAM. Far cheaper than N merged models.

The rule of thumb: one adapter and latency-sensitive โ†’ merge; many adapters and memory-sensitive โ†’ dynamic. The serving mechanics tie into our Transformers inference job support and, for throughput tuning, GPU optimization job support.

Pushing adapters to the Hub

Because adapters are tiny, the Hub is an excellent place to version them. Using the modern hf CLI (with Xet storage) and the push_to_hub API, publishing is a one-liner. Set the base model in the card so consumers know exactly what to load underneath.

from huggingface_hub import login
login()   # or: hf auth login

model.push_to_hub("your-org/my-lora-adapter")   # pushes adapter_model.safetensors + config
tok.push_to_hub("your-org/my-lora-adapter")

# Anyone can then load it against the recorded base:
from peft import PeftModel
reloaded = PeftModel.from_pretrained(base, "your-org/my-lora-adapter")

Everything ships in safetensors format, and Xet-backed uploads deduplicate blocks so re-pushing a lightly-changed adapter transfers only the delta. Hub workflows, private repos, and revisions are covered in our Hugging Face Hub job support page.

Get it right the first time

Adapters reward precision: match the base, confirm the targets attach, dequantize before merging a QLoRA adapter, and pick a serving strategy that fits your traffic. Miss one of those and you'll ship a model that quietly ignores everything you trained into it. If an adapter is misbehaving โ€” merging with dtype errors, or generating like the base โ€” bring it to our Hugging Face proxy job support team and we'll get it loading, merging, and serving correctly on your stack.