Fine-Tuning & Training Models with Unsloth

Fine-tuning adapts a general base model to your own task and data — a support bot on your docs, a classifier, a house-style writer. Done the old way it needs a cluster; with LoRA/QLoRA and Unsloth it fits on one consumer GPU. This guide explains how, then walks through a fine-tune end to end.

Full fine-tuning vs. LoRA vs. QLoRA

Full fine-tuning updates every parameter, holding weights, gradients, and optimizer state in memory at once — north of 60 GB for an 8B model. LoRA freezes the base and trains small low-rank adapter matrices, so you update a few million parameters instead of billions. QLoRA goes further and loads the frozen base in 4-bit, which is what gets an 8B model onto a 24 GB card. ^{2 6}

Figure 1 — LoRA and especially QLoRA bring fine-tuning within reach of a single GPU.

What Unsloth adds

Unsloth is an open-source library that makes LoRA/QLoRA training roughly 2× faster with up to ~70% less VRAM. It's a drop-in wrapper around the Hugging Face stack (Transformers, TRL, PEFT) that swaps in hand-written Triton kernels for the heavy operations and a memory-efficient gradient-checkpointing path — so your training code stays the same. ³ It supports 500+ models and exports to GGUF, merged 16-bit, or Hugging Face. ¹

Figure 2 — Load a 4-bit base, attach adapters, train on your data, then export for Ollama, vLLM, or the Hub.

Project — Fine-tune a model with Unsloth (QLoRA)

Prerequisite You need one GPU with ~8 GB (3B model) or ~24 GB (8B model). No GPU? Use Unsloth's free Colab/Kaggle notebooks, or rent a pod (see the RunPod guide).

• Install Unsloth.

pip install unsloth

• Load a 4-bit base model and attach LoRA adapters.

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/llama-3.1-8b-bnb-4bit",
    max_seq_length = 2048,
    load_in_4bit = True,   # QLoRA: 4-bit frozen base
)

model = FastLanguageModel.get_peft_model(
    model, r = 16, lora_alpha = 16,   # the LoRA adapters
)

• Prepare your dataset in an instruction/response format (a JSON or CSV of examples that teach the behaviour you want).
• Train with TRL's SFTTrainer. Unsloth plugs straight into the trainer you may already know:

from trl import SFTTrainer
from transformers import TrainingArguments

trainer = SFTTrainer(
    model = model, tokenizer = tokenizer,
    train_dataset = dataset,
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 8,   # effective batch ~16
        learning_rate = 2e-4,
        num_train_epochs = 2,
        output_dir = "outputs",
    ),
)
trainer.train()

• Test it. Switch to fast inference and try a prompt:

FastLanguageModel.for_inference(model)
# ...then generate as usual with model.generate(...)

• Export for deployment. Save straight to GGUF for Ollama, or merged 16-bit for vLLM:

GGUF for Ollama / llama.cpp / LM Studio

model.save_pretrained_gguf("my-model", tokenizer, quantization_method = "q4_k_m")

...or merged 16-bit weights for vLLM

model.save_pretrained_merged("my-model-16bit", tokenizer, save_method = "merged_16bit")

Hyperparameters Four knobs matter most: LoRA rank 16–32 (set lora_alpha equal to it or double), learning rate ~2e-4, an effective batch near 16 (raise gradient_accumulation_steps, not batch size, to avoid out-of-memory), and 1–3 epochs. A healthy training loss lands around 0.5–1.0; below ~0.2 you're memorizing the data, not learning from it.

When to fine-tune

Fine-tuning is the right tool for teaching a durable behaviour, format, or style. If you mainly need the model to know new facts, retrieval (RAG) or a bigger prompt is often cheaper and easier to keep current. Many production systems combine a light fine-tune for style with retrieval for knowledge.

Key takeaways

• LoRA trains small adapters instead of all weights; QLoRA adds a 4-bit base so an 8B model fine-tunes on a 24 GB GPU. ^{2 6}
• Unsloth makes this ~2× faster with ~70% less VRAM as a drop-in wrapper over Transformers/TRL/PEFT. ³
• The flow is: load 4-bit base → attach adapters → train with SFTTrainer → export to GGUF or 16-bit. ¹
• Fine-tune for behaviour and style; use retrieval for facts — and start from a free notebook if you have no GPU.