Fine-tuning a language model used to require expensive compute clusters. Today, using a technique called QLoRA, you can fine-tune a small model on a free Colab GPU. This guide walks through the entire process in about forty lines of code.

Why QLoRA makes it possible

Fully fine-tuning a seven-billion-parameter model requires roughly 140 GB of memory β€” a job for several GPUs working in parallel. LoRA brings that figure down to about 16 GB. Instead of updating the entire model, it trains a few small adapter matrices and keeps the base weights frozen. QLoRA goes one step further: it quantises the base model to four-bit precision and trains those small matrices on top of it. For the same seven-billion-parameter model, the memory footprint drops to about 6 GB β€” easily fitting on an ordinary free GPU. The trade-off is minor: training is slightly slower and final quality marginally lower, which remains perfectly acceptable for most workloads.

The path, step by step

The implementation comes down to a few parts: load the base model in four-bit, attach the LoRA adapters, prepare the dataset, configure the training parameters, and run the trainer.

from transformers import (AutoModelForCausalLM, AutoTokenizer,
                          BitsAndBytesConfig, TrainingArguments)
from peft import LoraConfig, get_peft_model
from trl import SFTTrainer
from datasets import load_dataset
import torch

model_name = "your-base-model"

# 1. Load the base model in four-bit (QLoRA)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
    model_name, quantization_config=bnb_config, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token

# 2. Attach the LoRA adapters
lora_config = LoraConfig(
    r=16, lora_alpha=32, target_modules="all-linear",
    lora_dropout=0.05, bias="none", task_type="CAUSAL_LM")
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

# 3. Load the data (point this at your own dataset)
dataset = load_dataset("your_dataset", split="train")

# 4. Training settings
args = TrainingArguments(
    num_train_epochs=1,
    per_device_train_batch_size=1,
    gradient_accumulation_steps=16,
    learning_rate=2e-4,
    lr_scheduler_type="cosine",
    warmup_ratio=0.03,
    fp16=True,
    gradient_checkpointing=True,
    logging_steps=10,
    output_dir="./out")

# 5. Train and save
trainer = SFTTrainer(model=model, args=args,
                     train_dataset=dataset, max_seq_length=2048)
trainer.train()
model.save_pretrained("./my-lora-adapter")

The key settings to know

A few hyperparameters in this configuration determine the outcome. The rank (r) sets the size of the adapter matrices; a reliable starting point is r=16. The scaling factor (lora_alpha) is typically set to twice the rank, rendering 32 in this instance. Passing target_modules="all-linear" applies adapters across all linear layers, which has become standard practice for modern architectures. A learning rate of 2e-4 generally suits QLoRA fine-tuning. Finally, limit the number of epochs. A single epoch is often sufficient, and further training sharply increases the risk of overfitting the base model.

Merging and inference

After training, two deployment paths exist. You can keep the adapter separate β€” usually just a few dozen megabytes β€” and load it dynamically alongside the base model. This works well when you need several different adapters for different tasks on the same base model. Alternatively, you can merge the adapter weights directly into the base model. This yields a single, unified model that runs at native inference speed, which is typically preferred for static production deployments. To evaluate the result, set the model to evaluation mode, generate outputs against a sample validation prompt, and compare the completions against the base model.

When you hit a problem

There are three common failure modes. If you run out of memory (OOM), apply these fixes in order: enable gradient checkpointing, drop the batch size to one while increasing gradient accumulation steps, ensure four-bit quantisation is active, and finally, truncate the sequence length. If the training loss fails to converge, slightly increase the learning rate and verify the adapters are correctly attached to the target modules. If the output degenerates into gibberish, the learning rate is likely too high, or the model has overfit due to excessive epochs. Tuning these specific parameters resolves the vast majority of initial fine-tuning hurdles.