Alibaba's Qwen3.8-27B is a 27-billion-parameter open-weight model (Apache 2.0 license) with a 262K-token native context and strong coding and reasoning scores — one of the most capable models you can run locally. Fine-tuning it teaches the model your domain, your tone, or your tool-calling schema instead of just prompting it harder. Unsloth, the open-source fine-tuning library, is the practical way to do it without a GPU budget: it claims up to 2x faster training with as much as 80% less memory than standard PEFT, and it ships free Colab notebooks. This guide walks the full QLoRA workflow — install, dataset, training, export — and is honest about where "free" stops.
The honest VRAM math before you start
QLoRA fine-tuning adds a small trainable adapter on top of a frozen, 4-bit-quantized base model, so you update roughly 1% of the parameters. For a 27B model that is still a lot of memory. Community measurements for Qwen3.8-27B put INT4 LoRA fine-tuning at roughly 22 GB, and Unsloth's own guidance says QLoRA on this 27B works with 24 GB of VRAM, while standard (non-quantized) LoRA needs more than 36 GB.
What that means for "free": a free Google Colab T4 has about 15 GB of VRAM, which handles Qwen models in the 1–8B range comfortably in 4-bit — but it cannot fit the 27B model for training. You have two honest paths:
- Free path: fine-tune a smaller Qwen3.8-family model (or use a free Kaggle T4/P100 notebook) to learn the full Unsloth workflow end to end at zero cost.
- 27B path: run the same notebook on a 24 GB GPU — Colab Pro's A10/L4, or a cheap hourly rental like an RTX 4090 or L4 — where QLoRA of Qwen3.8-27B fits. Everything below is identical either way; only the runtime changes.
Either way, never attempt full fine-tuning of a 27B model on a single GPU: full training needs gradients and optimizer state on top of the weights — several times the memory of inference.
Step 1: set up the free environment
Open a new Google Colab notebook and set Runtime → Change runtime type → GPU (T4 on the free tier). Then install Unsloth's Colab build:
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps xformers trl peft accelerate bitsandbytes
The second line installs the memory-efficient attention (xformers), the TRL trainer library, PEFT for LoRA adapters, and bitsandbytes for 4-bit quantization. The exact package set comes from Unsloth's current Colab notebooks — Unsloth moves fast, so if a notebook errors after an update, reinstalling with the latest pinned versions from their docs is the first fix. Confirm your GPU is visible:
from unsloth import FastLanguageModel
import torch
print(torch.cuda.is_available(), torch.cuda.get_device_name(0))
Step 2: load Qwen3.8 in 4-bit
Unsloth publishes pre-quantized 4-bit versions of new Qwen models on Hugging Face (check huggingface.co and verify the exact repo name before loading — it will carry an unsloth/ prefix). The load call looks like this:
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/Qwen3.8-27B", # verify the exact 4-bit repo ID on HF
max_seq_length=2048,
load_in_4bit=True,
load_in_8bit=False,
full_finetuning=False,
)
load_in_4bit=True is the QLoRA choice — it cuts model memory by roughly 4x. Keep max_seq_length at 2048 for training even though Qwen3.8 supports far longer context; longer sequences multiply memory cost linearly, and 2048 covers nearly every instruction-tuning dataset. The base model stays frozen from here on.
Next, attach the LoRA adapters:
model = FastLanguageModel.get_peft_model(
model,
r=16,
lora_alpha=16,
lora_dropout=0,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"],
bias="none",
use_gradient_checkpointing="unsloth",
)
Rank 16 is the standard starting point — on an 8B model it adds roughly 40 million trainable parameters, a fraction of a percent of the total. Unsloth's gradient checkpointing trades a little compute for about 30% less VRAM.
Step 3: format your dataset
Unsloth's SFTTrainer accepts two common formats. Alpaca-style instruction/response pairs are simplest:
{"instruction": "Summarize this support ticket in one sentence.", "input": "<ticket text>", "output": "Customer can't reset their password."}
Or chat-formatted conversations in ShareGPT style, which better matches how Qwen3.8-27B actually gets used. Apply Qwen's chat template and mask the loss so the model only trains on assistant responses — that focuses optimization on how the model replies rather than on memorizing the prompt side. Good data beats big data: a few thousand clean, domain-specific pairs outperform 100K scraped generic ones.
Load with Hugging Face datasets and map your formatting function over it, then split a small holdout for sanity-checking the loss curve after training.
Step 4: train with SFTTrainer
TRL's SFTTrainer does the loop. A sensible starting config:
from trl import SFTTrainer, SFTConfig
trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=dataset,
args=SFTConfig(
per_device_train_batch_size=2,
gradient_accumulation_steps=4,
learning_rate=2e-4,
max_steps=60,
warmup_steps=5,
optim="adamw_8bit",
fp16=not torch.cuda.is_bf16_supported(),
bf16=torch.cuda.is_bf16_supported(),
logging_steps=1,
seed=3407,
),
)
trainer.train()
The 8-bit AdamW optimizer cuts optimizer-state memory by roughly 75% versus full Adam — one of the key tricks that keeps QLoRA training inside consumer VRAM. Watch the loss: it should fall steadily for the first 20–30 steps. A loss that flatlines immediately usually means the learning rate is too low or the dataset formatting is broken; a loss that explodes means the rate is too high. Sixty steps on a few thousand examples takes on the order of minutes to an hour with Unsloth on a T4 (small models) — versus multiples of that with stock PEFT.
When training finishes, test before exporting. Switch to inference mode with FastLanguageModel.for_inference() and prompt the model with a few examples from your domain. If it ignores the new behavior, you undertrained; if it repeats your dataset verbatim, you overtrained and should stop earlier.
Step 5: save and export
Two outputs, two use cases:
- LoRA adapters only:
model.save_pretrained("qwen38-lora")saves just the small adapter weights. Anyone can re-apply them on top of the base model. - GGUF for local runtimes:
model.save_pretrained_gguf("qwen38-gguf", tokenizer, quantization_method="q4_k_m")exports a quantized GGUF that drops straight into Ollama or llama.cpp. For Qwen3.8-27B, Q4_K_M lands around 16.8 GB — small enough to run on a single 24 GB card with room for context, and the widely recommended balance point over lower-bit quants for quality-per-byte.
Push the GGUF to Hugging Face or your own storage, and you now own a personalized 27B-class model that runs locally with no API bills.
Why QLoRA instead of RAG here
A quick decision rule from the fine-tuning community: retrieval-augmented generation (RAG) is for external facts — give the model your docs at prompt time. Fine-tuning is for behavior — consistent tool-call JSON, a specific voice, strict output schemas, refusal patterns. If your problem is "the model drifts from my API schema under long conversations," fine-tuning teaches a behavioral pattern that prompting alone rarely fixes. If your problem is "the model doesn't know my product docs," RAG is cheaper and fresher.