What Is Fine-Tuning? A Beginner's Guide to Adapting LLMs

Fine-tuning means continuing the training of an already pretrained model on your own examples so that its weights change and it behaves the way you want by default. Instead of describing the behaviour in a prompt every time, you show the model hundreds or thousands of input and output pairs, and the model learns the pattern.

The idea in one paragraph

A large language model starts life as a pretrained network. It has read a huge amount of text and learned grammar, facts, code and reasoning patterns. The weights inside it are just numbers. Fine-tuning runs the same training loop that created those numbers, but on a much smaller, focused dataset. Each step compares the model's prediction with your target text, computes a loss, and nudges the weights to reduce it. After enough steps, the model produces outputs that look like your examples.

You do not start from random weights. That is why fine-tuning is cheap compared with pretraining. A model that took thousands of GPUs to pretrain can often be adapted on one GPU in a few hours, especially with parameter-efficient methods such as LoRA, which this course covers in depth.

What a training example looks like

For a chat model, one example is a short conversation. The model sees the user message and learns to produce the assistant message. Most toolkits accept data in a "messages" format:

{
  "messages": [
    {"role": "system", "content": "You classify support tickets."},
    {"role": "user", "content": "I was charged twice for my subscription."},
    {"role": "assistant", "content": "billing"}
  ]
}

A dataset is a file with many such lines, usually in JSON Lines format, one example per line. The quality of these lines matters far more than the model size or the fancy settings you choose later.

Types of fine-tuning

• Supervised fine-tuning (SFT): train on input and target output pairs. This is the most common type and the starting point for almost every project.
• Preference tuning: train on pairs of better and worse answers, for example with DPO, to shift style and judgement.
• Continued pretraining: train on raw domain text with no instructions, to absorb vocabulary and style.
• Distillation: train a small model to imitate a larger one.

There is also a split by how many weights you update. Full fine-tuning updates every parameter. Parameter-efficient fine-tuning (PEFT) freezes the base model and trains a small number of extra weights. Post 19 compares them.

A minimal code picture

With the Hugging Face stack, a supervised run is short. This sketch uses the TRL library; the model name comes from an environment variable so you can pick any small model you have access to.

import os
from datasets import load_dataset
from trl import SFTConfig, SFTTrainer

model_name = os.environ["BASE_MODEL"]  # e.g. a small instruct model from the Hub
dataset = load_dataset("json", data_files="train.jsonl", split="train")

trainer = SFTTrainer(
    model=model_name,
    train_dataset=dataset,
    args=SFTConfig(output_dir="out", num_train_epochs=2, per_device_train_batch_size=2),
)
trainer.train()
trainer.save_model("out/final")

Running it prints a loss value every few steps. If the loss falls and your held-out examples look better, the run worked. Later posts explain each part of this script.

What fine-tuning is good at

• Producing a consistent output format, such as strict JSON or a fixed report layout.
• Matching a tone, voice or house style.
• Teaching a narrow task to a small, cheap model so it matches a larger one on that task.
• Shortening prompts: behaviour that once needed a long instruction becomes built in.

What it is not

Fine-tuning is not a database. It is a poor way to add large amounts of changing facts, because the knowledge is smeared across weights and the model may still invent details. For facts that change, retrieval is the better tool (see post 2). Fine-tuning also does not remove the need for evaluation. A tuned model can get worse at things you did not test.

The workflow you will follow

1. Define the task and a metric.
2. Measure a prompting baseline.
3. Collect and clean data.
4. Train with a small run first.
5. Evaluate on held-out data.
6. Iterate on data, then on settings.
7. Deploy and monitor.

Most of the effort sits in steps 1 to 3 and 5. The training run itself is often the easy part. Keep that in mind as you go through the course: better data beats more training almost every time.