supervised finetuning

with sft, you can train models via next-token prediction to replicate tokens in a given demonstration exactly. sft is particularly useful in cases where you have clear expert outputs & ground truths. it’s also much cheaper to run than rl.

dataset format

an sft dataset is a list of chat message rows. the model is then trained to exactly imitate every message with role assistant in the row.

rows = [
    {
        "messages": [
            {"role": "system", "content": "Replace personal information with typed placeholders."},
            {"role": "user", "content": "Contact Jane Doe at jane@example.com."},
            {"role": "assistant", "content": "Contact [GIVENNAME] [SURNAME] at [EMAIL]."},
        ]
    },
    # ...
]

launching an sft run

launching is three calls: SftDataset.from_rows to create the dataset, upload_sft_assets uploads your dataset to castform, and TrainerClient.launch_sft_run starts the job. to score a held-out set during training, build a second SftDataset the same way and pass it as eval_dataset to upload_sft_assets (up to 2,048 rows) — this is what makes eval_interval meaningful; without an eval_dataset, setting eval_interval on the config has no effect.

from benchmax.sft import SftDataset
from castform.platform import SftTrainingConfig, TrainerClient, upload_sft_assets

dataset = SftDataset.from_rows(rows)
eval_dataset = SftDataset.from_rows(eval_rows)  # optional: scored during training

assets = upload_sft_assets(
    dataset=dataset,
    eval_dataset=eval_dataset,
    run_name="pii-masking",
)
run_id = TrainerClient().launch_sft_run(
    assets=assets,
    name="pii-masking",
    config=SftTrainingConfig(eval_interval=20),
)
# view at https://app.castform.com/train/{run_id}

SftTrainingConfig

SftTrainingConfig is a frozen dataclass exposing only the genuine per-run choices for v1 sft. everything else — model, lora topology beyond rank, batch size defaults, optimizer, checkpoint layout — is platform-owned and not configurable here. values are validated locally against the platform’s public ranges, so a bad value fails fast instead of at launch.

field	type	default	notes
num_epochs	int	1	how many passes over the training dataset. 1–100
learning_rate	float	1e-5	learning rate. must be > 0 and ≤ 0.1
max_context_tokens	int	8192	total prompt + response tokens the model trains on per row, longer rows get truncated. 256–8192, or one of 32768 / 65536 / 131072
save_interval	int	20	how often (in steps) to save a checkpoint. 1–10,000
seed	int	42	random seed for reproducible data ordering/shuffling.
lr_decay_style	str | None	constant	learning-rate schedule shape: "constant"
min_lr	float | None	None	floor the lr decays to, when lr_decay_style is "cosine". must be less than learning_rate
warmup_ratio	float | None	None	number of steps to ramp the lr up from zero before decay starts. 0–0.5
adam_beta2	float | None	0.98	adam optimizer’s second moment decay rate, same across every currently supported model. 0.9–0.999
grad_clip	float | None	0.05	max gradient norm, clipped to stabilize training, same across every currently supported model.
lora_rank	int | None	128	rank of lora adapter.
global_batch_size	int | None	4	total examples per training step.
eval_interval	int | None	= save_interval	how often (in steps) to run the eval loop.

when to use sft

• classification and structured extraction — routing, intent classification, field extraction. there’s a single correct answer and you likely have thousands of labeled examples. finetuning a small model can match a much larger one at a fraction of the cost and latency.
• distillation from a larger model — generate traces from a frontier model on your task, then train a smaller model on them. for simpler tasks you can match quality with a model that’s cheaper and fast enough to sit in a hot path.
• warm-start for rl — rl needs the model to reach a non-zero score before it can reinforce useful behavior. for harder tasks, a base model may never get there on its own. sft on expert traces or ground truths can move it into the right part of the distribution first, so rl has something to work with. see what is rl finetuning for the other half of this tradeoff.

example training run: pii masking

we trained a Qwen/Qwen3.5-4B to detect personally identifiable information (emails, ssns, names) from texts using the openpii 1m dataset. with 3,000 training examples, the finetuned model beat an untrained frontier model on a held-out set, cutting error rate from 40% to 3.4%. full code is can be found here

next steps

once your run is launched, monitor and evaluate it the same way as an rl run — see monitoring a run and evaluating your model.