fine-tuning with reinforcement learning is iterative. your reward function, dataset, and tools all shape the training signal. small design decisions can lead to large behavioral differences. castform structures the full pipeline so you can iterate quickly and trace results back to the choices that produced them.
the workflow follows five stages:
- design environment → define tools, rewards, and system prompt
- prepare dataset → build or generate training data from your data sources (docs, corpora, etc.)
- validate and launch → test locally, then start a training run
- monitor training → watch metrics and inspect model outputs in real time
- evaluate model → compare against baselines and external models
each stage feeds into the next. evaluation results inform environment tweaks, which lead to new training runs, forming a tight iteration loop.
design the environment
an environment is the complete specification of your task. it controls what the model can do, how it’s scored, and the inputs it sees during training.
what you’ll do:
- define tools the model can use during training (e.g. external APIs, search, code execution)
- define how success is measured, with reward components tied to the outcomes that matter for your use case
outcome: a complete environment where the model can observe, act, and be scored on your task.
create your first environment →
prepare your data
build a training dataset that represents the prompts your model will see in production. you can bring your own dataset or let castform generate training examples directly from your search corpora.
what you’ll do:
- connect your data sources (connect to your vector db or upload your own data)
- pre-process and filter based on what matters for your task
- generate training examples grounded in your data
- configure the train/eval split
outcome: a high quality training and evaluation dataset ready for the training loop.
validate and launch
test your environment locally before committing GPU time. a dry run catches broken tools, malformed rewards, and import errors before they waste a training run.
what you’ll do:
- run your environment against a few examples locally
- verify tools and reward outputs behave as expected
- configure and launch your training run
outcome: a validated environment uploaded and running on remote GPUs.
launch your first training run →
monitor
castform provides full observability as training progresses, streaming reward curves, rollout data, and more in real time. you can inspect individual completions to see what the model is actually generating.
what you’ll do:
- track reward, pass@k, and max@k trajectories
- inspect rollouts to see full prompt, completion, and tool call traces
- filter by reward score to find failure patterns
- chat with the model mid-training for a qualitative read
outcome: enough signal to decide whether to keep training, adjust rewards, or stop.
monitor your first training run →
evaluate and compare
evaluate your trained model against baseline and external models to compare across key metrics side by side.
what you’ll do:
- compare train vs eval metrics to detect overfitting
- run batch evals against the base model, and other external models (gpt-5, grok, etc.)
- inspect per-rollout results to see where your model wins and loses
- test edge cases interactively in the playground
outcome: quantitative evidence of how your model performs against the rest.