testing

test your environment locally and in a cheap hosted sandbox before committing GPU time. validation catches issues that would otherwise waste an entire training run.

validate_environment

validate_environment runs a mini training step locally: one real group of two sibling rollouts through your environment. it checks every part of the env contract — dataset loading, tool calls, reward computation, and the full rollout loop.

it is async, so drive it with asyncio.run:

import asyncio
from pathlib import Path

from castform import validate_environment

async def main():
    env = MyEnv(api_key="...", endpoint="...")
    report = await validate_environment(
        env,
        model="gpt-5.4-mini",
        split="train",
        base_dir=Path("."),
        max_context_tokens=2048,
        local_timeout_seconds=120,
    )
    assert report.ok

asyncio.run(main())

local validation runs your environment in-process against one example and two tracked llm-proxy sessions. the sessions enforce the same total context budget used by hosted validation, while the local timeout bounds dataset setup, tools, judges, rewards, and the rest of the environment lifecycle. pass remote_assets from upload_assets to run the same group contract in the hosted sandbox as well.

run it before every launch. if any check fails, the output tells you exactly what broke and how to fix it. each outcome carries termination_reason (what stopped the rollout) and, when scoring or trial infrastructure failed after the rollout stopped, an error message — a budget stop such as context_exceeded is never masked by a later failure. one transient failure does not fail the whole run: each validation phase retries once before reporting.

the math example is the smallest environment wired for both local and hosted validation, and a good template for your own validate entrypoint.