logging

use Python’s standard logging module to log from your environment. the trainer automatically captures your log records and attributes them to the rollout that triggered them, so they appear in the rollout inspector on the platform.

usage

create a module-level logger and call it from any env method. no special imports or rollout IDs required.

import logging

logger = logging.getLogger(__name__)

class MyEnv(BaseEnv):
    async def run_tool(self, rollout_id, tool_name, **tool_args):
        logger.info(f"calling {tool_name} with {tool_args}")
        result = await self._dispatch(tool_name, **tool_args)
        logger.info(f"result: {str(result)[:200]}")
        return result

    async def compute_reward(self, rollout: BaseRollout) -> dict[str, float]:
        text = self._extract_answer(rollout.messages)
        score = await self._judge(text, rollout.example_args["ground_truth"])
        logger.info(f"score: {score}, answer: {text[:200]}")
        return {"correctness": score}

all standard logging levels work: logger.debug(), logger.info(), logger.warning(), logger.error(), and logger.exception() (which includes the traceback automatically).

group rewards

compute_group_rewards runs over a whole group of rollouts at once, so by default each log line is attributed to every rollout in the group, which is useful for group-level summaries. to attribute a log to one specific rollout instead (for example inside a per-rollout loop), wrap that line in rollout_context from benchmax.envs.logging (a log-attribution helper, not to be confused with the rollout_context lifecycle hook on BaseEnv):

from benchmax.envs.logging import rollout_context

async def compute_group_rewards(self, rollouts):
    logger.info("scoring group")  # attributed to all rollouts

    results = {}
    for r in rollouts:
        with rollout_context(r.rollout_id):
            logger.info("scoring individual rollout")  # attributed to this rollout only
            results[r.rollout_id] = {"quality": self._score(r.messages)}
    return results

what to log

• reward debugging: intermediate scores, judge reasoning, which gate fired
• tool calls: what arguments were passed, what the tool returned, whether it errored
• rollout state: what resources rollout_context set up, what context the model saw
• exceptions: use logger.exception() in except blocks to capture full tracebacks

where logs appear

open a rollout from the train tab and switch to the rollout logs tab (next to messages) to see its full log stream alongside the messages and reward breakdown.

this is the primary debugging tool for understanding why a specific rollout scored the way it did.