“eh, can’t be more than a few hours” was what i told myself 2 weeks ago when i picked up an innocent ticket titled, “add qwen3.5 support to trainer”. really, how bad could it be, i thought. maybe a config change here, string change there. was pretty sure i could get it done by lunch.

surprise reveal, it was not.

little did i know i was in for a half-week bender through the wild wild west of pinned ml packages depending on other pinned ml packages hosted on branches of random github repos.

this is the story of how i made it out 🙂

context for the ticket

the ticket was seemingly simple - add qwen3.5 support in our rl trainer. but, there were a bunch of moving parts we had to work with:

  • slime → we use a fork of slime, with custom speed-up/algorithmic improvements. (we’re big fans of slime, folks should check it out for a clean, easy-to-read, fast rl trainer)
  • slime is built on top of megatron-lm, nvidia’s framework of training neural nets
  • slime also relies on sglang for inferencing models during rl to sample outputs
  • our slime fork also relies on megatron-bridge to support being able to do more efficient finetuning using loras

to support qwen3.5, all of these different moving pieces needed to work together well.

first wall: gated delta net

qwen 3.5 is a hybrid model, i.e., it contains both classical attention and the more memory-efficient linear attention (i.e., gated delta net) layers. the first hiccup was that slime was pinned to an older version of megatron that didn’t have support for gated delta net.

i feel extremely dirty saying this, but we ended up creating our own fork of megatron-bridge and adding a patch with the gated delta net implementation from a more recent version of megatron-lm.

i kicked off a test job and got an error message that said "GDN does not support packed sequence for now."

so, what is sequence packing?

when training, sequences are processed in batches. the problem is that gpus want to do matrix math, and matrix math requires rectangular tensors — every row has to be the same length. but in practice, sequences in a batch are all different lengths.

the standard fix is padding: take the longest sequence in the batch, and pad every shorter one with dummy tokens (usually a special [PAD] token) until they all match. neat rectangle, problem solved. except now you’re running the model over thousands of tokens that carry zero information and will be masked out anyway — pure wasted compute.

sequence packing is the smarter alternative: instead of padding, you concatenate multiple short sequences end-to-end into one long sequence. you fill up the available context window like a bin-packing problem. a separate array called cu_seqlens (cumulative sequence lengths) tracks where each original sequence starts and ends, so the attention mechanism knows not to let tokens from one sequence influence tokens from another.

so, i deactivated sequence packing in our trainer and kicked off the training run. stuff worked, rewards went up. all looked good. BUT, train step time was much higher (5-6x per step) compared to our previous training runs with our previous qwen3-4b models.

this was because things get a lot less efficient when you don’t pack sequences because of the extra padding you have to add.

thankfully, the fix for this was quick. the current gdn implementation relied on flash linear attention, which allowed us to pass cu_seqlens in to support packed sequences. cu_seqlens is structured as [1, 20, 50], each denoting the length of that sequence packed together.

second wall: packing 2.0

after figuring out the above, training worked & stuff ran. unfortunately though, the reward numbers on our toy math task weren’t going up ☹️. after a little bit of digging, i found that there was a significant divergence in the probabilities outputted by our inference engine and our trainer. i spent hours digging into the above gated delta net code and flash linear attention kernels but to no avail.

i then noticed this comment on megatron-bridge model’s forward pass code

# this function needs the position_ids and attention_mask in BSHD format,
# no matter use packed_seq or not

as claude would say, that was the smoking gun. our trainer was piping in stuff in the sequence packing format while the model supported bshd only. bshd is the un-packed format with padding we covered earlier. all of this failure was happening silently.

the specific dependency on bshd came through the get_rope_index.

► what get_rope_index does

get_rope_index figures out the position number to assign to every token in the input, which the model later uses to know where each token sits in the sequence.

the tricky part is that images and videos take up many tokens but represent a 2D or 3D thing, so instead of numbering them 1, 2, 3… in a line, it gives each visual token three coordinates (time, height, width) that reflect where it actually sits in the picture or video.

regular text tokens just get simple incrementing numbers, and the function stitches the text and visual numbering together so the whole sequence has consistent positions.

we ended up writing a wrapper around the function that first converts the input to bshd, calls get_rope_index, and then converts the output back to thd.

what broke next

”phew,” i thought, finally made it out of the woods and pushed my change.

one day later, my colleague thariq texted me “dude, my jobs using qwen 3 don’t ever converge now”. (note that he’s referring to qwen3 and not qwen3.5)

fml.

so i started digging into it further and i noticed that the output probabilities from our sglang inference engine now deviated heavily from that of the trainer. which was incredibly perplexing, because i’d done nothing to change the qwen3 path in our codebase.

after pulling my hair out for a few hours, i decided to manually revert things commit-by-commit to see where things exactly broke. i soon noticed that the commit where we bumped our transformers version to 5.3.0 was the culprit. i then dove deep into the model files in the transformers repo to see if there were any architectural differences, but to no avail.

as i kicked off a script for possibly the 500th time, i noticed this weird debug log from our sglang inference engine.

Transformers version 5.3.0 is used for model type qwen3. If you experience issues related to RoPE parameters, they may be due to incompatibilities between Transformers >=5.0.0 and some models. You can try downgrading to transformers==4.57.1 as a workaround.

so i looked into what change happened between 4.57.1 and 5.0.0 that affected rope parameters. turns out what happened was the following:

Before (v4): RoPE settings were scattered across separate config attributes.

config.rope_theta = 10000.0
config.rope_scaling = {"rope_type": "linear", "factor": 8.0}

After (v5): All RoPE settings live in one unified rope_parameters dict.

config.rope_parameters = {"rope_type": "linear", "rope_theta": 10000.0, "factor": 8.0}

so my next hypothesis was that maybe sglang was reading rope params wrongly. but when i headed over to github to read it, it seemed to be compatible. i proceeded to bang my head on the window next to me.

        if (
            hasattr(config, "rope_parameters")
            and config.rope_parameters
            and "rope_theta" in config.rope_parameters
        ):
            rope_theta = config.rope_parameters["rope_theta"]
            rope_scaling = config.rope_parameters
        else:
            rope_theta = getattr(config, "rope_theta", 1000000)
            rope_scaling = getattr(config, "rope_scaling", None)

after another hour or two of dropping an f-bomb every 2 mins, i noticed that slime was not dependent on the most recent sglang version but on an older one.

        rope_theta = getattr(config, "rope_theta", 1000000)
        rope_scaling = getattr(config, "rope_scaling", None)

as claude would say, that’s the smoking gun! because, config.rope_theta didn’t exist with transformers v5, sglang was defaulting to 1000000. this would have been fine if we were using the vanilla qwen3-4b model → but we were using qwen3-4b-instruct-2507, where rope_theta was 5000000. this was causing the train/inference mismatch.

it’s over

with that, i was finally done. (and no one has reported any more weird issues since then either)

took me 4 days and nights to get out of this mess, which, candidly, i’ll take. could’ve been a lot worse. and as much as i’d like to whine for hours about how much it sucked, i’m actually glad i went through this exercise. for one, bugs like this help you build deeper intuition into the broader system instead of relying on mr. claude and hoping for the best. beyond that, the scars from this one have lit a fire under me to find a proper long-term fix to the whole mess. more on that in another blogpost 🙂