naive packing — G = 4 rollouts
micro-batch
prompt (1,000 tok)
resp₁
prompt (1,000 tok)
resp₂
prompt (1,000 tok)
resp₃
prompt (1,000 tok)
resp₄
p2, p3, p4 are
identical copies
of p1 — wasted compute
3 × 1,000 = 3,000 extra
4,400
tokens processed
1,400
unique tokens
↺ replay