step time — seconds
uncached
cached
prompt=16384 tok · output=64 tok
586s
7.5×
78s
7.5×
prompt=16384 tok · output=128 tok
584s
7.3×
80s
7.3×
prompt=16384 tok · output=1024 tok
616s
5.4×
114s
5.4×
prompt=8192 tok · output=4096 tok
435s
1.7×
250s
1.7×
lower is better