Self-attention
0 comparisons
Linear attention
token
t
+ S
t−1
→ S
t
fixed-size state
Press next to add a token.
Self-attention must re-read every stored token embedding for each new token. Linear attention never re-reads embeddings — each token only needs the running state and nothing else.
← back
next →
reset