Hacker News
Favorites
Setup
☰
Home
Favorites
Setup
Comment by Philpax |
original
|
Retrofitting language models to operate over bytes
[−]
Philpax
· 2026-10-11 Sun 12:16 UTC ·
link
fave
The point is not to have byte tokens: it's to have
only
byte tokens, so that the usual failures of tokenisation (e.g. the number of Rs in strawberry) can be avoided.