Hacker News

Favorites Setup
☰
Comment by armcat | original | Retrofitting language models to operate over bytes
[−]armcat · 2026-10-11 Sun 09:59 UTC · link
There has been extensive research into token-free LLMs, but for some reason we are still operating in a token domain, so there is something to that.

Byte Latent Transformers (BLT): https://arxiv.org/abs/2412.09871

Charformer: https://arxiv.org/abs/2106.12672

[−]gchamonlive · 2026-10-11 Sun 11:50 UTC · link
Aren't diffusion models token-free?
[−]armcat · 2026-10-11 Sun 12:04 UTC · link
No, they still operate on tokens, it's just the difference in how text is sampled. It's done through iterative denoising and in parallel, across multiple token positions.
[−]cubefox · 2026-10-11 Sun 11:55 UTC · link
H-Net: https://proceedings.iclr.cc/paper_files/paper/2026/hash/f1aa...