Hacker News

Favorites Setup
Retrofitting language models to operate over bytes (nature.com)
2026-10-08 Thu | 71 points by theanonymousone | original
[−]serioussecurity · 2026-10-11 Sun 04:47 UTC · link
Wow nature got rolled. Should have stayed closer to their expertise. They were already being hustled by a lot of the applied AI work they were accepting.
[−]phildenhoff · 2026-10-11 Sun 04:51 UTC · link
What’s bad about this paper?
[−]serioussecurity · 2026-10-11 Sun 05:03 UTC · link
Obvious work that is behind state of the art. Not field defining. A reasonable paper to publish at NeurIPS or ICML but very middle of the pack.

If your paper is accepted to Nature it should be among the top results in your field for the year. This is just fine.

Edit to clarify: Nature has not been, historically, a venue for pure machine learning papers. It's been a venue for field changing work in the physical sciences. They already have a Machine Intelligence subjournal.

What this paper shows me is how desperate they are for ML papers, and how poorly their staff understand the field.

[−]bobmarleybiceps · 2026-10-11 Sun 08:31 UTC · link
I think nature sort of has a reputation for sensationalism / probably overhyped (or outright wrong) stuff in the sciences now anyway... would not surprise me at all that they're desperate for ml stuff :-I
[−]tel · 2026-10-11 Sun 08:43 UTC · link
A professor two decades ago, one with dozens of Nature pubs, told me that Nature was all about having a pretty picture.
[−]mrkn1 · 2026-10-11 Sun 09:34 UTC · link
subword tokenizers were never causal in the first place, so BPE was peeking at future bytes all along! TIL
[−]armcat · 2026-10-11 Sun 09:59 UTC · link
There has been extensive research into token-free LLMs, but for some reason we are still operating in a token domain, so there is something to that.

Byte Latent Transformers (BLT): https://arxiv.org/abs/2412.09871

Charformer: https://arxiv.org/abs/2106.12672

[−]JonChesterfield · 2026-10-11 Sun 11:12 UTC · link
Weird paper. Models have had tokens for each byte for ages now. They can read and write individual bytes just fine, in addition to also having multibyte tokens.