It’s not any of that. It’s because of overcommit being the default for basically every Linux system. With that, malloc will never fail, and it’s the later access of that memory that will. In practice, you’ll virtually never see malloc actually return a failure, and so most software, no matter the language, is generally not robust to this condition.
recently noticed that iced (rust ui framework) crashes on oom - from the logs it seems to be aware and crash explicitly.
I wish it would just reduce the fps or hang a bit instead of crashing, perhaps use exponential backoff
How is it then that my stock KUbuntu system won't let me allocate 50,000,000,000 bytes?
$ uname -a
Linux boxcar 7.0.0-31-generic #31~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Aug 10 09:38:02 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
$ cat tmp.c
#include <stdio.h>
#include <stdlib.h>
int main() {
char *s = malloc(50000000000ULL);
if (s == NULL) {
printf("Boo, hoo!\n");
} else {
printf("Look at all that memory!\n");
}
return 0;
}
$ cc tmp.c
$ ./a.out
Boo, hoo!
As to the lack of robustness of most software, that's a fact. But, for example, Daniel Stenberg of curl fame is a developer of robust software who does not appreciate how Rust handles out-of-memory errors makes it impossible to implement libcurl in the way he expects a library to work. See https://www.youtube.com/watch?v=HFH2vZRTKrA&t=2080s from 4.5 years ago as an example.
To get back to the essay, I can easily understand why someone who has Rust as their first-and-only systems language, and therefore expects abort-when-out-of-memory, will not immediately consider how C allows a different approach to how to handle that condition, but instead will try to replicate Rust's behavior in C.
> How is it then that my stock KUbuntu system won't let me allocate 50,000,000,000 bytes?
I can't speak to the specific implementations, however servicing an allocation request has multiple fallible steps: reserve address space, then confirm/acquire/defer the physical memory backing the allocation.
My understanding is that overcommit allows deferring assigning physical memory to the allocation, however it could still fail to find a chunk of address space to service the allocation request.
For context, steveklabnik suggested that malloc will virtually never fail on basically every Linux system, because of overcommit being the default.
I showed a trivial example of malloc failing, when trying to allocate more space than on my machine.
Another example is when using resource limits. I've modified my code to malloc only 500M bytes, the set a virtual limit of 500000KiB, which works, then 40000KiB, which doesn't
$ grep malloc tmp.c
char *s = malloc(500000000ULL);
$ cc tmp.c
$ ulimit -S -v 500000
$ ./a.out
Look at all that memory!
$ ulimit -S -v 400000
$ ./a.out
Boo, hoo!
I clearly disagree with steveklabnik's because it's easy to demonstrate cases where malloc fails on Linux-based machines.
It’s not any of that. It’s because of overcommit being the default for basically every Linux system. With that, malloc will never fail, and it’s the later access of that memory that will. In practice, you’ll virtually never see malloc actually return a failure, and so most software, no matter the language, is generally not robust to this condition.
To get back to the essay, I can easily understand why someone who has Rust as their first-and-only systems language, and therefore expects abort-when-out-of-memory, will not immediately consider how C allows a different approach to how to handle that condition, but instead will try to replicate Rust's behavior in C.
"I can program FORTRAN in any language." :)
I can't speak to the specific implementations, however servicing an allocation request has multiple fallible steps: reserve address space, then confirm/acquire/defer the physical memory backing the allocation.
My understanding is that overcommit allows deferring assigning physical memory to the allocation, however it could still fail to find a chunk of address space to service the allocation request.
For context, steveklabnik suggested that malloc will virtually never fail on basically every Linux system, because of overcommit being the default.
I showed a trivial example of malloc failing, when trying to allocate more space than on my machine.
Another example is when using resource limits. I've modified my code to malloc only 500M bytes, the set a virtual limit of 500000KiB, which works, then 40000KiB, which doesn't
I clearly disagree with steveklabnik's because it's easy to demonstrate cases where malloc fails on Linux-based machines.