In Suricata we’re often not printing malloc errors. The reason is that we’re not willing to print such errors based on (attacker controlled) traffic. So often such cases are silently handled.
We came across a bug though, where a integer underflow led to -1/0xffffffff being passed to malloc. Luckily, malloc just failed by returning NULL, and this return was properly handled. Still, passing such a large value to malloc is a bug, so I would like to catch it.
This turned out to be trickier than I thought.
valgrind says: ==10274== Warning: silly arg (-1) to malloc(). However, it does not count it as an error. So calling valgrind with –error-exitcode=255 (my usual choice) doesn’t work.
glibc’s malloc responds to an environment variable, MALLOC_CHECK_ (I tried values 0 to 7), but this didn’t catch it at all.
AddressSanitizer also detects it as something non-fatal: ==18885== WARNING: AddressSanitizer failed to allocate 0xffffffffffffffff bytes. Not fatal like the usual errors, and no exit code.
tcmalloc, preloaded through LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so.0, also just prints something to stderr: tcmalloc: large alloc 0 bytes == (nil) @
I also tried electric-fence, however this failed to work for me altogether.
As I wanted this check to be part of my QA, I needed an automated check. In this case however, I saw no other way than to just inspect the stderr output of one of the tools above. My choice was tcmalloc, as it’s fast and doesn’t require compile time options.
