algoatson.github.io ~/posts
segfault@algoatson.github.io
back to index

when an integer becomes a memory bug

2026-01-083 min readpwn

The most deniable memory-corruption bugs don't look like memory bugs at all. The code allocates the size it computed and copies the length it was given — both "correct." The error happened earlier, in arithmetic, where a value wrapped, truncated, or changed sign. By the time it reaches malloc or memcpy, the bug is already baked into a perfectly innocent-looking call.

target./alloc
arch
amd64-64-little
bug class
integer overflow -> undersized allocation / OOB
  • NXon

three ways the math betrays you#

Multiplication overflow in a size. The classic. count * size wraps past SIZE_MAX, so the allocation is tiny — then the loop writes count elements into it:

undersized bufferc
1
2
3
4
5
void *make(uint32_t count) {
    char *buf = malloc(count * 16);     // count=0x10000001 -> 0x10 bytes (wrapped)
    for (uint32_t i = 0; i < count; i++)
        buf[i*16] = read_byte();        // writes far past the 0x10-byte buffer
}

Signedness / negative length. A signed length passes a <= check while negative, then becomes enormous when used as an unsigned size:

the check that doesn'tc
1
2
3
4
int len = get_len();                    // attacker sends -1
if (len <= MAX) {                       // -1 <= MAX is TRUE
    memcpy(dst, src, len);              // len -> (size_t)-1 = huge copy
}

Truncation. A 64-bit length stored into a 32- or 16-bit field loses its high bits. The bounds check runs on the wide value; the copy uses the narrow one (or vice-versa), and the two disagree.

why it lands as corruption#

There's no out-of-bounds expression anywhere — the OOB is emergent. The allocation believes one size, the population believes another, and the gap between them is your overflow. For the multiplication case, picking count so that count * 16 mod 2^64 is small but count is large gives a heap overflow whose length you control:

trigger.pypython
1
2
3
count = 0x1000000000000001        # *16 wraps to 0x10; loop still runs 2^60+ times
# in practice you choose count so the wrapped size is a clean chunk and the
# loop writes exactly into the next chunk's header -> classic heap overflow

From there it's an ordinary heap overflow — corrupt the next chunk's size, or its fd, and you're back in tcache poisoning or unsorted-bin territory.

the allocation was 'right' the whole time

malloc got exactly the (wrapped) number it was handed; the bug was three lines upstream, in a multiply:

txt
segfault{0x10000001_times_16_is_not_what_you_think}

the defences#

  • Checked arithmetic. __builtin_mul_overflow(count, size, &n) and friends turn the wrap into a detectable error; calloc(count, size) does this check for you — prefer it over malloc(count*size).
  • size_t everywhere, and check after promotion. Mixing int/size_t invites the signed-negative-becomes-huge bug; validate the value in the type it'll be used as.
  • -fsanitize=integer,undefined in testing catches wraps and bad conversions at runtime; -Wconversion flags the truncations at compile time.

The lesson: bounds checks protect the access, but the bug is often in the arithmetic that feeds the access. Audit every a * b, every signed-to-unsigned boundary, every assignment into a narrower type — those are where a memory-safety bug hides inside code that never touches a pointer wrong.