Wednesday, September 21, 2011

Clobbering a memory leak

I spent today dealing with a subtle bug that had me slogging through code trying to figure out why my application was running out of memory. Adding more memory allowed the program to run longer, so I suspected a (relatively) slow leak. I added reference counters to ensure that every allocation had a matching free, and the reference counters did not indicate any memory regions were being lost. I should not have been exhausting the available memory, and yet I was.

By checking the return value of free I was able to determine that I was passing free an invalid pointer to a location not on the heap. But I was passing it a pointer to an object I allocated! Something was corrupting the pointer.

Normally I would suspect a stack overflow, but since I was using some inline assembly macros nearby I double-checked them. Sure enough I had forgotten to list one of the clobber registers in one of the macros so the compiler did not know that I was using that register. The register was assigned to hold the argument to free but was overwritten by the assembly code causing the original pointer to be silently leaked. By adding the correct register to the clobber list I fixed the memory leak.

No comments:

Post a Comment