mem: fix potential underflow on mem size calculation