Identifying Rust's collect::<Vec<_>>() memory leak footgun (blog.polybdenum.com)
from tedu@inks.tedunangst.com to inks@inks.tedunangst.com on 18 Jan 2024 17:32
https://inks.tedunangst.com/l/5066

This is the story of how I identified the bug. (TLDR: collect::<Vec<_>>() will sometimes reuse allocations, resulting in Vecs with large excess capacity, even when the length is exactly known in advance, so you need to call shrink_to_fit if you want to free the extra memory.)

Ordinarily, that wouldn’t have been a problem, since the into_iter().map().collect() line used to pack them into (u32, u32)s would allocate a new vector with only the exact amount of space required. However, thanks to the allocation reuse optimization added in Rust 1.76, the new vec shared the backing store of the input vec, and hence had a capacity of 16560, meaning it was using 132480 bytes of memory to store only 16 bytes of data.

#malloc #programming #rust #turtles

#inks #malloc #programming #rust #turtles