Quoting @andrás-kovács
in the default moving garbage collector, mutable arrays are always present in a list ("mutable list") that gets scanned on every GC run. Each array has its own dirty flag and bitmask for detecting changed elements. An unsafe freezing marks the array so that on the next garbage collection it gets removed from the list. After this, if you do a mutable write to the array, that marks the array itself as dirty, but it does not add the array back to the mutable list, so it does not get scanned on subsequent GC runs.
So retaining (and using) a mutable reference after unsafeFreeze-ing is the culprit sequence of operations.
(Sidenote: for a moment one might think Storable vector doesn't have this problem, as its memory is not managed by GHC RTS, and refers the same ForeignPtr.. until one realizes that Storable vectors can't store references anyway. So in a sense Storable vectors duck this problem, but it would still not be nice to rely on this pattern - in case the operation gets generalized beyond Storable and this detail is missed).