Vectors are owned by tracked_vectors_. Tombstoned slots still need to have their vectors tracked or else they’re freed prematurely and the raw vector pointer in the HNSW slots are dangling. This was the issue when an update overwrote the tracked_vectors_ entry but didn’t overwrite the same slot at the HNSW level addressed in #1283.
I’m now seeing we also have a clear memory leak with allow-replace-deleted enabled. UnTrackVector() is a no-op no matter what. When a tombstoned slot is overwritten, we need to pass the label up and clean it up from tracked_vectors_. Alternatively we could pass ownership of the vector to the HNSW layer for tombstoned slots and it could be deleted at the library level.
Vectors are owned by
tracked_vectors_. Tombstoned slots still need to have their vectors tracked or else they’re freed prematurely and the raw vector pointer in the HNSW slots are dangling. This was the issue when an update overwrote the tracked_vectors_ entry but didn’t overwrite the same slot at the HNSW level addressed in #1283.I’m now seeing we also have a clear memory leak with allow-replace-deleted enabled.
UnTrackVector()is a no-op no matter what. When a tombstoned slot is overwritten, we need to pass the label up and clean it up from tracked_vectors_. Alternatively we could pass ownership of the vector to the HNSW layer for tombstoned slots and it could be deleted at the library level.