[8.2] Fix incoming edges ghost memory leak (MOD-13761)#922
Merged
Conversation
* Add incoming edges ghost memory benchmarks (MOD-13761) Add three benchmarks to measure performance and memory impact of the incoming edges shrink_to_fit fix: 1. DeleteZeroVectorsAsync - async deletion path (production default) 2. DeleteZeroVectorsInPlace - in-place deletion path (worst-case latency) 3. InsertZeroVectorsTimed - insertion path (heuristic pruning cost) Stress scenario: 40K random + 50K zero vectors with COSINE metric, which forces hub nodes with large incoming edge vectors. Each benchmark measures ghost memory (wasted capacity) before and after shrink_to_fit, with detailed stats (percentiles, top-10, mean). Run with: make benchmark BM_FILTER=bm-index-internals-incoming-edges * results before * shrinking logic * fix uncoditionally shrink * add bm-index-internals-incoming-edges * use 1 thread * use ratio = 2, remove min * remove results before * rename to bm-hnsw-internals-incoming-edges better output
🛡️ Jit Security Scan Results
✅ No security findings were detected in this PR
Security scan by Jit
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 8.2 #922 +/- ##
==========================================
- Coverage 97.03% 96.98% -0.06%
==========================================
Files 126 126
Lines 7353 7358 +5
==========================================
+ Hits 7135 7136 +1
- Misses 218 222 +4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
alonre24
approved these changes
Mar 25, 2026
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
backport #920 to 8.2
Note
Medium Risk
Touches core HNSW graph bookkeeping by adding
shrink_to_fit()during incoming-edge removals, which could impact indexing latency and allocation behavior. Also adds new benchmarks and CI wiring, but no API or data-format changes.Overview
Fixes HNSW incoming-edges ghost memory growth by opportunistically shrinking
incomingUnidirectionalEdgesafter successful removals when capacity is disproportionately larger than size.Adds a dedicated benchmark (
index_internals_incoming_edges_fp32) to reproduce and measure the memory/latency impact of this scenario (async delete, in-place delete, and insert paths), and wires it into the benchmark runner scripts and GitHub Actions benchmark workflow.Written by Cursor Bugbot for commit eeab33f. This will update automatically on new commits. Configure here.