feat(btreemap): add direct-mapped node cache for read operations#416
Merged
sasa-tomic merged 55 commits intomainfrom Apr 1, 2026
Merged
feat(btreemap): add direct-mapped node cache for read operations#416sasa-tomic merged 55 commits intomainfrom
sasa-tomic merged 55 commits intomainfrom
Conversation
Add a 32-slot direct-mapped node cache to BTreeMap that avoids re-loading hot nodes from stable memory. Modeled after CPU caches: O(1) lookup via (address / page_size) % 32, collision = eviction. Read paths (get, contains_key, first/last_key_value) use a take+return pattern to borrow nodes from the cache without RefCell lifetime issues. Write paths (insert, remove, split, merge) invalidate affected cache slots. Key changes: - Switch get() from destructive extract_entry_at to node.value() - Remove unused extract_entry_at method - Change traverse() closure from Fn(&mut Node) to Fn(&Node) - Invalidate cache in save_node, deallocate_node, merge, clear_new Expected improvement: ~15-20% for random reads, ~65% for hot-key workloads, ~0% overhead for writes (cache.get_mut() bypasses RefCell).
|
|
|
|
|
|
maksymar
reviewed
Mar 25, 2026
…expose hit/miss stats
maksymar
reviewed
Mar 25, 2026
maksymar
reviewed
Mar 26, 2026
maksymar
reviewed
Mar 26, 2026
maksymar
reviewed
Mar 27, 2026
maksymar
reviewed
Mar 27, 2026
sasa-tomic
commented
Mar 30, 2026
Member
Author
sasa-tomic
left a comment
sasa-tomic
left a comment
There was a problem hiding this comment.
Addressing open review comments with code changes and replies.
…ard NULL - Change DEFAULT_NODE_CACHE_NUM_SLOTS from 0 to 16 (covers top 2 tree levels) - Remove redundant cache_num_slots field from BTreeMap; NodeCache methods now self-guard via is_enabled() and callers no longer need checks - Add NULL address guard in NodeCache::take to prevent false cache hits
maksymar
approved these changes
Mar 30, 2026
Contributor
maksymar
left a comment
maksymar
left a comment
There was a problem hiding this comment.
Great results! 🎉 Thank you!
maksymar
reviewed
Mar 31, 2026
Contributor
|
[DONE] |
Contributor
|
The chart shows how increasing node cache size reduces instruction count (relative to no cache) across operation categories.
From this data I'm happy to see that "the default 16" is already good enough while being reasonably small, at the same time bigger caches will provide more value. 
|
maksymar
reviewed
Apr 1, 2026
sasa-tomic
changed the title
sasa-tomic
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Solution
with_node_cache()andnode_cache_resize().NodeCacheMetricsfor monitoring hit ratio, cold misses, and collision misses.read_value_uncached()to avoid inflating cached nodes with large values.Impact
Details
get(),contains_key(),first/last_entry()now use the cache.insert,remove,merge) invalidate affected cache slots.Related to #166