Issue 2422: Fix SyncLedgerIterator.hasNext() failing to iterate across ZK ledger ranges#4731
Open
geniusjoe wants to merge 1 commit intoapache:masterfrom
Open
Issue 2422: Fix SyncLedgerIterator.hasNext() failing to iterate across ZK ledger ranges#4731geniusjoe wants to merge 1 commit intoapache:masterfrom
geniusjoe wants to merge 1 commit intoapache:masterfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix #2422
Related PR: #2457
Motivation
The
SyncLedgerIterator.hasNext()method inBookKeeper.javahas a logic bug that causes it to stop iterating prematurely when ledgers span across multiple ZooKeeper ranges.ZooKeeper stores ledgers in a hierarchical tree structure (e.g.

/ledgers/00/0000/...). Each range (znode) holds at most 10,000 leaf-level ledger IDs.The 10,000 limit is determined by the 4-digit last level (0000-9999) as defined in
LegacyHierarchicalLedgerManager(2-4-4 split) andLongHierarchicalLedgerManager(3-4-4-4-4 split). When the number of ledgers exceeds 10,000, they are distributed across multiple ranges.The original
hasNext()implementation had the following logic:When
currentRangeis non-null but exhausted (no more elements), it falls through toreturn falsewithout checkingiterator.hasNext()for more ranges.Changes
SyncLedgerIterator.hasNext()inBookKeeper.java:Simplified the conditional logic to properly fall through to
iterator.hasNext()when the current range is exhaustedtestListLedgersLargeScale()test inLedgerMetadataTest.javaAdded a new test case with
numOfLedgers = 10010(> 10,000 per-range limit) to verify cross-range iteration works correctly.