Skip to content

Issue 2422: Fix SyncLedgerIterator.hasNext() failing to iterate across ZK ledger ranges#4731

Open
geniusjoe wants to merge 1 commit intoapache:masterfrom
geniusjoe:bugfix/SyncLedgerIterator-hasNext
Open

Issue 2422: Fix SyncLedgerIterator.hasNext() failing to iterate across ZK ledger ranges#4731
geniusjoe wants to merge 1 commit intoapache:masterfrom
geniusjoe:bugfix/SyncLedgerIterator-hasNext

Conversation

@geniusjoe
Copy link

@geniusjoe geniusjoe commented Mar 17, 2026

Fix #2422

Related PR: #2457

Motivation

The SyncLedgerIterator.hasNext() method in BookKeeper.java has a logic bug that causes it to stop iterating prematurely when ledgers span across multiple ZooKeeper ranges.

ZooKeeper stores ledgers in a hierarchical tree structure (e.g. /ledgers/00/0000/...). Each range (znode) holds at most 10,000 leaf-level ledger IDs.
The 10,000 limit is determined by the 4-digit last level (0000-9999) as defined in LegacyHierarchicalLedgerManager (2-4-4 split) and LongHierarchicalLedgerManager (3-4-4-4-4 split). When the number of ledgers exceeds 10,000, they are distributed across multiple ranges.
image

The original hasNext() implementation had the following logic:

@Override
public boolean hasNext() throws IOException {
    parent.checkClosed();
    if (currentRange != null) {
        if (currentRange.hasNext()) {
            return true;
        }
    } else if (iterator.hasNext()) {
        return true;
    }
    return false;
}

When currentRange is non-null but exhausted (no more elements), it falls through to return false without checking iterator.hasNext() for more ranges.

Changes

  1. Fix SyncLedgerIterator.hasNext() in BookKeeper.java:
    Simplified the conditional logic to properly fall through to iterator.hasNext() when the current range is exhausted
  2. Add testListLedgersLargeScale() test in LedgerMetadataTest.java
    Added a new test case with numOfLedgers = 10010 (> 10,000 per-range limit) to verify cross-range iteration works correctly.

Copy link
Contributor

@hanmz hanmz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, good catch

Copy link
Member

@wenbingshen wenbingshen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BP-42: New Client API - list ledgers

4 participants