scroll-9.3-python.md

endpoint	scroll
lang	python
es_version	9.3
client	elasticsearch==9.3.0

Elasticsearch 9.3 scroll endpoint (Python example)

Use client.scroll() to retrieve the next batch of results from a scrolling search. Start by calling search with a scroll parameter, then repeatedly call scroll until no more hits are returned:

response = client.search(
    index="products",
    scroll="1m",
    size=2,
    query={"match_all": {}},
)

scroll_id = response["_scroll_id"]
hits = response["hits"]["hits"]

while hits:
    for hit in hits:
        print(f"{hit['_id']}: {hit['_source']['name']}")

    response = client.scroll(scroll_id=scroll_id, scroll="1m")
    scroll_id = response["_scroll_id"]
    hits = response["hits"]["hits"]

Cleaning up

Always clear the scroll context when you are done to free server resources. Pass the last scroll_id you received:

client.clear_scroll(scroll_id=scroll_id)

To clear every open scroll context at once, omit the scroll_id:

client.clear_scroll()

Using helpers

The scan helper wraps scrolling into a simple iterator. Prefer this over manual scrolling in most cases:

from elasticsearch.helpers import scan

for hit in scan(client, index="products", query={"query": {"match_all": {}}}):
    print(f"{hit['_id']}: {hit['_source']['name']}")