Scroll id is not changing while querying

Elastic Stack Elasticsearch
1

I written this code to test scroll api. As mentioned in docs the scroll id will change for every request.
But in my case it is same for all requests.

The initial search request and each subsequent scroll request returns a new _scroll_id — only the most recent _scroll_id should be used.

Here is my code sample

from elasticsearch import Elasticsearch

es = Elasticsearch(['10.225.253.130:9200'], sniff_on_start=True)

my_dict = {}
my_dict['query'] = {}
my_dict['query']['match_all'] = {}

result = es.search(index='.monitoring-kibana-2-2017.10.27',  body=my_dict, size=1000, scroll="1m")

print result['_scroll_id']

for x in range(10):
    result_in_scroll = es.scroll(scroll_id=result['_scroll_id'], scroll="1m")
    print result_in_scroll['_scroll_id']
    result['_scroll_id'] = result_in_scroll['_scroll_id']
    if (len(result_in_scroll['hits']['hits']) == 0):
        break
    
print "Terminated correctly"

This is the output I got

DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
DXF1ZXJ5QW5kRmV0Y2gBAAAAAACXzVwWTXZBaGJGOE5UZW1vWGxabjhfM0JnQQ==
Terminated correctly

My scroll is same in all cases. Is this expected behaviour or not???

I am using 5.4.0 ES

2

Short answer: yes, if you have a single shard index (as seems to be in your case) - it is expected behavior, but it can happen even if you have multiple shards. Longer answer: the scroll basically contains a list of shards where your search is running plus information about how to find your scroll data on each shard. As you exhaust results from each shard, you will notice that the scroll id becomes shorter, because we no longer need to search these shards and therefore don't need to list them on scroll. But if only have one shard or all shards will get processed at the same time, your scroll id might never change. Saying this, I wouldn't rely on this behavior since it might change in the future and always copy scroll id from the previous response.

2 Likes
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.