Consistent Query Impact on Sharding and Health in Elasticsearch

Elastic Stack Elasticsearch
1

We are using elasticsearch version 1.4, and have a looping pagination query used to collect docs from one index.

How does querying an entry in elasticsearch repeatedly have an impact on sharding, and why would it have an impact on shards of other indexes as well?

2

If this is indeed the version you are using I would strongly recommend you upgrade to the latest version as it is very, very old and it may be difficult to find anyone with any reasonably recent experience of it nowadays.

I also do not really understand your query so it would help if you could explain in more detail exactly what you are doing and what problem you are facing.

3

Hi Christian,

Our query is just set to gather docs that fulfill a case of having set flags.

For each pagination query (in a loop), we hit 10,000 documents (the maximum?) and adjust the FROM parameter to obtain the next set of documents. The total number of documents is > 10 million.

We also use the cluster health endpoint to wait for a green status, but is it necessary to wait on the number of active/relocating shards as well? How does querying documents impact that?

Is efficient pagination available in es versions <= 1.4?

4

Furthermore, it was noticed that the cluster health was impacted after the script was done running?

How could that happen?

5

I have no idea. Elasticsearch 1.4 is almost 10 years old and I have not used it for probably at least 8 or 9 years, so will not be able to help much since A LOT has changed since then.