Under what circumstances does hits.total change while scrolling?

Elastic Stack Elasticsearch
1

Dear community,

I searched Google and Stackoverflow but could not find anything on this but 'totalHits' gets changed unexpectedly while scrolling SearchResponse which seems to be from 2010 only offers a workaround.

One of our ES clusters (8x data, 3x master, quay.io/pires/docker-elasticsearch-kubernetes:6.3.0) runs on GKE running on preemptible nodes. Much like AWS' spot instances they restart every few hours. We use settings.index.unassigned.node_left.delayed_timeout: 15m to reduce reallocation.

  • We have processes that use scroll to do long-running exports
  • We encountered that the hits.total decreases while scrolling
  • Our assumption is that the scroll context is not replicated and even a single node leaving/restarting therefore makes part of the scroll context unavailable.

My questions:

  1. Is my assumption correct, and is this intended behavior?
  2. Is there a better solution than sorting, using from and hoping that not too much changed in the meantime?
  3. Is it possible to make scrolls stateless, aka request the exact same window multiple times? In that case we could simply wait until the node finished restarting and everything would be fine.

Thank you very much!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.