I searched Google and Stackoverflow but could not find anything on this but 'totalHits' gets changed unexpectedly while scrolling SearchResponse which seems to be from 2010 only offers a workaround.
One of our ES clusters (8x data, 3x master,
quay.io/pires/docker-elasticsearch-kubernetes:6.3.0) runs on GKE running on preemptible nodes. Much like AWS' spot instances they restart every few hours. We use
settings.index.unassigned.node_left.delayed_timeout: 15m to reduce reallocation.
- We have processes that use
scrollto do long-running exports
- We encountered that the
hits.totaldecreases while scrolling
- Our assumption is that the scroll context is not replicated and even a single node leaving/restarting therefore makes part of the scroll context unavailable.
- Is my assumption correct, and is this intended behavior?
- Is there a better solution than sorting, using
fromand hoping that not too much changed in the meantime?
- Is it possible to make scrolls stateless, aka request the exact same window multiple times? In that case we could simply wait until the node finished restarting and everything would be fine.
Thank you very much!