Elasticsearch and scrolling in increments

Elastic Stack Elasticsearch
1

Say i have 100,000 indices. i can do

curl -XGET '10.55.91.111:9200/house/node2/_search?scroll=10m&size=100000' -d

but as the indices grow, i would probably run out of heap space.

So my question is can i do the search in say increments of 1000?

so something like this...

while size < 1000

count = 1000
#curl -XGET '10.55.91.111:9200/house/node2/_search?scroll=10m&size=<count>' -d

_search the first 1000
process the information
now get the next 1000
process the information
until i hit 10000


I saw something where the curl command allows you to search from 1 to 1000,
then search from 1001 to 2000...etc..
2

Scroll like that should retune a scroll id which you can use with the
scroll API to continue where that API left off.

6

Okay. i got it to work. The one below works:

curl -XGET '10.55.304.22:9200/_search/scroll?scroll=20m' -d 'cXVlcnlUaGVuRmV0Y2g7NTsxODM5MTE6alVhTTY0bnVTdy0wc2RpR3lndFhXZzsxODQzODU6eG94by1rY1dSM2VFSWNJTkJJZEJYQTsxODM5MTI6alVhTTY0bnVTdy0wc2RpR3lndFhXZzsxODQzODY6eG94by1rY1dSM2VFSWNJTkJJZEJYQTsxODQzODc6eG94by1rY1dSM2VFSWNJTkJJZEJYQTswOw=='

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.