we would like to export elastic index into local file (or by rest), to some kind of format (for example, json).
our index contains million of documents (~1 GB size).
is there a recommended way for doing this using some kind of API?
fetching the data by scrolling have it's own limitations (the number of docs to fetch is limited by 10K by default) - so it's less of an option, there is also the search After API - but we have some limitations that are currently prevent using it.
our final goal is to create a script that will fetch the data into a local file.
any help will be appreciated,