ElasticSearch partial re-index with Logstash

Hi All!

I'm trying to re-index data from one ES cluster to another with { "match_all": {} } query, but I'd like to limit the result set by a certain number. Currently, all "total" hits are being fetched from the source ES cluster.
Is there a way to stop the input when reaching a certain number of documents?
The "size" parameter of the ES input plugin is applicable for the scrolling only.

Thank you!

There is not.

Thank you for the fast reply!
Do you think it'd make sense to implement a new config parameter (e.g. "count") to achieve that? Thanks!

Maybe, but what is the use case, why do you only need a certain number of things?

For what it is worth _reindex in Elasticsearch 2.3+ supports limiting the number of documents copied. I added that because it makes it very convenient to play with smaller data sets.

@warkolm, the use case - to split one big index into chunks, in order to analyse them separately.
@nik9000, thank you for the hint! You are the genius! :slight_smile:

Can you possibly give a small example of implementing _reindex? It sounds like it's what I need, but I can't find any documentation on the website.

It's an ES thing - https://www.elastic.co/guide/en/elasticsearch/reference/2.3/docs-reindex.html

Great, thanks for the quick reply, and link!