Logstash-elasticsearch input plugin, size not working as expeted

Problem: Every 20 minutes logstash fetches all the records matching as per query in batches of 100(size=100) and waits for next schedule.

in input plugin i have something like this:

elasticsearch{
size => 100
schedule => "*\20 * * * *"
}

NEED: I wanted to retrieve only 100 records from matching set of records against the query every 20 minutes from elasticsearch using logstash.

I will really appreciate the some help here.

I have changed my logstash configuration and removed size parameter.
Now it is like this:

input {
elasticsearch{
query => '{
"size": 5,
"sort": [
{
"esUpdateDate": {
"order": "desc"
}
}
],
"_source": [
"esUpdateDate",
"beUpdateDate"
]
}
schedule => "*\20 * * * *"
}
}

This also not restricting the size of documents returned by the query.

Really appreciate some help here.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.