Problem: Every 20 minutes logstash fetches all the records matching as per query in batches of 100(size=100) and waits for next schedule.
in input plugin i have something like this:
elasticsearch{
size => 100
schedule => "*\20 * * * *"
}
NEED: I wanted to retrieve only 100 records from matching set of records against the query every 20 minutes from elasticsearch using logstash.
I will really appreciate the some help here.
I have changed my logstash configuration and removed size parameter.
Now it is like this:
input {
elasticsearch{
query => '{
"size": 5,
"sort": [
{
"esUpdateDate": {
"order": "desc"
}
}
],
"_source": [
"esUpdateDate",
"beUpdateDate"
]
}
schedule => "*\20 * * * *"
}
}
This also not restricting the size of documents returned by the query.
Really appreciate some help here.
system
(system)
Closed
February 11, 2019, 6:46pm
3
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.