How to index documents in Elastic and Logstash 5.5 in case I am interested in analyzing documents created only in last 10 minutes , should I reindex and delete index every 10 minutes ? What's the best approach.
Do you wish to store the documents longer than 10 minutes even if you are only interested in analysing the docs from the last 10 minutes?
If you filter by timestamp using a range query then you'll be able to exclude all docs older than 10 minutes.
Something like:
"filter": {
"range": {
"whatever_timestampfield": {
"gte": "now-10m",
"lte": "now"
}
}
See:
https://www.elastic.co/guide/en/elasticsearch/reference/5.5/query-filter-context.html
and
https://www.elastic.co/guide/en/elasticsearch/reference/5.5/query-dsl-range-query.html
Thank you . I can store the documents longer than 10 minutes but I want my search to stop after last 10 min . I am looking for query optimization. Are documents sorted by timestamp ?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.