Elastic Search Logstash input ->output AWS s3

Elastic Stack Logstash
1

Is it possible not to query all the data(duplication) when we ingest data from Elasticsearch input ->aws s3 output?

I want to ingest the data daily but it would duplicate it querying all the data from the start again but I want to ingest the data daily from the latest point so it would keep ingesting data daily

Here is the current query for logstash input

query => '{ "query": {
"match_all": {}
}}'

thanks

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.