Elastic Search Logstash input ->output AWS s3

Is it possible not to query all the data(duplication) when we ingest data from Elasticsearch input ->aws s3 output?

I want to ingest the data daily but it would duplicate it querying all the data from the start again but I want to ingest the data daily from the latest point so it would keep ingesting data daily

Here is the current query for logstash input

query => '{ "query": {
"match_all": {}


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.