How Logstash reparse Elasticsearch input

Elastic Stack Logstash
1

Hi All,

Suppose I have elasticsearch index_1 with some data. I've created a new index index_2 and created a logstash pipeline to replicate data from index_1 to index_2. After having all data in index_2, pipeline is closed.

Now I've inserted new documents in index_1, and run logstash pipeline again. What will happen now,

  1. will it replicate all the data from index_1?

  2. or only newly created documents will be considered? if yes, how logstash keep tracking of documents. I can see 'timestamp' added in logs, but not able to understand the behaviour.

Thanks,
Abhishek

2

It will do the first one unless you put a query that's associated with the timeframe in the input.