Logstash Configuration doubts

Elastic Stack Logstash
1

Hi All,

We are working on Indexing system logs ( nagios log ) to Elasticsearch using logstash.
We have two basic doubts on this .

  1. We are going to parse nagios log to logstash for every minute,
    For this every minute , we need to send whole nagios log to logstash , then there is duplicate issue for all records expect new lines in log.
    How to overcome this issue?

  2. which filter is flexible for this nagios log parsing , please suggest us!!

Please Help on this and Thanks in Advance :slight_smile:

2

For this every minute , we need to send whole nagios log to logstash

Why? It doesn't appear to make any sense.