Last line of log repeatedly gets indexed/processed to elastic/kibana

Elastic Stack Logstash
1

I have a filebeat > logstash > elastic set up and I am working with a multi-line log format.

I am not sure why the last line of the log file is repeatedly getting indexed. What I would ideally want is that filebeat would wait for new log entries and process them as they arrive.
Here's my filebeat.yml:

filebeat.inputs:
- type: log
  enabled: true

  paths:
    - /var/log/some.log

  ### Multiline options
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after

output.logstash:
  hosts: ["xx.xx.xx.xx:5044"]

For testing purposes, I delete the index before starting logstash & filebeat. So the index should be clean. By the way, I am not creating the index manually - it's created by the logstash config code in its output section.

Here's an example of repeated entries of the last line in kibana:

image

What config changes do I need to make to fix this? Thanks!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.