How to find lag between filebeat and logstash based on timestamp

(Gangadhar Mahadevan) #1


I am trying to find lag between filebeat sending a message and time at which logstash reads and process it from topic and not sure how to achieve it.

This is snippet of my filebeat config


  - '/var/log/syslog'
  - '/var/log/auth.log'
  - '/var/log/messages'
input_type: log
  source: filebeat
  format: syslog
document_type: "{{ stack_env }}-{{ datacenter }}-{{ datacenter_type }}-syslog"

And we have logstash runners in kafka topic with input, filter and output configration

below is sample message I read from syslog topic,

{"@timestamp":"2017-09-11T21:17:02.778Z","beat":{"hostname":"xxx","name":"xxx","version":"5.2.0"},"fields":{"format":"syslog","source":"filebeat"},"input_type":"log","message":"Sep 11 21:17:01 xxx)","offset":,"source":"/var/log/syslog","type":"xx-syslog"}

I had a filter like
kv {
field_split => ","
ruby {
code => "


if k.include? '@timestamp'

         event.set('access_input_filebeat_time', event.get('@timestamp'))

to capture @timestamp and my understanding was the @timestamp field when I read message is time at which filebeat sent the log

I am also aware logstash creates @timestamp field when it parses log. But when I look at kibana the @timestamp(which I believe logstash creates) and new filed access_input_filebeat_time has same value.

My goal is to find the lag between time at which filebeat read the message from /var/log file and time at which logstahs runner received and processed it and capture them in 2 different field in kibana. Kindly advise. Thanks?

(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.