Wrong timestamp in kibana index

Elastic Stack Elasticsearch
1

I get worng time data in my index in kibana there is around lot of difference in @timestamp and log_timestamp field.

grok filter which i am using

          grok {
            match => [ "message" , '%{IP:remote_addr} - - \[%{HTTPDATE:log_timestamp}\] "%{IPORHOST:http_host}" "%{WORD:request_method} %{DATA:uri} HTTP/1.1" %{NUMBER:response} %{NUMBER:bytes} (?:%{DATA:http_refer}|-) %{QS:agent} "(?:%{IPV4:upstream_addr}|-)" (?:%{NUMBER:upstream_connect_time}|-) (?:%{NUMBER:upstream_header_time}|-) (?:%{NUMBER:upstream_response_time}|-) (?:%{NUMBER:request_time}|-)' ]
            overwrite => [ "message" ]
          }
         mutate {
           convert => ["response", "integer"]
           convert => ["bytes", "integer"]
           convert => ["responsetime", "float"]
           rename => ["agent", "filebeat_agent" ]
         }
         geoip {
           source => "remote_addr"
           add_tag => [ "nginx-geoip" ]
         }
         date {
           match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
           remove_field => [ "timestamp" ]
#           target => "@timestamp"
         }
         useragent {
           source => "agent"
         }
  }
}

To push log from client i am using filebeat nginx module with some fields. See below attached image you can see time difference in index

elk
elk1234×2947 252 KB

2

Is this your full configuration pipeline?

Your date filter is trying to match the field timestamp, but your grok does not create a field called timestamp, it creates a log_timestamp.

Have you tried using log_timestamp in your date filter?

3

Thanks it worked for me.
Thanks for your help

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.