@timestamp kibana and log time are differents

[zolthar-z (Luis Zabala)]

HI

I have a situation with my elastic platform, I send logs from filebeat directly to elasticseach and visualize it using kibana, the problem is that the logs sent from filebeat have the time for the client servers and kibana use the clock from the web-browser, I has setup the same time format (UTC) in the clients/kibana but I still have some mismatch between the time inside the logs and Kibana @timestamp (due they not have the same NTP server the time differs in seconds).

The idea is have the same time so the filters can be properly applied.

I was wondering if maybe one of you have the same situation before.

Thanks for your help.

Regards

[Wolfram_Haussig]

Hi,

filebeat does not parse the message read from a logfile so it doesn't know that there is a time field in the log entry and it doesn't know about the date format either.

We use LogStash to parse the message using grok filter and use the date from the log as timestamp using the date filter.

Best regards
Wolfram

[zolthar-z (Luis Zabala)]

Hi

Thanks, with logstash I could applied a filter and works fine, this is the filet so anyone can use in the future, this filter replace the @timestamp variable with the time that comes from the client logs

input {
beats {
port => 5044
}
}
filter {
grok {
match => ["message", "%{TIMESTAMP_ISO8601:timestamp}"]
}
date {
match => ["timestamp", "ISO8601"]
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
}
}