Confused about timezone settings

Elastic Stack Logstash
1

Hi Guys

I have bunch of servers that has logfiles shown in danish local time, which currently is UTC+1.
The filebeat on the server where it harvest the logfile does NOT have a timezone set in the config.
In the logstash I have this set up:

date {
        match => [ "Time", "MM/dd/yyyy HH:mm:ss aa", "M/d/yyyy HH:mm:ss aa", "M/d/yyyy H:m:ss aa", "dd-MM-yyyy HH:mm:ss", "d-M-yyyy HH:mm:ss", "MM/dd/yyyy", "M/d/yyyy", "dd-MM-yyyy", "d-M-yyyy" ]
        timezone => "Europe/Copenhagen"
}

When I look in Kibana it is shown like this (for an event occuring at 9:39 local time):

{
  "_index": "portraitcounting-6.5.1-2019.01",
...
    "@timestamp": "2019-01-21T09:39:27.975Z",
    "message": "  2019-01-21  9:39:27.975 10B8 [ZULU -] Sent 5 messages in transaction 1 on output channel 1001, queue 5",
...
  "fields": {
    "@timestamp": [
      "2019-01-21T09:39:27.975Z"
    ]
...
}

When viewed in Kibana it then shows 10:39

Shouldn't the timestamp be 8:39 because it is in Zulu time (UTC), and then with the locale set in the browser, it should be changed to 9:39?

2

What is the content of the Time field you are matching against? The timestamp in the message field does not seem to match the formats you have specified in the date filter. How come? Why not solve this by deploying Filebeat on the hosts where the data is being generated?

1 Like
3

Sorry, please forget the above date matching, it was the wrong filter, thus the wrong format, and the wrong date match.
The right filter did not have a timezone line added, so I did that.
Thankyou for pointing out the problem (sort of anyways :smiley: )

5

Ok, so it seems that it doesn't hit the mark still.

The correct filter looks like this:

grok {
  match => {
     "message" => [ "^  %{NOTSPACE:date}%{SPACE}%{NOTSPACE:clock} ...

...

mutate {
  add_field => {
    "datetime" => "%{date} %{clock}"

...

date {
  match => [ "datetime", "yyyy-MM-dd H:mm:ss.SSS",  "yyyy-MM-dd HH:mm:ss.SSS"]
  timezone => "Europe/Copenhagen"
 }

The JSON looks like this:

"@timestamp": "2019-01-21T11:23:44.510Z",
"message": "  2019-01-21 11:23:44.510 12C0 [TAXILA -] Sent 1 messages in transaction 1 on output channel 1001, queue 6",
...
"@timestamp": [
  "2019-01-21T11:23:44.510Z"

Oh, and I do have filebeat deployed on the 'source' server.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.