Timestamp explanation

[amhulli (Andrew)]

Hi all,

I'm using ELK 7.4.2
I have an issue where Kibana throws up an error when I click on any of the bars on the graph in the discover tab (it complains about:

Request to Elasticsearch failed: {"error":{"root_cause":[{"type":"parse_exception","reason":"failed to parse date field [-813227399999] with format [epoch_millis]: [failed to parse date field [-813227399999] with format [epoch_millis]]"},
(many of these messages are displayed in the popup - presumably once for each message from the search)

I can fix it by changing the "advanced settings => timezone for date formatting" from "browser" to "GMT" but I don't want all my dates to be in GMT. I want the display to be in my local timezone as per the timestamp on all of the logs that filebeat-logstash-elasticsearch is ingesting.

What am I doing wrong here?
regards,
Andrew.

[Badger]

@amhulli

What am I doing wrong here?

Asking in the logstash forum when kibana logs an elasticsearch error.

[amhulli (Andrew)]

@badger thanks for the heads up, but that it appears to me that logstash or filebeat is adding a GMT timestamp before the data is pushed to elasticsearch despite my syslogs having a local timestamp. Hence my tagging as logstash. Apologies if this is still incorrect. I've only been at this for a couple of months.

Ouput of curl http://localhost:9200/logstash-2020.01.13-000035/_search?pretty
{
"_index" : "logstash-2020.01.13-000035",
"_type" : "_doc",
"_id" : "jmY3nG8Bx3_pbs7_1Wq4",
"_score" : 1.0,
"_source" : {
"agent" : {
"ephemeral_id" : "925f5e46-14dc-4dc1-93ff-a8c49ae04405",
"hostname" : "",
"version" : "7.4.2",
"id" : "03f53367-f3f7-482a-a69b-fae2771451a5",
"type" : "filebeat"
},
"@timestamp" : "2020-01-13T00:04:54.000Z",
"log" : {
"offset" : 4965718,
"file" : {
"path" : "/data/remotelogs/rsyslog/2020-01-13/.log"
}
},
"tags" : [
"_unfiltered"
],
"syslog_severity_code" : 5,
"ecs" : {
"version" : "1.1.0"
},
"syslog_facility" : "user-level",
"syslog_severity" : "notice",
"syslog_message" : "at 9:34:53 AM",
"@version" : "1",
"syslog_timestamp" : "2020-01-13T09:34:54+09:30",
"syslog_hostname" : "",
"input" : {
"type" : "log"
},
"syslog_program" : "",
"syslog_facility_code" : 1
}
},
note: some fields redacted from output above (hostnames etc).