Filebeat harvests log from elasticsearch and store field 'timestamp' without converting to utc time

[wangqinghuan (Wang)]

hi
I am using elasticsearch module of filebeat to harvest log files of elasticsearch and visualizing log info in kibana. Having configured and started filebeat, the log is successfully stored in elasticsearch and showed in kibana.However, the time in kibana web UI differs 8 hours from the time in log. For example, the @timestamp field is '2019-01-16T12:48:52,399' ( the local time without suffix letter Z), the time in kibana Discover is 'January 16th 2019, 20:48:52' that is added by 8 hours.
The @timestamp without letter Z implicits its local time rather than utc time, kibana should have parsed @timestamp with local zone(+0800) rather than zero zone. How to set kibana parse @timestamp(without suffix letter Z) with local zone ?
A result i wanted as follows:
the @timestamp field in elasticsearch is '2019-01-16T12:48:52,399' and the time in Kibana web ui is 'January 16th 2019, 12:48:52''. thanks

[wangqinghuan (Wang)]

Maybe I should ask how to log the timezone to @timestamp, so the kibana could parse or interpret @timestamp correctly.

[shaunak (Shaunak Kashyap)]

Hi, a few questions to start off:

1. Which exact version of Filebeat are you using?

2. You mentioned you enabled the elasticsearch module in Filebeat. The module consists of various filesets, all of which are enabled by default. Do you know which fileset is in use in your case? Alternatively, which Elasticsearch log file are you trying to harvest?

My suspicion is that you are running into this bug: https://github.com/elastic/beats/issues/9756. This was fixed recently (see https://github.com/elastic/beats/pull/9761) and the fix will be released shortly.

[wangqinghuan (Wang)]

yes, I got issue-9756 that you had resolved and added "timezone:'Asia/Shanghai'" in pipeline temporarily. It works, Thanks.