Last 15/30 min data being displayed in UTC whereas in Today its the actual Timestamp inserted in ES

Elastic Stack Kibana
1

Hello Everyone.

My filebeat is pushing logs into ES via pipeline. The current Timezone (IST) is applicable on the filebeat server and also on the ES server. As such, if log gets generated at 11/04/2018 13:40:00 IST same is captured by filebeat and also inserted in ES. I can see the same timestamp in ES as well.

Now, in dashboard visualization as well as in Discover tab, If I select any option from "Today" vertical column and "Yesterday" column, all the data is being displayed in captured Timestamp (i.e. IST)

However, when I view data for "Last 15 min" column, everytime the data fetched is as per UTC and not IST although data inserted in ES is IST.

This is abnormal as if i see the data from "Today" its showing as per the Timestamp I indexed, whereas if i select "Last 15 min or Last 30 min", Kibana falls back to UTC.

PS. In Kibana>Management>Advanced Settings : Timezone is set to UTC.

IF i change the Timezone from UTC to IST in Kibana (Advanced Settings), then "Today" column starts showing data with IST (Already indexed+5:30)

Under normal circumstances, any timerange selected to view the data from Kibana, it should follow one TimeZone which is not the case.

Please guide.

Info : ES Version : 5.2
Kibana Version : 5.2
TimeZone in Kibana Management>Advanced Settings : UTC

2

Hey @jiouser, are you Logstash to ingest your logs or are you using the Elasticsearch Ingest Node? Generally, you want to make sure that your dates are being stored in UTC so that Kibana is able to present them properly.

3

Hi Brandon, We are using Elastic Ingest Node. Logstash is not used. We cannot store date in UTC format as logs generated are in IST timezone and we want to preserve that time zone.
This is a strange issue that half of the things are working as per correct timestamp (IST) in Kibana and other half is working in UTC.

Can you please assist so that there is a uniform behaviour across Kibana Dashboard Visualization.

4

Timestamps in Elasticsearch are always stored internally as UTC, and the logic around timezone conversions in Elasticsearch and Kibana rely on this. This allows Kibana to convert timestamps in visualisations to the local timezone or the one specified in the advanced settings. The time-picker will also convert according to this based on the assumption that UTC is used internally. If you store timestamps that are not in UTC you are breaking this and Elasticsearch the transformations Elsticsearch and Kibana performs will be wrong. The result is that you are likely to see issues like the ones you are experiencing now.

1 Like
5

Christian, you mean to say I should insert application data in ES as per UTC but not my local timezone ? This is pretty strange as application hosted in a particular timezone with users/application operating and accessing the data should always have data with proper timestamp(specific to TimeZone they are in). Queries are also modelled similarly in application. This is pretty odd.

Problem with datepicker is half of the datepicker is working as per current IST probably relying on the TS inserted in ES while indexing, while the other options in Timepicker are working as per UTC or some other problem.

Kindly advise what should be the approach as we surely cannot change the Timestamp of the indexed data to UTC nor we can ensure that data being indexed should be in UTC.

6

Yes, that is exactly what I am saying. Elasticsearch will assume that all timestamp are in UTC, so if you store anything else you are likely to have issues.

That is what you need to do.

7

Well, this seems to be a bug in Kibana. For sure, no Database or search store can put restrictions/advise on applications to store date in UTC format while application is generating data/logs in local timezone.
Instead, Kibana should be able to present the data in Dashboard as per the timestamp indexed. Additionally, giving an option for end user to change it by changing the time zone.
This involves extra overhead on application end to convert time in UTC format which is insane to say the least.

Are we also saying that anyone and everyone relying on Kibana for dashboard visualizations need to store data in UTC format for correct rendering and this is what organizations are doing worldwide? Somehow things are not adding up.!!

Is there any potential solution that Kibana still renders data properly as per the Timezone in which data was indexed. ?

8

Yes, that is correct.

Not that I am aware of.

9

Okay Christian. Thanks for the assistance.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.