Elastic UTC time

Elastic Stack Elasticsearch
1

I have a small issue with date time stamps. We currently pass date of birth to Elastic as

YYYY-MM-DDT00:00:00 for example 2019-08-08T00:00:00

However, when in Kibana this is interpreted as UTC (looks like) because the date in kibana shows up as next day. 2019-08-09 so everyones DOB appears to be 1 day later. How to set Elasticsearch to understand all times in EST or not have a timezone at all and just default to browser time (without conversion)?

2

Bump

3

Bump

4

Elasticsearch doesn't have a native concept of a "pure" date, only of instants in time. Given that, I think that it is reasonable to represent a pure date as the instant of midnight UTC on that date. If you use a different timezone then you will encounter problems as some "dates" will not be a whole number of days apart, because of occasions when the clocks go forwards or back for daylight savings.

Formatting a date as an instant without an explicit timezone is probably a bad idea, because something somewhere will have to guess what timezone to use and you may get inconsistent results if those guesses are inconsistent. Always specify the timezone, and in this case I recommend UTC as I mentioned above.

When you run an aggregation involving times, you can tell Elasticsearch what timezone to use. It defaults to UTC according to those docs.

I do not, however, know how to get Kibana to interpret these dates as you want in a visualisation. It's probably best to ask on the Kibana forum for help with that.

5

The issue is not Kibana. But I get what you are saying. If there is no concept of just a date, then fields lile birth date wont make sense. You can send 00:00:00 but then any age calculations you have would get messed up. I think for me for the entries that do have the time info, the easiest way would be to either

  1. Specify timezone with each elastic send
  2. Always convert to UTC before sending.
6

Turns out that this is a bug and I have to manually edit the ingest pipeline for system. While configuring Elastic is ignoring whats set in the config file. I solved it by deleting the pipeline and recreating it.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.