How to change index rotation timezone for Elasticsearch 8.6 for UTC to localtimezone

Elastic Stack Elasticsearch
1

Hi everyone,

I am facing an issue while running queries on Elasticsearch, we are unable to fetch data between 12:00 AM to 05:30 AM, issue no data can be retrived from index between given time.

Upon further investigation it has been discovered that issue occurs mainly because timezone difference between UTC and local timezone (+05:30 HRS).

we run the queries on specific index created for that particular day (e.g my_index-YYYY-mm-dd), as we use +05:30 HRS timezone, date changes around 5 hours 30 minutes earlier on our servers whereas on Elasticsearch cluster data keep on populating in previous day's index.

I want my Elasticsearch cluster to rotate the index based on localtime zone this will allow us to run queries more efficiently.

Also I would like to know if we in any-way change timezone on Elasticsearch Cluster will it effect other components of ELK stack as mentioned below?

  1. Logstash
  2. Kibana
  3. Filebeat

(if yes!, please guide me to document material where I can read more about it.)

Thank you.

2

That's not possible, both Logstash and Elasticsearch use date time in UTC and this can not be changed.

If your Logstash has an output like index-%{+YYYY-MM-dd} it will use the value of the @timestamp field to extract the pattern YYYY-MM-dd, and this will always be in UTC.

How are you parsing the field @timestamp? It is getting the value of the date in your logs correctly, with the timezone offset?

Please share how you are collecting logs, how you are parsing them and some sample messages.

3

How are you parsing the field @timestamp ? It is getting the value of the date in your logs correctly, with the timezone offset?

Yes @timestamp field get populated which reflects localtime on Kibana interface.

Still Not sure if it's Kibana which is displaying the field as per UTC.

We are using filebeat to push access logs into cluster which are parced via Logstash.

Do we have any way to check what timezone is getting recorded on these fields?

Please share how you are collecting logs, how you are parsing them and some sample messages.

what sample message are you expecting?

Raw log or json out from kibana? or some screenshot of sample document?

4

But how are you populating it?

Kibana will convert all Date fields from UTC to the Browser timezone.

Logstash date filter will always create a UTC field, if your date string does not have any timezone information, Logstash will assume that this is already in UTC and this can lead to some issues.

You need to share your configuration.

You need to share the raw logs and your logstash configuration.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.