Index creates in different timezone other than UTC?

Elastic Stack Elasticsearch
1

Hello,

My ELK servers are located in Asia/Seoul timezone which is GMT+9:00. Elasticsearch creates indices on a daily basis, but the indices are created at 9AM (instead of 12AM) due to timezone difference.

I know Kibana corrects the timezone with browser's timzone but I need to run Curator at 1AM every day to move indices from hot to warm nodes. currently Curator is unable to execute allocation at 1AM because Logstash/Elasticsearch use 'yesterday' index until 9AM.

  • How can i create index with different timezone other than UTC?
  • To do this, should I change @timestamp? or is there anything i should do in Elasticsearch "_template"?
  • If I change @timestamp with 'Asiz/Seoul' timezone (maybe using Ruby?), below output for elasticsearch "YYMMdd" will work properly and index will be created at 12AM (probably 12:00:01)?

Please let me know.

thank you!

My Logstash output looks like below

output {
elasticsearch {
hosts => ["els-01:9200","els-02:9200"]
index => "index_%{+YYYYMMdd}"
}
}

Curator: action.yml

actions:
1:
action: allocation
description: "Apply shard allocation filtering rules to the specified indices"
options:
key: box_type
value: warm
allocation_type: require
wait_for_completion: true
timeout_override:
continue_if_exception: false
disable_action: false
filters:
- filtertype: pattern
kind: prefix
value: index_
- filtertype: age
source: name
direction: older
timestring: '%Y%m%d'
unit: days
unit_count: 1

2

As mentioned in your other thread, Elasticsearch uses UTC and so does Logstash, you cannot change that sorry.

3

You also should not change it. Just figure out what your offset from UTC is, and work accordingly with your local cron setup that way to compensate.

4

KST(UTC+9)

To do this with cronjob, i need to run cronjob (for curator) after 9AM after ‘today’ new indices are created. But there is too many traffics already. That’s why i wated to run cronjob at 1AM during not busy hours.

Thank you!

5

Fyi,
I guess i didn’t mention that i would store only 1day(‘today’) indices in hot node due to storage issue.

6

Sounds like a use case for rollover indices instead, then. Rollover the indices at midnight in your time zone, and migrate them to the warm node. There's no real need to keep one day's data in a single index, is there? The index patterns in Kibana don't see day boundaries.

7

I guess below options are potentials with limited disk storage (hot nodes) situation.

#1. add disk space and configure Elasticsearch config file to use new disk.
#2. indices creation period change so Logstash will call Elasticsearch to create HOURLY based indices. then, curator will run every 12hours to move previous 12hours indices from hot to warm node.

I am also considering to do above #1 option, but it can be happened sometime next month. When it is done, we can store more than 1day indices in hot node, but currently hot node disk size is not enough so as I posted here I would store only 1day 'today' indices.

If I will go for above #2 option, is there anything I need to change/configure on Elasticsearch side? I guess not but wanted to confirm.

Please advise.

Thank you!

8

Go with the rollover index API as Aaron suggested, as this is very flexible and allows you to cut indices at any point in time. Switching to hourly indices seems like a bad idea as having lots of small indices/shards can be very inefficient.

2 Likes
9

@Christian_Dahlqvist beat me to it. Rollover is a much better approach.

10

Thank you! I will try Rollover tomorrow :slight_smile:

11

you can specify a timezone in the elasticsearch date math... if that helps (even though I personally prefer to stay within UTC bounds here).

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.