# Create daily indices based on local timezone

**URL:** https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330
**Category:** Logstash
**Created:** [October 20, 2019, 2:43am UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330 "2019-10-20T02:43:47Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![tomr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomr/32/48260_2.png) [@tomr](https://discuss.elastic.co/u/tomr)
#### Post date: [October 20, 2019, 2:43am UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/1 "2019-10-20T02:43:47Z")

</div>

While trying to figure out how to make indices rollover at midnight local time, I found a bunch of people asking how to do it but no clear solution.

The ruby code below generates a `YYYY.MM.dd` formatted string based on `@timestamp` in the specified timezone (in my case Australia/Melbourne). The offset is calculated in a separate step to be transparent with daylight savings.

```auto
  ruby {
    code => "
      require 'tzinfo'
      tz = 'Australia/Melbourne'
      offset = TZInfo::Timezone.get(tz).current_period.utc_total_offset / (60*60)
      event.set('[@metadata][index_day]', event.get('@timestamp').time.localtime('+' + offset.to_s + ':00').strftime('%Y.%m.%d'))
      "
  }

```

Then `-%{[@metadata][index_day]}` can be used in elasticsearch output instead of %`{+YYYY.MM.dd}`.

I had a bit of a look at the rollover API, and I suspect xpack's ILM might take care of this use case, but I wanted something simple that Just Works with Logstash OSS.

Hope it helps someone!

---

<div class="post-metadata">

### Author: ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)
#### Post date: [October 20, 2019, 9:22am UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/2 "2019-10-20T09:22:05Z")

</div>

You can use date filter with logstash to grand any event date from source and map it to @timestamp, if no source field with the date logstash will populate the @timestamp with the system date

```
## Convert event date time into @timestamp
date { 
		match => ["TimeStamp", "yyyyMMddHHmmss"]
		timezone => "UTC"
		target => "@timestamp"
		}

```

Then in the output, append any date format to your index name

```
   elasticsearch {
    hosts => "http://localhost:9200"
    index => "index-name-%{+YYYY-MM-dd}"
}
```

---

<div class="post-metadata">

### Author: ![tomr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomr/32/48260_2.png) [@tomr](https://discuss.elastic.co/u/tomr)
#### Post date: [October 20, 2019, 6:23pm UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/3 "2019-10-20T18:23:47Z")

</div>

I think you've missed the point, which is: to ensure indices rollover at midnight localtime in a non-UTC timezone, including during summer time.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 20, 2019, 9:20pm UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/4 "2019-10-20T21:20:07Z")

</div>

If your servers clock is set to local time what is wrong with

```
    event.set("[@metadata][indexName]", Time.now.strftime("%Y.%m.%d"))

```

local time -- check. daylight savings handled -- check.

---

<div class="post-metadata">

### Author: ![tomr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomr/32/48260_2.png) [@tomr](https://discuss.elastic.co/u/tomr)
#### Post date: [October 20, 2019, 10:12pm UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/5 "2019-10-20T22:12:08Z")

</div>

@Badger that would be fine under those circumstances, but the server's time is UTC. Which I guess would be a pretty common scenario.

Yours in definitely a good point to have in this thread for posterity though!

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 20, 2019, 10:55pm UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/6 "2019-10-20T22:55:14Z")

</div>

You could parse it once telling the date filter it is UTC, then parse again as local time, but that loses the DST handling

```
    date { match => ["message", "YYYY-MM-dd HH:mm"] }
    mutate { add_field => { "[@metadata][indexName]" => "%{+YYYY.MM.dd}" } }
    date { match => ["message", "YYYY-MM-dd HH:mm"] timezone => "Etc/GMT+4" }

```

I think you have to parse it as the local timezone to get the right DST handling, so the question you are really asking is how to get ruby (not ruby on rails) to display a timestamp in a non-local timezone. I do not know the answer to that.

---

<div class="post-metadata">

### Author: ![tomr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomr/32/48260_2.png) [@tomr](https://discuss.elastic.co/u/tomr)
#### Post date: [October 20, 2019, 11:21pm UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/7 "2019-10-20T23:21:28Z")

</div>

My post was a solution to this problem, rather than a question. Is there a problem with it I'm not seeing? (apart from not handling sub-hour TZ offsets in its present form)

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 20, 2019, 11:23pm UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/8 "2019-10-20T23:23:22Z")

</div>

> [@tomr](#):
>
> Is there a problem with it I'm not seeing?

No, I misunderstood the question. Thanks for posting a solution.

---

<div class="post-metadata">

### Author: ![tomr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomr/32/48260_2.png) [@tomr](https://discuss.elastic.co/u/tomr)
#### Post date: [October 29, 2019, 1:08am UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/9 "2019-10-29T01:08:05Z")

</div>

The ruby code initially used `utc_offset` (which doesn't take into account DST) rather than `utc_total_offset` (which does).

I've fixed the example in the OP.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 26, 2019, 1:08am UTC](https://discuss.elastic.co/t/create-daily-indices-based-on-local-timezone/204330/10 "2019-11-26T01:08:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
