# How logstash create indices based on date - wrong date

**URL:** <https://discuss.elastic.co/t/how-logstash-create-indices-based-on-date-wrong-date/253826>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [October 30, 2020, 2:05pm UTC](https://discuss.elastic.co/t/how-logstash-create-indices-based-on-date-wrong-date/253826 "2020-10-30T14:05:17Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dorinand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dorinand/32/70521_2.png) [@dorinand](https://discuss.elastic.co/u/dorinand)\
**Post date:** [October 30, 2020, 2:05pm UTC](https://discuss.elastic.co/t/how-logstash-create-indices-based-on-date-wrong-date/253826/1 "2020-10-30T14:05:17Z")

</div>

I am running ELK with filebeat in kubernetes. Filebeat is harvesting logs and sending it to logstash. This is my logstash filter:

```auto
      if [kubernetes][annotations][elastic_index] {
        mutate {
          add_field => { "[@metadata][es-index]" => "%{[kubernetes][annotations][elastic_index]}" }
        }
      } else if [kubernetes][pod][name] {
        mutate {
          add_field => { "[@metadata][es-index]" => "default-%{[kubernetes][pod][name]}-%{+YYYY.MM.dd}" }
        }
      }
    }

```

If my pod has annotation `elastic_index`, it will write into one rollover index, otherwise, it should create default index with date. I am using policies for default indices:

```auto
{
  "default" : {
    "version" : 9,
    "modified_date" : "2020-10-20T09:50:31.399Z",
    "policy" : {
      "phases" : {
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "set_priority" : {
              "priority" : null
            }
          }
        },
        "delete" : {
          "min_age" : "6d",
          "actions" : {
            "delete" : {
              "delete_searchable_snapshot" : true
            }
          }
        }
      }
    }
  }
}

```

So if I am understand correctly, the index will get into `hot` stage immediately after creation. Then, if index is in `hot` for 6 days, it would move to `delete` stage -\> index should be deleted.

Now, I can see, there is app with index older than 15 days (even older than 22 days):

```auto
default-myapp-nmnj2-2020.10.08

```

Today is `2020.10.30`. When I check this index in `Kibana` -\> `Index Management` section, I can see that index with timestamp `2020.10.08` has cration time `2020-10-29 13:43:19`.

Because of this, I cant make older indices readonly, because indices, e.g. myapp, has indices created 2020-10-19 but they are 22 days old.

Questions:

1. Do logstash create date based on date in filebeat document or from logstash local time?
2. Do I understand coreectly the index movement between stages, in this case from `hot` to `delete`?
3. Why is there different time creation? What do I missing? How to configure it properly?

I am running everything in kubernetes, deployed via [helm](https://github.com/elastic/helm-charts)  
Version: **7.8.1**

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 30, 2020, 2:36pm UTC](https://discuss.elastic.co/t/how-logstash-create-indices-based-on-date-wrong-date/253826/2 "2020-10-30T14:36:21Z")

</div>

> [@dorinand](#):
>
> `add_field => { "[@metadata][es-index]" => "default-%{[kubernetes][pod][name]}-%{+YYYY.MM.dd}" }`

That time reference is constructed using the @timestamp value from the event. If you process an event that has a timestamp from a month ago you will get an index name that refers to a month ago.

---

<div class="post-metadata">

**Author:** ![dorinand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dorinand/32/70521_2.png) [@dorinand](https://discuss.elastic.co/u/dorinand)\
**Post date:** [October 30, 2020, 2:59pm UTC](https://discuss.elastic.co/t/how-logstash-create-indices-based-on-date-wrong-date/253826/3 "2020-10-30T14:59:49Z")

</div>

Hi, thank you for fast reply.

I will investigate why I am receiving events that are older than 22 days. Is there a way how to use actual timestamp instead of event timestamp?

What about my `questions 2`, am I understand correctly the ILM logic in this case??

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 27, 2020, 2:59pm UTC](https://discuss.elastic.co/t/how-logstash-create-indices-based-on-date-wrong-date/253826/4 "2020-11-27T14:59:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.