Date in index name doesn't change on rollover on the next day

Elastic Stack Elasticsearch
1

For some reason, although the index naming convention is set in the Filebeat clients' configuration, the timestamp of the date isn't change when the index is rolled over on the next day.

For example let's say I had index "index-2020.04.02-000001" and "index-2020.04.02-000002" already created, the day following it created "index-2020.04.02-000003" instead of "index-2020.04.03-000001".

What do I miss?

2

ES will automatically do the date math only if the initial index was created using date math, you can see what the index was created by with:

GET /<your-index>

And then look for the provided_name key, what does it say?

3

The provided name for provided_name give the same as the index name, i.e for the index "index-2020.04.02-000001" it is index-2020.04.02-000001.

4

Okay, in that case it looks like the first index was not created with date math, rollover will only perpetuate the date math in the name if the initial index was created with date math (it would be named something like <index-{now/d}-000001> if it were created with date math.

In order to migrate it, it depends on what version of ES you're running, what version are you using? (Alternatively, if you're just testing, you can start over, but I understand that's not feasible with live data)

5

Well, this has probably happened, after some re-indexing. Is it possible to create an index with a REST call using date math?

I'm currently on 7.6

6

@dakrone How do I migrate to an index with date math in the name on 7.6?

7

ILM is built around the date in the index name being the date the first index in the policy was created. There's currently no way to change that to have subsequent index names created with the date it was then created on.
If it's something you would like, it'd be worth creating a feature request so we can track it :slight_smile:

If the client you are using can translate that into an actual index name, yes. Elasticsearch cannot calculate that for you via the REST API.

8

So you are basically saying that the only way would be delete all the productive indices and let the Filebeat client generate the next index with the date math, correct?

That's not very realistic for me, as I don't want to delete all the data, considering those are productive data.

9

What's the driver behind wanting date based indices?

10

The ILM is configured so, it rollover each day or when it reaches 50GB.

We use the stack to be able to search our logs. The driver would be to see in one go which indices contains the logging data of specific day.

11

It has been a pattern we have used for such a long time now, so it makes sense. But ILM removes that requirement, and Kibana now uses the field_caps API to get that info from an index when it needs to.

12

I do not think Kibana has used field caps in quite some time. A while back querying indices with no matches was made much more efficient so I think it now just queries all matching indices.

13

Date-based indices are much easier to deal with as an administrator when doing things like trying to figure out what index to restore from a snapshot, having index-2018-05-21-000234 gives a better idea of when the index was created and what data it has than index-000234.

14

So where does that leave me? As I said dakrone hinted that there is way to migrate my index with the desired behaviour, depending on the version of the ES stack I'm using. (btw, we recently updated to 7.6.2). Refer to the quote below to see what I mean. And as I am productive, I'm loathe to throw simply all my data away.

15

@EldrosKandar here are some steps for it:

  1. Stop ILM
POST /_ilm/stop
  1. Create the new index with date math (don't change the alias yet)

(replace 999 with the number needed, whichever is the next number in your sequence):

PUT %3Cindex-%7Bnow%2Fd%7D-000999%3E
  1. Update the aliases to correct the write_alias pointer

(replace the index-2020.07.30-000999 index name with the index created in step 2 and replace index-2020.06.19-000NNN with the latest index that exists, replace youralias with your alias name):

POST /_aliases
{
  "actions" : [
    {
      "add" : {
        "index" : "index-2020.07.30-000999",
        "alias" : "youralias",
        "is_write_index" : true
      }
    },
    {
      "add" : {
        "index" : "index-2020.06.19-000NNN",
        "alias" : "youralias",
        "is_write_index" : false
      }
    }
  ]
}
  1. Update the first index to say that indexing is complete

(skip the rollover since you already created the next index), make sure to replace the NNN with the correct number!

PUT /logstash-atl_fdmvcc-2019.06.19-000NNN/_settings
{
  "index.lifecycle.indexing_complete": true
}
  1. Start ILM
POST /_ilm/start
  1. Verify with ILM explain that everything looks copacetic
3 Likes
16

Thanks a lot! I've applied it today, now let's see if it works when it creates the next index.

This ought to be better documented, though. Would it be a case where I should open an issue?

1 Like
17

I had to do a re-index yesterday, due to another issue I had, and I adapter your index creation call formula in the re-index call. Last night a new index was created and it took the newest date.

Thanks @dakrone for your support and your patience with me.

1 Like
18

Great to hear!
If you would like to open an issue here - https://github.com/elastic/elasticsearch/issues/new/choose

1 Like
19

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.