Elastic: ilm-history-3-000004 index created with wrong number_of_replicas according to template

Elastic Stack Elasticsearch
1

Elastic 7.11.1

Today, two of my elastic hosts (where there is 0 replicas), somehow created this index with number_of_replicas=1

yellow open ilm-history-3-000004

That is why it is in yellow state.

When I look index templates, then I see this:

image
image1111×443 46.7 KB

This ilm-history template has the following settings:

{
  "index": {
    "lifecycle": {
      "name": "ilm-history-ilm-policy"
    },
    "number_of_shards": "1",
    "auto_expand_replicas": "0-1",
    "number_of_replicas": "0"
  }
}

Here it says that "number_of_replicas": "0"
But look at the "Index patterns" from the picture above.

Template has: "ilm-history-5*"
But the index which was created: "ilm-history-3-000004"

So why ilm-history index was created with different name than its template..?
If there is no template for certain index, is the default value for number_of_replicas = 1 ? (if so, can I change this default?)

Raul

2

I believe auto_expand_replicas takes precedence here resulting in 1 replica.

3

But look at the index pattern.

"ilm-history-5*"
vs
"ilm-history-3-000004"

There is no way these could match.

Raul

4

Yes.

Effectively yes, that's what index templates are for.

5

Effectively yes, that's what index templates are for.

But the problem.

ilm-history <-- this is not user created index, this is internally managed index/template.

Am I supposed to create index template for internally managed indexes..?
(I don't think so)

Raul

6

Ah I see. I don't think a 7.11.1 cluster should create an index matching ilm-history-3-*, do you have a mix of different versions in your cluster?

7

Nop.
As I said. I have single host cluster with 7.11.1 version.

I don’t have any template for “ilm-history-5*” also.
For some reason template is for “ilm-history-3*”
And index was created starting with “ilm-history-5..”

Strange.

Raul

8

That's precisely the opposite of what you said in your original post: the index is called ilm-history-3-000004 but the template in the screenshot matches ilm-history-5*.

9

Yeah my bad.
That previous post was wrong.

This is the correct one:

I don’t have any template for “ilm-history-3*” also.
For some reason template is for “ilm-history-5*”
And index was created starting with “ilm-history-3..”

Raul

10

I have checked all my 7.11.1 clusters.
All of these have the same behaviour - at midnight clusters all went to "yellow" state due to this "ilm-history-3" index.

E: "yellow" state followed for all the clusters where I had single host in cluster. But the issue still stands for multi node clusters also, since it is not using its designated index template.

Raul

11

I think you could report this as a bug on Github, it certainly sounds like Elasticsearch isn't doing the right thing here.

12

Ok, I see this beahviour with 7.11.2 also.
I will create an issue.

Raul

13

:+1: thanks

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.