Resource metricbeat-7.17.1 exists, but it is not an alias

[Evgeny_Barykin]

Hello!
I have an issue with connecting second cluster to Elasticsearch. I changed dynamic mappings and added best_compression to the settings, but now I get an error every time I try to connect new beats to it.
If I leave the field from the automatically created index pattern:

"lifecycle": {
      "name": "metricbeat",
      "rollover_alias": "metricbeat-7.17.1"
    },

It will give me an error:

[stephenb (Stephen Brown)]

Take a look at this post see if it helps

Metricbeat not creating Index correctly

So when you did that you deleted the write alias metricbeat-7.13.4 as well besides the actual indexes with the data in it. You don't want to do that. When you initially ran setup the metricbeat-7.13.4 write alias was created it looked something like this you can read about them here. They are part of ILM. running setup will re-create it. "metricbeat-7.13.4-2021.10.07-000001" : { "aliases" : { "metricbeat-7.13.4" : { "is_write_index" : true } When working properly, …

[Evgeny_Barykin]

Thank you, that is helpful. So I need to add alias to the index template to solve the problem, right?

[stephenb (Stephen Brown)]

Yes and you will need to clean up the current index as it is not an alias it is a real index.

If you are using metricbeat all this is done for you if you just run setup per the quick start.

Macro Steps

Stop all metricbeats
Clean Up the bad index
Run metricbeat setup -e
Start All metricbeats

[Evgeny_Barykin]

Can you give me an example of how do I add alias to the index template, please?
Not sure how should I run setup, I am using helm charts to deploy metricbeat.

[stephenb (Stephen Brown)]

I think the alias is probably in the template you can check by just looking here

The problem at some point you deleted the alias and then started one of the metricbeat is it started to write to metricbeat-7.17.1 which the write alias did not exist and so it started writing to a regular index named metricbeat-7.17.1

So to fix...

Stop all your metricbeats.

Clean up / delete the bad index
DELETE metricbeat-7.17.1 from the Kibana - Dev Console and YES you will lose data

Re-create the write alias

PUT metricbeat-7.17.1-000001
{
  "aliases": {
    "metricbeat-7.17.1": {
      "is_write_index": true
    }
  }
}

Or if you want the date component you need to do a little extra trick see here

# PUT /metricbeat-7.17.1-{now/d}-000001
PUT /%3Cmetricbeat-7.17.1-%7Bnow%2Fd%7D-000001%3E?pretty
{
  "aliases": {
    "metricbeat-7.17.1": {
      "is_write_index": true
    }
  }
}

Which will result in

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "metricbeat-7.17.1-2022.06.02-000001"
}

[Evgeny_Barykin]

Thank you!