ES Migrated from 7.7.0 to 7.15.0 - all application 'logs-*' legacy templates superseded by new managed logs index

Greetings!

We have a simple Elasticsearch/Kibana cloud deployment primarily used for application logs against which we generate graphs and dashboard. Logs are shipped directly from applications directly to ES search, i.e. not via APM.

Yesterday I upgraded our 7.7.0 based Elasticsearch stack to 7.15.0 and all appeared well (graphs continued updating based on application logs) until the lifecycle policies kicked in last night. Since then, none of our graphs are updating.

All our application logs have separate index templates per application and they all prefixed with logs-*.

Taking a look at the Data -> Index Management -> Index Templates section, I can see that all our previous application log index templates are now under the Legacy section and it appears that a logs managed index template, with a index pattern of logs-*-* is now being applied to all our application logs.

I've tried setting the logs index template priority to 0 (from 100) but this makes no difference. I am also not able to delete the logs index template since data streams are constantly being referenced by it. Reading the docs, legacy indexe templates are superseded by the new index templates, the logs index template in my case since all our application logs have a logs-* prefix.

How to I migrate my 'legacy' index templates to the new templates? We have a few dozen of these with several weeks worth of graphs and dashboards and I'd hate to recreate these by hand.

How do I delete the logs new index template so that application logs fallback to being indexes by the existing legacy indexes?

I searched through the docs but couldn't find any relevant sections on migrating legacy indexes, especially if there are a large number of them.

Thank you for your help.

According to the breaking changes in 7.9 document, setting stack.templates.enabled might help my issue of stream being created and breaking all our application logging.

To disable all built-in index and component templates, set stack.templates.enabled to false in elasticsearch.yml before start up. If the templates already exist, this setting ensures Elasticsearch does not recreate the built-in templates after deletion.

I'm not able to set this setting on my elastic cloud page:

Which fails to save:

How to I set stack.templates.enabled on my Elasticsearch server which is hosted on cloud.elastic.co?

I'd encourage you to open a Support request with our team for this one.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.