In our Elasticsearch we use Datastreams which split our data to several streams like bellow.
So all the Datastreams have the same template and ILM policy. Since one of them has increased a lot I want to apply different ILM policy so I can delete data.
How can I change the ILM policy from only one of those templates, for example prod-data-metric2-ds?
Our index_template:
Index pattern
prod-data-*
Priority
200
Component templates
None
Data stream
Yes
Version
None
If you are using component templates, you will need a new template with a higher priority number.
This new template should match only the desired data-stream, for example prod-data-metric2-*.
If you are using legacy templates you will also need a new template matching only the desired data-stream, but in this case you will need it to have a higher order value.
I am using Index Template and it is configured to have
index pattern:
prod-data-*
ILM policy:
prod_data_metrics_policy
If I create a new index template with
index pattern:
prod-data-metric2-*
ILM policy:
prod_data_metric2_policy
won't I have conflicts with duplicate datastreams with different templates?
Since prod-data-metric2-ds is included in both prod-data-metric2-* and prod-data-*
To solve the errors I removed the ILM policy from all the indices, recreate the policy and applied the new policy to the latest index.
Now the rollover works correctly with the data ingestion.
Note that when I tried to remove the policy targeting the Datastream,
POST prod-data-metric2-ds/_ilm/remove
didn't remove it from all the indices. I needed to remove it manually from each remaining index.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.