I am wondering how I can set the "logstash" alias to rollover the daily index if it matches the ILM policy rules AND also apply automatically the logstash alias on the new daily indices automatically.
For example, if I set my "logstash-policy" ILM to rollover when the daily index reaches 1GB in size, I'd like it to rotate to "logstash-2021.03.17-000002" for today, as an example. I would also lie to have tomorrow's index "logstash-2021.03.18" to automatically get the "logstash" alias attached to it when it gets created and also rollover automatically as per the "Logstash-policy" ILM.
Can you help me with steps to modify my infrastructure to set those automatically.
If you are using rollover I think it is best to not have date in the underlying index names and just rely on the generated sequence numbers as ILM does not need date in the underlying index names to work. I would recommend setting up a new rollover index with a write alias of e.g. logstash. You then change your fluentd config (I assume this must be possible) to write to the logstash alias instead of generating index names with date in them.
I would like to add that most of my other issues trying to create an index name that started with fluentd-* were caused by not having permission to create that new index with the FLUENT_ELASTICSEARCH_USER user. Check that first.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.