Hey everyone!
I have a super basic Elastic Stack set up in docker on my personal server. I am using Logstash to gather the logs of my non-elastic containers. The stack works great and has for years. The problem I'm running into is that I am unable to get ILM working. The elastic and logstash containers are bare-bones - pretty much all default settings files. I am managing the ILM stuff in Kibana on the same server. If I'm going about this totally the wrong way, please let me know what a more appropriate approach would be.
This is the pertinent info from error I'm getting:
"message": "policy [30-days-default] for index [logstash] failed on step [{\"phase\":\"hot\",\"action\":\"rollover\",\"name\":\"check-rollover-ready\"}]. Moving to ERROR step",
"error.stack_trace": "java.lang.IllegalArgumentException: index name [logstash] does not match pattern '^.*-\\d+$' ... etc.
By default, logstash created an index named logstash that is added in Kibana. This is the logstash output plugin I'm using:
output {
elasticsearch { hosts => ["elasticsearch:9200"] }
}
I created an index template for it called logstash_index_template for the pattern logstash*:
{
"template": {
"settings": {
"index": {
"routing": {
"allocation": {
"include": {
"_tier_preference": "data_content"
}
}
}
}
},
"aliases": {
"logstash_alias": {}
},
"mappings": {}
}
}
I added the alias "logstash_alias" to the index to match the template.
I am using the 30-days-default lifecycle policy with the goal of deleting documents older than 30 days. I applied this policy to the index.
So as far as I can tell, the rollover is failing because my index isn't using the indexname-xx format. I am not sure how to make this happen. I tried it with the alias being logstash_alias-0001 but it still is targeting the index name itself. I tried renaming the index to logstash-0001 but the original index just gets created again whenever a new document is written.
Is this a valid approach for my goal? I appreciate any help I can get on this.
Thank you!