I think I've run myself in circles too many times trying to iron out index lifecycle management for my test environment. Hopefully someone here can nudge me in the right direction.
Currently I'm using Logstash to put syslog events into an index that is named per day as syslog-1.0-%{+YYYY.MM.dd}. This is working as expected creating a new index each day. I have 3 nodes with the node.attr.data=hot attribute and one with node.attr.data=warm. I've also enabled ilm in Logstash via ilm_enabled => true (just did this today). My index template has the following set:
"lifecycle": {
"name": "syslog-1.0",
"rollover_alias": "syslog-1.0"
},
What I'm hoping to do for my test is move an index after 2 days to a warm storage node. Then after 30 days delete the index. I've been reading Implementing Hot-Warm-Cold in Elasticsearch with Index Lifecycle Management | Elastic Blog and other articles, but I can't get this to work. With the Beats indices, it seems to append a number which I assume is what I should see on my index as well. I have the following error reported in ILM:
illegal_argument_exception: index name [syslog-1.0-2019.09.26] does not match pattern '^.*-\d+$'
Is this because the syslog-1.0-2019.09.26 index was created before I enabled ILM in Logstash? I noticed even the Beats indices with the -00000# suffix don't appear to be getting any inserts. They don't have lifecycle errors though. I did include "is_write_index" : true when I updated the aliases.
alias index filter routing.index routing.search
syslog-1.0 syslog-1.0-2019.09.26 - - -
.security .security-7 - - -
.kibana .kibana_1 - - -
auditbeat-7.3.1 auditbeat-7.3.1-2019.09.26-000002 - - -
filebeat-7.3.1 filebeat-7.3.1-2019.09.26-000002 - - -
winlogbeat-7.3.1 winlogbeat-7.3.1-2019.09.26-000002 - - -
Any guidance on this would be appreciated.