Hi all!
I'm trying to implement ILM time-based indices in my elasticsearch cluster.
Right now, I have custom indexes. My configuration is as follows:
BEATS(metricbeat, filebeat) ---> Logstash ---> Elasticsearch.
I have somehow managed to do this with the configuration below:
In logstash:
elasticsearch {
hosts => ["http://10.0.1.11:9200"]
index => "test-logs-%{+YYYYMMdd}-000001"
ilm_rollover_alias => "test-logs"
ilm_pattern => "000001"
ilm_policy => "policy"
In elasticsearch I've followed the documentation and issued the following commands:
PUT /_ilm/policy/policy
{"policy": {"phases": {"hot": {"actions": {"rollover": {"max_size": "100MB", "max_docs": 10000}}}}}}PUT _template/logs
{"index_patterns":["test-logs-*"],"settings":{"number_of_shards":1,"number_of_replicas":1 ,"index.lifecycle.name": "policy","index.lifecycle.rollover_alias": "test-logs"}}PUT %3Ctest-logs-%7Bnow%2Fs%7Byyyy-MM-dd%7D%7D-000001%3E
{"aliases": {"test-logs": {"is_write_index": true}}}
As far as I can tell, following this github post, should automatically rename indices as days or months or years change but, although I've changed the date and logs were sent as they were from another month(and also waiting till the next day didn't help), this hasn't changed my index name from test-logs-20200304-0000010 to test-logs-20200305-000001 as I was expecting but instead it incremented it to test-logs-20200304-0000011.
What am I missing? Is there additional configuration to be made? Anyone any thoughts?
Thank you,
Radu