I would like to setup the index lifecycle by using index template to create a new index (test_index-yyyy.MM.dd) every 30 days
My log will send form filebeat --> logstash --> Elasticsearch.
-
configuration a policy (test_policy) --> enable rollover-->set maximum age:30 days
-
create index templates (test_template)--> index pattern (test_index*) -->setting (index.lifecycle.name= test_policy) (index.lifecycle.rollover_alias=test_index)
component, mappings and aliases no set.
3.Bootstrap the initial time series index with a write index alias
PUT test_index_2024.06.05-000001
{
"aliases": {
"test_index": {
"is_write_index": true
}
}
}
- configure filebeat.yml to set the tag of the log
- type: log
enabled: true
paths:
- /etc/monitor/*.log
tags: ["test_index"]
restart filebeat serivce
- configure /etc/logstash/conf.d/test_index.conf
output {
if "test_index" in [tags] {
elasticsearch {
hosts => ["https://x.x.x.x:9200"]
user => "filebeat_user"
password => "filebeat_password"
ssl_certificate_authorities => "/etc/certs/xxxx.crt"
ilm_rollover_alias => "test_index"
ilm_pattern => "{now/d}-000001"
}
}
Restart logstash service
- When the index rollover i find one more template created automatically (test_index), in additional to my test_template,
which pointed to the index pattern (test_index*)
settings
{
"index": {
"lifecycle": {
"name": "logstash-policy",
"rollover_alias": "test_index"
},
"mapping": {
"total_fields": {
"limit": "10000"
}
},
"refresh_interval": "5s"
}
}
With some mappings too.
Is there anything wrong to my steps??