Hello,
I am attempting to setup index lifecycle policies for monthly log indexes. Currently, I have Logstash indexing to multiple monthly indices based on log type.
The pattern is as follows:
index => "logtype1-%{+YYYY.MM}"
I understand I need to change that to:
index => "logtype1-index"
in order to get the rollover working properly with logstash writing to the alias.
I have read through the elastic documentation on rollovers, as well as this blog about this subject.
Here is an excerpt from my template:
PUT _template/logtype1
{
"index_patterns" : ["logtype1-*"],
"settings" : {
"number_of_shards" : "4",
"number_of_replicas" : "2",
"lifecycle.name" : "logs_lifecycle",
"lifecycle.rollover_alias" : "logtype1-index"
},
"aliases" : {
"logtype1-index" : {},
"logtype1-search": {}
If I create an index 'logtype1-2019-10-001' to start October
PUT /%3Clogtype1-%7Bnow%2Fd%7BYYYYMM%7D%7D-001%3E
{
"aliases": {
"logtype1-index": {
"is_write_index": true
}
}
}
I know it will be written to when logstash writes to 'logtype1-index'
I have the lifecycle policy to rollover when the index reaches a certain size.
What I want to have happen is when the first log event for the November comes in, a new monthly index is created, 'logtype1-2019-11-001', with the same lifecycle policy applied. I know this will happen when the rollover policy is triggered by size at some point in November, but I would like to perhaps add an OR to make this happen when the month changes.
Is there a way to accomplish this without manually creating a new index at 12:01 on 11/01 and setting it as a the write index for the 'logtype1-index' alias?
Any help is much appreciated, thank you.