Hello,
I have a hot-warm architecture and a couple of ILM that will move the index from the hot nodes to the warm nodes based on the age.
I have an index that is created daily and is moved to the warm nodes after 7 days, to do this I created a Lifecycle Policy using the Kibana UI where I set the number of days and I checked the force merge
option, I didn't checked the read only
option.
It is possible that for some cases I can get documents that need to be indexed in these indices even after they were moved to the warm
nodes, that's why I didn't checked the read only
option, but I'm getting some FORBIDDEN/8/index write (api)
errors in logstash and when I looked at the index they were set to read only ("index.blocks.write": "true"
).
I'm using version 7.12.1 currently, an update is planned but no date defined yet.
Is this a bug? What could be setting the index to read only
?
This is my policy:
{
"redacted-ilm-policy" : {
"version" : 5,
"modified_date" : "2021-09-19T14:48:23.390Z",
"policy" : {
"phases" : {
"hot" : {
"min_age" : "0ms",
"actions" : {
"set_priority" : {
"priority" : 100
}
}
},
"delete" : {
"min_age" : "61d",
"actions" : {
"delete" : {
"delete_searchable_snapshot" : true
}
}
},
"warm" : {
"min_age" : "7d",
"actions" : {
"allocate" : {
"include" : { },
"exclude" : { },
"require" : {
"node_type" : "warm"
}
},
"forcemerge" : {
"max_num_segments" : 1
},
"set_priority" : {
"priority" : 50
}
}
}
}
}
}
}
The modified date is recent because I saved it again but no changes were made.
These are the settings of an index in the warm phase with the writes blocked.
{
"index.blocks.read_only_allow_delete": "false",
"index.write.wait_for_active_shards": "1",
"index.mapping.ignore_malformed": "true",
"index.refresh_interval": "30s",
"index.blocks.write": "true",
"index.query.default_field": [
"message"
],
"index.priority": "50",
"index.number_of_replicas": "0",
"index.lifecycle.name": "redacted-ilm-policy",
"index.routing.allocation.require.node_type": "warm"
}
I was able to temporarily solve this issue by manually setting the index.blocks.write
to null in every index, but I expect the ILM to works as defined so I would not need to create a crontab just to set this.