I have spent the last 3 weeks off and on trying to get ILM to work. I have datastreams going but one client I am using (syslog-ng) does not yet know about datastreams and uses index
in the _bulk
rather than create
.
I found the documentation tantalizingly incomplete. It seems that the authors made assumption about what was obvious that were clearly wrong in my case. Common problem with technical documentation.
The curicial bit of information that I lacked was the link between the rollover alias and the alias set on the initial creation of the index with "is_write_index": true
and that this is what you use to write to the index.
Assuming that I have this all right! It seems to be working as expected.
So here is my howto for an sanity check and for anyone else as stupid as me who tries to set up ILM from scratch. I found this this link most useful but it still requires some joining of dots...
- create your index and bootstrap the write/rollover alias:
curl -X PUT "localhost:9200/timeseries-000001?pretty" -H 'Content-Type: application/json' -d'
{
"aliases": {
“timeseries-write": {
"is_write_index": true
}
}
}
I could not find any way of doing this in kibana but the following steps can be...
- create the lifecycle policy (say timeseries-policy)
- create the index template with these
settings
{
"index": {
"lifecycle": {
"name": “timeseries-policy",
"rollover_alias": “timeseries-write"
}
}
}
- start writing to timeseries-write
Improvement or corrections solitited