Logstash ILM problem

Elastic Stack Logstash
1

I've created a life cycle policy and added it to an index-template.
In the policy I've enabled the hot phase which will rollover the index if max_docs is 50 or max_age is 1h.
Now the problem is that the new indices are created only when the document count exceeds 150.

Below is my life cycle policy

"logstash-policy" : {
"version" : 4,
"modified_date" : "2019-09-17T12:59:47.001Z",
"policy" : {
"phases" : {
"hot" : {
"min_age" : "0ms",
"actions" : {
"rollover" : {
"max_docs" : "50",
"max_age" : "1h"
}
}
}
}
}
}

Index settings:

"index": {
  "lifecycle": {
    "name": "logstash-policy",
    "rollover_alias": "logstash",
    "indexing_complete": "true"
  },
  "number_of_shards": "5",
  "provided_name": "<logstash-000004>",
  "creation_date": "1568725169438",
  "number_of_replicas": "1",
  "uuid": "YJOVjP2UT1etiVF-dYazeA",
  "version": {
    "created": "6060199"
  }
}

}

2

ILM only check limits every 10 minutes or so which is generally fine for real life scenarios but can trip up testing.

3

As Christian mentioned, you can check out the setting (indices.lifecycle.poll_interval) here: https://www.elastic.co/guide/en/elasticsearch/reference/7.4/ilm-settings.html

4

Thanks @dakrone and @Christian_Dahlqvist I'll give it a try.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.