ILM and max size with logstash ELK 7.10.0

Elastic Stack Elasticsearch
1

Hello

I am using ELK stack 7.10.0 and I have an issue with ilm and max_size

    {
      "indices" : {
        "logstash_netflow-2020.11.26" : {
          "index" : "logstash_netflow-2020.11.26",
          "managed" : true,
          "policy" : "soc_policy",
          "lifecycle_date" : "2020-11-26T11:03:45.871Z",
          "lifecycle_date_millis" : 1606388625871,
          "age" : "6.21m",
          "phase" : "hot",
          "phase_time" : "2020-11-26T11:03:48.194Z",
          "phase_time_millis" : 1606388628194,
          "action" : "rollover",
          "action_time" : "2020-11-26T11:09:44.461Z",
          "action_time_millis" : 1606388984461,
          "step" : "check-rollover-ready",
          "step_time" : "2020-11-26T11:09:44.461Z",
          "step_time_millis" : 1606388984461,
          "phase_execution" : {
            "policy" : "soc_policy",
            "phase_definition" : {
              "min_age" : "0ms",
              "actions" : {
                "rollover" : {
                  "max_size" : "7gb",
                  "max_age" : "1d"
                }
              }
            },
            "version" : 17,
            "modified_date" : "2020-11-26T10:52:50.016Z",
            "modified_date_in_millis" : 1606387970016
          }
        }
      }
    }

As you can see the policy rules are 7gb or 1d

Unfortunately my index runs beyond this limit

Maybe this is related but I have sometimes

"llegal_argument_exception: index.lifecycle.rollover_alias [logstash_netflow] does not point to index [logstash_netflow-2020.11.26]" in kibana, but this message disappear after some seconds and failed step is none

My index is well created every day by logstash

I guess I missed something but what ?

2

I saw that -0000N (where N is a digit) is required for rollover

{
  "indices" : {
    "logstash_netflow-2020.11.26-000005" : {
      "index" : "logstash_netflow-2020.11.26-000005",
      "managed" : true,
      "policy" : "soc_policy",
      "lifecycle_date" : "2020-11-26T10:59:41.504Z",
      "lifecycle_date_millis" : 1606388381504,
      "age" : "1.16h",
      "phase" : "hot",
      "phase_time" : "2020-11-26T10:59:43.239Z",
      "phase_time_millis" : 1606388383239,
      "action" : "rollover",
      "action_time" : "2020-11-26T11:09:44.169Z",
      "action_time_millis" : 1606388984169,
      "step" : "check-rollover-ready",
      "step_time" : "2020-11-26T11:09:44.169Z",
      "step_time_millis" : 1606388984169,
      "phase_execution" : {
        "policy" : "soc_policy",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "7gb",
              "max_age" : "1d"
            }
          }
        },
        "version" : 17,
        "modified_date" : "2020-11-26T10:52:50.016Z",
        "modified_date_in_millis" : 1606387970016
      }
    }
  }
}

logstash_netflow-2020.11.26-000005 is now beyond 17 Go

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.