ILM isn’t it working properly

Hey guys, I have a question about ILM...
I am working on devices logs, have Logstash pipeline for logs, and all of them work correctly. After Installing ILM policy, I have added the policy to my logstash’s elasticsearch output plugin. That all works, but not properly, I Installed 10mb max_size in Hot phase, but my index is in the rollingover in 16-17mb random sizes. Why ILM policy isn’t working properly?
Below you will find all needed information.

logstash output

output {
  if [type] == "syslog"{
    elasticsearch {
      hosts => ["localhost:9200"]
      ilm_rollover_alias => "syslog"
      ilm_pattern => "000001"
      ilm_policy => "syslog_policy"
    }
  }
}

ILM policy

"syslog_policy" : {
  "version" : 8,
  "modified_date" : "2020-10-25T15:22:55.418Z",
  "policy" : {
    "phases" : {
      "hot" : {
        "min_age" : "0ms",
        "actions" : {
          "rollover" : {
            "max_size" : "10mb"
          },
          "set_priority" : {
            "priority" : 200
          }
        }
      },
      "delete" : {
        "min_age" : "45m",
        "actions" : {
          "delete" : {
            "delete_searchable_snapshot" : true
          }
        }
      }
    }
  }
},

finally my indices

Is this to test given that 10MB shard sizes are very small and not generally recommended? Rollover by default checks at an interval (10 minutes?) which means that it will not cut exactly if you are indexing too fast with a low threshold.

1 Like

thanks bro, now I'm using g with the max_size 500mb and delete after 7 days, and working in right way

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.