Too many small segments getting created

We are using most of the default settings and increased the refresh interval to 30 sec.
We are observing too many small segments are getting created. Can you please let us know what might be issue for creating too many small segments. Also, as because of too many small segments merging is happening continuously and CPU is almost above 90% on hot nodes. This is very critical issue for us and any help to identify the root case is much appreciated . Thanks Ravi

do you see this to all indices or just to a specific index? did u set refresh_interval at cluster level?

for most of indices we are seeing this issue. refresh interval set in default mapping which will be applicable to all indices. Please find the index settings applied for a index:

  "index" : {
    "refresh_interval" : "30s",
    "translog" : {
      "sync_interval" : "30s",
      "durability" : "async"
    },
    "auto_expand_replicas" : "false",
    "provided_name" : "log_instance-demo-2020.05.20",
    "creation_date" : "1589932961715",
    "priority" : "100",
    "number_of_replicas" : "0",
    "uuid" : "E1Jyyi-xTKiSH-jyuVCWFQ",
    "version" : {
      "created" : "7040299"
    },
    "lifecycle" : {
      "name" : "log_h_w2_d4"
    },
    "codec" : "best_compression",
    "routing" : {
      "allocation" : {
        "require" : {
          "box_type" : "hot"
        }
      }
    },
    "number_of_shards" : "6"
  }

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.