Index blocks write operation when it moved to warm phase by ILM

Elastic Stack Elasticsearch
1

Hi Elasticsearch team,

Index blocks write operation when it moved to the warm phase by ILM.

{
  "indices" : {
    "nxlog-2019.08.06" : {
      "index" : "nxlog-2019.08.06",
      "managed" : true,
      "policy" : "compass-clients-policy",
      "lifecycle_date_millis" : 1564987134180,
      "phase" : "warm",
      "phase_time_millis" : 1565325477233,
      "action" : "complete",
      "action_time_millis" : 1565329539339,
      "step" : "complete",
      "step_time_millis" : 1565329539339,
      "phase_execution" : {
        "policy" : "compass-clients-policy",
        "phase_definition" : {
          "min_age" : "3d",
          "actions" : {
            "allocate" : {
              "include" : { },
              "exclude" : { },
              "require" : {
                "box_type" : "warm"
              }
            },
            "forcemerge" : {
              "max_num_segments" : 1
            },
            "set_priority" : {
              "priority" : 50
            }
          }
        },
        "version" : 1,
        "modified_date_in_millis" : 1565264841307
      }
    }
  }
}

Could you explain why the block write operation has been set to the index?

2

Hey,

I don't see an index block here and no read-only ILM action. Did you create your own custom policy? Also, which Elasticsearch version is this?

--Alex

3

Hi Alex, I'm using 7.2 version. I create my own policy. I also configure to force merge index in the warm phase. The number of segment is 1.

I noticed that when the index moved to warm phase. The index blocks the write operation.

4

I think I know the cause. The force merge in warm phase will block the write operation until the force merge completed.

My indexing nodes receive the old logs from clients (> 8 days) which should be rejected by indexing nodes but it does not.
So, it send to the warm indexes during the force merge operation.

5

indeed, you are right. the force_merge action sets the index to read only. I missed that in the above policy. See https://www.elastic.co/guide/en/elasticsearch/reference/7.3/_actions.html#ilm-forcemerge-action

6

This cluster is for non-production environment. I configure 1 shard and 1 replica for weekly.

Could you please suggest that should I use force merge option to move index from hot phase to warm phase?

7

Hi, I updated my policy to do not execute force merge in warm phase.
However, it always call force merge.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.