Rollover exceeds defined limit

Elastic Search
1

I am creating a policy where I want to perform the rollover to the 10 documents as a test that I am performing but I find that if you are generating the new datastream indexes but the number of documents is greater than 10, this is normal?

Something similar happens when I configure the rollover to run at 5 minutes, the new indexes are generated but spend more than 5 minutes to generate a new index.

This is the policy

{
  "vision_policy": {
    "version": 3,
    "modified_date": "2024-07-30T18:30:35.361Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_docs": 10
            }
          }
        },
        "delete": {
          "min_age": "365d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      }
    },
    "in_use_by": {
      "indices": [
        ".ds-visionone-pruebas-2024.07.30-000009",
        ".ds-visionone-pruebas-2024.07.30-000018",
        ".ds-visionone-pruebas-2024.07.30-000007",
        ".ds-visionone-pruebas-2024.07.30-000008",
        ".ds-visionone-pruebas-2024.07.30-000019",
        ".ds-visionone-pruebas-2024.07.30-000005",
        ".ds-visionone-pruebas-2024.07.30-000016",
        ".ds-visionone-pruebas-2024.07.30-000006",
        ".ds-visionone-pruebas-2024.07.30-000017",
        ".ds-visionone-pruebas-2024.07.30-000014",
        ".ds-visionone-pruebas-2024.07.30-000003",
        ".ds-visionone-pruebas-2024.07.30-000004",
        ".ds-visionone-pruebas-2024.07.30-000015",
        ".ds-visionone-pruebas-2024.07.30-000012",
        ".ds-visionone-pruebas-2024.07.30-000001",
        ".ds-visionone-pruebas-2024.07.30-000013",
        ".ds-visionone-pruebas-2024.07.30-000002",
        ".ds-visionone-pruebas-2024.07.30-000010",
        ".ds-visionone-pruebas-2024.07.30-000011",
        ".ds-visionone-pruebas-2024.07.30-000020"
      ],
      "data_streams": [
         "visionone-pruebas"
      ],
      "composable_templates": [
        "vision_template"
      ]
    }
  }
}
2

Yes, this is normal, the ILM policy runs on a schedule of 10 minutes by default and you may end with more documents before a rollover is triggered.

Also, check this answer for a similar question: ILM rollover when max_docs reached - #2 by Christian_Dahlqvist

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.