Rollover exceeds defined limit

[juancamiloll (Juan C)]

I am creating a policy where I want to perform the rollover to the 10 documents as a test that I am performing but I find that if you are generating the new datastream indexes but the number of documents is greater than 10, this is normal?

Something similar happens when I configure the rollover to run at 5 minutes, the new indexes are generated but spend more than 5 minutes to generate a new index.

This is the policy

{
  "vision_policy": {
    "version": 3,
    "modified_date": "2024-07-30T18:30:35.361Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_docs": 10
            }
          }
        },
        "delete": {
          "min_age": "365d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      }
    },
    "in_use_by": {
      "indices": [
        ".ds-visionone-pruebas-2024.07.30-000009",
        ".ds-visionone-pruebas-2024.07.30-000018",
        ".ds-visionone-pruebas-2024.07.30-000007",
        ".ds-visionone-pruebas-2024.07.30-000008",
        ".ds-visionone-pruebas-2024.07.30-000019",
        ".ds-visionone-pruebas-2024.07.30-000005",
        ".ds-visionone-pruebas-2024.07.30-000016",
        ".ds-visionone-pruebas-2024.07.30-000006",
        ".ds-visionone-pruebas-2024.07.30-000017",
        ".ds-visionone-pruebas-2024.07.30-000014",
        ".ds-visionone-pruebas-2024.07.30-000003",
        ".ds-visionone-pruebas-2024.07.30-000004",
        ".ds-visionone-pruebas-2024.07.30-000015",
        ".ds-visionone-pruebas-2024.07.30-000012",
        ".ds-visionone-pruebas-2024.07.30-000001",
        ".ds-visionone-pruebas-2024.07.30-000013",
        ".ds-visionone-pruebas-2024.07.30-000002",
        ".ds-visionone-pruebas-2024.07.30-000010",
        ".ds-visionone-pruebas-2024.07.30-000011",
        ".ds-visionone-pruebas-2024.07.30-000020"
      ],
      "data_streams": [
         "visionone-pruebas"
      ],
      "composable_templates": [
        "vision_template"
      ]
    }
  }
}

[leandrojmp (Leandro Pereira)]

@juancamiloll

I am creating a policy where I want to perform the rollover to the 10 documents as a test that I am performing but I find that if you are generating the new datastream indexes but the number of documents is greater than 10, this is normal?

Yes, this is normal, the ILM policy runs on a schedule of 10 minutes by default and you may end with more documents before a rollover is triggered.

Also, check this answer for a similar question: ILM rollover when max_docs reached - #2 by Christian_Dahlqvist