How to rollover data stream during reindex

Elastic Stack Elasticsearch
, ,
1

I have an index with size 5gb.
I would like to reindex it to a data stream with rollover policy of max size 1gb and 7 days.

This is my policy

PUT _ilm/policy/jacob_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_age": "1d",
            "max_size": "1gb"
          }
        }
      }
    }
  }
}

When I perform the reindex, all the data is saved in a single backing index, while I expected it to be ~5 indices.

Why isn't the rollover triggered?
Is there a way to force the rollover during reindex?

I also tried to read from the source index and bulk insert to the data stream.
This did create a new backing index, but not after the index size reached 1gb.
The new backing index was created after the 1st index reached 2.4gb.
Why is that?
How can I make sure a rollover occurs as expected?

Thanks!

2

Hi there, the ILM rollover is only checked every 10 minutes by default, you can change the indices.lifecycle.poll_interval setting to a lower setting during your reindexing to make sure that it checks more frequently.