Issues on rollover in the datastream

Elastic Search
1

I'm currently using Elasticsearch version 8.7.1.
Some indexes are converted to datastreams, and are set to rollover every 200 GB and per day.
The problem is that every day there should be a rollover by date, and the index should be created, but after it is created in 200GB increments, the rollover is carried out according to the date.
I'm wondering if we can increase the priority of being rollover to dates.

"hot": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_size": "200gb",
"max_primary_shard_size": "200gb",
"max_age": "1d"
},
"set_priority": {
"priority": 100
}
}
}

2

Here are some clarifications from the official document:

  1. A rollover action must specify at least one max_* condition, it may include zero or more min_* conditions. An empty rollover action is invalid.
  2. The index will roll over once any max_* condition is satisfied and all min_* conditions are satisfied. Note, however, that empty indices are not rolled over by default.
  3. When you specify multiple rollover conditions, the index is rolled over when any of the max_* and all of the min_* conditions are met.

So in your case, a new index will be created after max_size reaches 200GB or 24 hours after index creation.

I'm wondering if we can increase the priority of being rollover to dates.

There is no priority for that but you can add the min_age: 1d condition. In that way, the index can't be rollover if it's not older than 1 day.

Caution! Please keep in mind that, if the index grows too big it will still wait one day for rollover.

1 Like