Rollover by month

Elastic Stack Elasticsearch
1

Hello,

I am trying to use the rollover api to help create indexes by month. However, the rollover api does not seem to support months as a max_age unit: Failed to parse setting [max_age] with value [1M] as a time value: unit is missing or unrecognized.

Is there a better approach for achieving indexes by month with the rollover api?

Thanks

2

That should work. Can you show your Rollover API call body?

Also, for reference, using date math with Rollover API.

3

Here is the rollover api body

POST /rollovertest/_rollover?dry_run
{
  "conditions": {
    "max_age":   "3M"
  }
}
4

What indices are behind the alias rollovertest?

Since you're doing a dry_run, what are the results if you set max_age to 90d?

5

I created a new index with:

PUT /%3Crollovertest-%7Bnow-6M%2Fd%7D-1%3E
{
  "aliases": {
    "rollovertest": {}
  }
}

Here are the results of max_age with 90d:

POST /rollovertest/_rollover?dry_run
{
  "old_index": "rollovertest-2017.05.16-1",
  "new_index": "rollovertest-2017.05.16-000002",
  "rolled_over": false,
  "dry_run": true,
  "acknowledged": false,
  "shards_acknowledged": false,
  "conditions": {
    "[max_age: 90d]": false
  }
}

Does max_age use the index creation date or the date in the index name?

6

It uses index creation_date.

7

Thats what I thought. Do you have any idea why months do not work for max age?

8

I did some asking. It turns out that you need a TimeValue, rather than a DateMath object. This list shows the supported units for TimeValues:

  • d - days
  • h - hours
  • m - minutes
  • s - seconds
  • ms - milliseconds
  • micros - microseconds
  • nanos - nanoseconds

This makes sense, because how would you calculate a month back? The first of the current month? The one before? 30 days ago? The same day of the month for the month previous (e.g. you're on the 30th of March, and there's no 30th of February)?

1 Like
9

Ah, thanks. I was looking for a list like that but had trouble finding it.. I care about the same month. I will probably take a different approach using the automatic index creation with templates. Thanks for the help!

10

The best thing about rollover is that it supports rolling over based on size or age. If you want completely fixed periods for your indices, sticking with the old and trusted index naming convention might work just as well.

1 Like
11

In that case, you might find what you're looking for with Elasticsearch Curator, specifically the period filter. You can combine that with Curator's implementation of the Rollover API, called an action in Curator.

Curator could help you determine the age of an index, and you could use even a 1s max_age, because you'd have vetted the age in the filters already.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.