Automate reindex operation with curator and dynamically create destionation indices

[victor.nilsson]

Hi,

We are as of now creating daily indices which we want to reindex to a monthly index. We want the flow to look something like this if possible:

1. Curator checks if index is older than 1 month
2. If index is older than said month, let curator reindex the source index to a dynamically created destination index that would be created for each month.

So for example, lets say we have "logstash-index1-2018-07-01" & "logstash-index1-2018-07-29":

1. Curator runs with the reindex action and checks for source indices which are older than 1 month
2. Curator finds "logstash-index1-2018-07-01" which is older than 1 month
3. Curator creates the destination index for July and starts reindexing the source index to our newly created index created for July.
4. Curator also finds the second index "logstash-index1-2018-07-29" which is also in July and is older than 1 month.
5. Curator checks if the destination index already exists and if it does, proceeds to reindex the source index to the July index

Is this possible? If so would date math be the way to achieve this?

Something like this perhaps

---
actions:
1:
    description: "Reindex operation for indices older than 1 month"
    action: reindex
    options:
        disable_action: True
        wait_interval: 9
        max_wait: -1
        request_body:
          source:
            index: REINDEX_SELECTION
          dest:
            index: "<logstash-{now/M-1M{YYYY.MM}}>"
    filters:
      - filtertype: age
      source: creation_date
      direction: older
      unit: month
      unit_count: 1

[theuntergeek (Aaron Mildenstein)]

Is there a reason you require the data to be in daily indices at all? Why not dodge this monthly stuff altogether and use rollover indices? I mean, I read over your flow, and while it seems plausible for Curator to do, it seems an awful lot of work for your cluster to do which could be altogether avoided by using rollover indices.