Curator: delay deleting shrunk indices

[dandrestor (Dan Nestor)]

Hello,

I am trying to implement the rollover pattern, as described in this excellent blog post, using curator, and I'm having some trouble with the shrinking part.

I would like to:

1. Shrink the index, and create the shrunk index with zero replicas, then
2. perform the other actions on the shrunk index (forcemerge, etc.), then
3. create replicas for redundancy on the shrunk index, and
4. only then delete the original unshrunk index.

I'm having trouble doing this with curator, which seems to only support deleting the original index immediately following a successful shrink. I tried finding other workarounds but couldn't find any. I believe some features that enable this may be missing, and I have a few feature requests prepared, but I wanted to check first with the community if maybe I'm missing something?

I'm posting this in the Elasticsearch category, as there is no Curator category.

Thanks for your thoughts.
//Dan

PS: My action file (WIP):

actions:
  1:
    action: shrink
    description: Shrink index to one shard
    options:
      shrink_node: DETERMINISTIC
      node_filters:
        permit_masters: True
      number_of_shards: 1
      number_of_replicas: 0
      delete_after: True
      shrink_suffix: -inactive
      wait_for_active_shards: 1
      extra_settings:
        settings:
          index.codec: best_compression
    filters:
      - filtertype: pattern
        kind: suffix
        value: -inactive
        exclude: True
      - filtertype: pattern
        kind: prefix
        value: log-
      - filtertype: age
        unit: days
        unit_count: 7
        direction: older
        source: field_stats
        field: '@timestamp'
        stats_result: min_value
      - filtertype: count
        count: 1
        exclude: False
  2:
    action: forcemerge
    description: Forcemerge the index to one segment
    options:
      max_num_segments: 1
    filters:
      - filtertype: pattern
        kind: suffix
        value: -inactive
      - filtertype: forcemerged
        max_num_segments: 1
        exclude: True
  3:
    action: replicas
    description: Set forcemerged indexes to have replicas
    options:
      count: 0
      wait_for_completion: True
    filters:
      - filtertype: pattern
        kind: suffix
        value: -inactive
      - filtertype: forcemerged
        max_num_segments: 1

[theuntergeek (Aaron Mildenstein)]

So, you're saying that you want the Shrink action to do all of those actions as part of the shrink action, and not require other actions/steps?

That's a bit scary in terms of how long it could take, but I see the appeal, especially if you're pressed for space (as deleting after doing those things would be more space consuming).

It's not a bad idea. Please submit a feature request to https://github.com/elastic/curator

The flow I'm imagining would be as follows:

1. Shrink takes place with 0 replicas
2. forceMerge this index after (ostensibly with best_compression)
3. update the number of replicas to number_of_replicas
4. delete the original index.

This will be a bit of a change from the current behavior, so it will need to be a special flag, in case other users don't want this as part of their sequence.

[dandrestor (Dan Nestor)]

No, I'm not saying that. I actually believe the shrink action is a little bit monolithic in its current form, and I totally agree that it would take too long to handle the entire process under a single action.

A couple of features that could be interesting to have which would enable this pattern:

1. Have an option to enable assigning the index to an alias, following the successful and unsuccessful processing of that index by the shrink action. This could even be a good idea to have as a generic option for all actions. Subsequent actions could use the alias filter to perform cleanup or follow-up actions on indexes as required based on their alias membership.
2. Have a sort of "name transformation" option on certain index, such as the delete index action. One could then apply the delete action for all filters ending in, say, "-inactive", transform the index name using some regex, and then apply the action (delete) using the transformed name as a parameter. This would enable for instance selecting all "...-inactive" indexes using a filter, but deleting those that actually end in "...-active".

I think idea number one is way better as it's more modular. Let me know if you like either of those; in the meantime I'll open a feature request also.

//Dan

[theuntergeek (Aaron Mildenstein)]

I do like option 1. It's like "tagging" an index. But since Elasticsearch does not have such a feature, and alias is a good way to tag it.