Hello,
I am trying to implement the rollover pattern, as described in this excellent blog post, using curator, and I'm having some trouble with the shrinking part.
I would like to:
- Shrink the index, and create the shrunk index with zero replicas, then
- perform the other actions on the shrunk index (forcemerge, etc.), then
- create replicas for redundancy on the shrunk index, and
- only then delete the original unshrunk index.
I'm having trouble doing this with curator, which seems to only support deleting the original index immediately following a successful shrink. I tried finding other workarounds but couldn't find any. I believe some features that enable this may be missing, and I have a few feature requests prepared, but I wanted to check first with the community if maybe I'm missing something?
I'm posting this in the Elasticsearch category, as there is no Curator category.
Thanks for your thoughts.
//Dan
PS: My action file (WIP):
actions:
1:
action: shrink
description: Shrink index to one shard
options:
shrink_node: DETERMINISTIC
node_filters:
permit_masters: True
number_of_shards: 1
number_of_replicas: 0
delete_after: True
shrink_suffix: -inactive
wait_for_active_shards: 1
extra_settings:
settings:
index.codec: best_compression
filters:
- filtertype: pattern
kind: suffix
value: -inactive
exclude: True
- filtertype: pattern
kind: prefix
value: log-
- filtertype: age
unit: days
unit_count: 7
direction: older
source: field_stats
field: '@timestamp'
stats_result: min_value
- filtertype: count
count: 1
exclude: False
2:
action: forcemerge
description: Forcemerge the index to one segment
options:
max_num_segments: 1
filters:
- filtertype: pattern
kind: suffix
value: -inactive
- filtertype: forcemerged
max_num_segments: 1
exclude: True
3:
action: replicas
description: Set forcemerged indexes to have replicas
options:
count: 0
wait_for_completion: True
filters:
- filtertype: pattern
kind: suffix
value: -inactive
- filtertype: forcemerged
max_num_segments: 1