Hi
I have an issue with Curator that I would like an opinion on. This is elk 6.2.0 with curator 5.4, but also happens on 6.1.2. I hope I am missing somethng obvious.
I have a cluster , that implements the hot/warm architecture. I am using the roll-over approach of handling the indexes- After 1 day , I migrate the indixes to the warm node. This works fine.
After 1 more day, I want to shrink and forcemerge. I want this to take place on my warm nodes and I want the indexes to remain on the warm nodes.
Curator picks one of my warm nodes as shrink node, moves the shards to the shrink node.Shrinks the index to 1 shard. After this is complete the index is moved to the HOT node. Then curator starts the post allocation step, which moves the index back to the WARM node.
So obviously, this is taking much longer than it should.
My skrink step looks like this at the moment:
actions:
5:
action: shrink
options:
ignore_empty_list : true
shrink_node: DETERMINISTIC
node_filters:
permit_masters: False
number_of_shards: 1
number_of_replicas: 0
shrink_suffix: '-shrink'
delete_after: false
post_allocation:
allocation_type: require
key: box_type
value: warm
wait_for_active_shards: 1
extra_settings:
settings:
index.codec: best_compression
index.routing.allocation.require.box_type: warm
wait_for_completion: True
wait_interval: 30
max_wait: -1
filters:
- filtertype: count
count: 2
pattern: '^(dns-active)-\d+$'
reverse: true
exclude: true
- filtertype: age
source: field_stats
direction: older
unit: days
unit_count: 2
field: '@timestamp'
stats_result: min_value
I have been trying to fix this with index templates , but no luck so far.
Anything I can do to avoid this copying of the index back to the HOT node and then back to the WARM node?
Regards
Kim