shrink using curator python never completes and the cluster status goes to unassigned shards.
AWS support explanation :
Cause of the issue: From ES 7.0 , Elastic deprecated the use of "copy_settings" parameter in the shrink api call and made "copy_settings=true" as the default value. There is no way to modify this parameter anymore. Reference --> https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#copy-settings-deprecated-shrink-split-apis "copy_settings" --> allows users to specify whether they want the target index (shrunken index) to have the same index settings as the source index. Now, with "copy_settings" set to true in the shrink api calls for 7.x ES clusters, all the index settings from the target index including "index.routing.allocation.require._name" and "index.blocks.read_only" are also copied down to the target index. The above setting forces ES to assign shards of target index to a specific node that is defined in "index.routing.allocation.require._name" but with replicas enabled, ES cannot allocate replica of a primary shard onto the same datanode. So, Cluster settings prevent ES from allocating replicas onto the same datanode as primaries and index settings prevent the shards to be allocated to any other node but "index.routing.allocation.require._name" value. Previously, these settings were never inherited down to the shrunken indices as they
the workaround is to
Step1. Update the source index routing setting to a specific node and block all writes.
Step2. Perform Shrink operation.
Step3. Remove routing and write block settings from the shrunken index.
the workaround works, but I loose the benefit of using curator shrink DETERMINISTIC option ( https://www.elastic.co/guide/en/elasticsearch/client/curator/current/option_shrink_node.html ), automatically determines the node with most free space
anyone successfully ran shrink using curator on 7.x ES AWS cluster without any workarounds ?