Hey All,
I am working on a Disaster Recovery Process for our 3 Node Elasticsearch cluster. The Warm Node in the cluster is using Curator to Snapshot all Indexes on the cluster to an S3 Snapshot Repository every hour. During a Disaster Recovery process we are deleting the 3 nodes and creating 3 new ones and then restoring all of the indexes including the .system ones, however I am coming across errors with the ".triggered_watches", ".watcher-history-", ".security_audit_log-", ".monitoring-*" and many other indexes failing to have their shards assigned. I have tried using the cluster reroute API to assign them manually but they won't assign. We are on Elasticsearch Version 6.2.3 any suggestions?