CCR follower for large index fails

Elastic Stack Elasticsearch
1

Hello!
I'w trying to configure Elasticsearch CCR using manuals at elastic.co

Everything works fine for small indexes.

But, when it comes to indexes large then 2 GB, something strange happens - index following is paused and follower index is red.

After this, I've deleted follower index both from Cross Cluster Replication and Index Management tabs in kibana and then run follower creation with Kibana Dev Tools:

    PUT /my_index_new_ccr/follow?wait_for_active_shards=1
          {
              "remote_cluster": "remote",
              "leader_index" : "my_index"
          }

Execution of this command resulted in this error:

> {
  "error": {
      "root_cause": [
        {
          "type": "snapshot_restore_exception",
          "reason": "[_ccr_remote:_latest_/_latest_] cannot restore index [my_index_new] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"
        }
      ],
      "type": "snapshot_restore_exception",
      "reason": "[_ccr_remote:_latest_/_latest_] cannot restore index [my_index_new] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"
    },
    "status": 500
  }

Elasticsearch logs for the same time shows a lot of errors on failed recovery. This string is present in almost all of them:

> Caused by: org.elasticsearch.ElasticsearchTimeoutException: [internal:admin/ccr/restore/file_chunk/get] timed out after [1m]

Here is my configuration:

  • Elasticsearch and Kibana versions are 7.5, the same for both local and remote cluster.
  • Remote Elasticsearch cluster consists of 4 nodes, 3 of them are both master and data nodes.
  • All of the servers are physical nodes with SSD, 1 GigE network channel and have ~50 GB RAM for heap and cache.
  • Local cluster consists of 3 master and data nodes.
  • Remote index size is 410 gb and 6 primary shards. Follower index creation fail.
  • The remote other index - 17 gb, 8 primary shards. Follower index creation fail.

I'm using Elasticsearch trial licence for these tests.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.