Recurring errors when backing up to GCS - FileAlreadyExistsException

Elastic Stack Elasticsearch
,
1

Hello, I have a recurring errors when Im doing snapshot to GCS bucket:

{\"type\":\"index_shard_snapshot_failed_exception\",\"reason\":\"IndexShardSnapshotFailedException[Failed to perform snapshot (index files)]; nested: FileAlreadyExistsException[xyz/indices/tTYieIisQdCeGLfPnH6NKQ/2/__HRq6f247SmW_OPCTm2KdhA: Precondition Failed];

Generally I get this error sometimes - every couple of days,

I have a couple of Elastic clusters next to each other that are doing snapshot to the same bucket just different base path. Usually the error occurs for several shards during a single erroneous snapshot - for example for 3 of 1640

Cluster build - 3 master nodes, 3 client nodes, 10 data nodes.

curl -X GET "localhost:9200/_slm/policy?pretty"
{
  "dc0001" : {
    "version" : 1,
    "modified_date_millis" : 1621756086856,
    "policy" : {
      "name" : "<dc-0001-snap-{now/d}>",
      "schedule" : "0 0 0 * * ?",
      "repository" : "dc0001",
      "config" : {
        "indices" : [
          "*"
        ]
      },
      "retention" : {
        "expire_after" : "0ms",
        "min_count" : 7,
        "max_count" : 7
      }
    },

Repository-gcs plugin version: 7.5.1

Elasticsearch version: 7.5.1

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.