# Snapshots are failing in ECE minio

**URL:** <https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553>\
**Category:** Elastic Cloud Enterprise (ECE)\
**Created:** [November 12, 2019, 3:42pm UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553 "2019-11-12T15:42:28Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [November 12, 2019, 3:42pm UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/1 "2019-11-12T15:42:28Z")

</div>

Hi there,

I have integrated a repository with minio. After I recognized that elasticsearch 6.x was not able to resolve hostnames I changed to IP. Elasticsearch 7.x worked fine for a while.

I use version 2.4.1 and updated yesterday from 2.3.1. But the error has already been in 2.3.1.  
Now one deployment has some error by creating a snapshot. But it has 100 working snapshots already:

`Unknown s3 client name [cloud-REPONAME]. Existing client configs: Default`

In the advanced settings I see:

```auto
  "snapshot": {
    "enabled": true,
    "repository": {
      "config": {
        "repository_id": "REPONAME",
        "snapshot_config_type": "reference"
      }
    },
    "suspended": {}
  }

```

Why is in the Error message a "cloud-" prefix?

---

<div class="post-metadata">

**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\
**Post date:** [November 12, 2019, 5:07pm UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/2 "2019-11-12T17:07:51Z")

</div>

From 7.x/2.4 (not 100% what the version boundary is), the snapshot config code is distributed between 2 places - the `_snapshot` endpoint, and the secure settings

It looks from the error like the `_snapshot` endpoint includes the auto-generated `client` field (you can confirm by doing `GET _snapshot/found-snapshots`), but the cluster does not include the secure settings (eg you won't find the S3 settings in the secure settings in the advanced cluster data lower down)

Can you post (sanitized versions of) those two JSON object?

Can you try applying a no-op plan to the cluster?

---

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [November 13, 2019, 6:34am UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/3 "2019-11-13T06:34:43Z")

</div>

```auto
{
  "found-snapshots": {
    "type": "s3",
    "settings": {
      "bucket": "ece-es7",
      "client": "cloud-REPONAME",
      "base_path": "snapshots/c371465ec830483d8a640e8cb49e3b",
      "endpoint": "https://IPMINIO:9000",
      "protocol": "https"
    }
  }
}

```

This is the only Information in advanced I found:

```auto
  "snapshot": {
    "enabled": true,
    "repository": {
      "config": {
        "repository_id": "REPONAME",
        "snapshot_config_type": "reference"
      }
    },
    "suspended": {}
  },

```

---

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [November 13, 2019, 7:06am UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/4 "2019-11-13T07:06:48Z")

</div>

I do not find the secure settings. This could be the error.

How do I get the settings set on default?

---

<div class="post-metadata">

**Author:** ![dimatkach](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@dimatkach](https://discuss.elastic.co/u/dimatkach)\
**Post date:** [November 13, 2019, 3:20pm UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/5 "2019-11-13T15:20:14Z")

</div>

Could you show the repo definition from `Platform/Repositories` (switch "Repository Type" to Advanced to get to the actual json). Remove the values for the secrets, I just want to see that the keys are there, and look right.

---

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [November 15, 2019, 7:56pm UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/6 "2019-11-15T19:56:07Z")

</div>

I can do this when I am back in office on Tuesday.

But it is the same from the docs:  
[https://www.elastic.co/guide/en/cloud-enterprise/current/ece-configuring-minio.html](https://www.elastic.co/guide/en/cloud-enterprise/current/ece-configuring-minio.html)

```auto
 {
     "type": "s3",
      "settings": {
         "bucket": "ece-backup",
         "access_key": "<your Minio AccessKey>",
         "secret_key": "<your Minio SecretKey>",
         "endpoint": "https://IP>:9000",
         "protocol": "https"
      }
  }

```

thanks it advance.

---

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [November 19, 2019, 12:24pm UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/7 "2019-11-19T12:24:48Z")

</div>

Hi there,

as I have already written. The configuration is:

```auto
{
  "settings": {
    "access_key": "ACCESS_KEY",
    "bucket": "ece-backup",
    "endpoint": "https://IP:9000",
    "protocol": "https",
    "secret_key": "SECRETKEY"
  },
  "type": "s3"
}

```

---

<div class="post-metadata">

**Author:** ![dimatkach](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@dimatkach](https://discuss.elastic.co/u/dimatkach)\
**Post date:** [November 19, 2019, 4:31pm UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/8 "2019-11-19T16:31:37Z")

</div>

Hi,

Could you just clarify ... in your last post the name was `"bucket": "ece-backup"` ,  
but you had posted `_snapshot/found-snapshots` is returning `"bucket": "ece-es7"` ..  
are there multiple repos?

---

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [November 20, 2019, 6:04am UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/9 "2019-11-20T06:04:52Z")

</div>

Hi,

It is `"bucket": "ece-es7"`. I just changed it to the same like the example.

We have multiple repos. One for Elasticsearch 6.x and one for Elasticsearch 7.x.

---

<div class="post-metadata">

**Author:** ![dimatkach](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@dimatkach](https://discuss.elastic.co/u/dimatkach)\
**Post date:** [November 21, 2019, 5:32pm UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/10 "2019-11-21T17:32:28Z")

</div>

So, the way this is supposed to work is that when ES container is allocated, we take the secrets from your shared repo definition and add them to the Elasticsearch keystore on each node. Looks like this did not happen in your case for some reason.

One more thing to try is to go to that repo definition under `Platform/Repositories`, switch to advanced, and (after making sure the settings are correct), `Save` it. That should cause the credentials to be written to keystore again in case it got out of sync somehow.

After saving, wait a few minutes to give it time to sync to all nodes, then from the API console do

```auto
PUT _snapshot/found_snapshots
{
    "type": "s3",
    "settings": {
      "bucket": "ece-es7",
      "client": "cloud-REPONAME",
      "base_path": "snapshots/c371465ec830483d8a640e8cb49e3b",
      "endpoint": "https://IPMINIO:9000",
      "protocol": "https"
    }
 }

```

(replacing "REPONAME" and "IPMINIO" with the real things of course).  
If that works, it should be all fixed.

If it fails though, it seems that we would have to take a closer look at your configuration.  
In that case, would it be possible for you to open a support ticket to make it easier to exchange information? We would then circle back and post the result here once we understand what is going on?

I hope this helps!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 5, 2019, 5:32pm UTC](https://discuss.elastic.co/t/snapshots-are-failing-in-ece-minio/207553/11 "2019-12-05T17:32:32Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
