# ElasticSearch S3 Snapshot Retrieval Failing in DR- EC2

**URL:** <https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801>\
**Category:** Elasticsearch\
**Created:** [March 7, 2018, 12:59am UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801 "2018-03-07T00:59:27Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![titan1978](https://avatars.discourse-cdn.com/v4/letter/t/c37758/32.png) [@titan1978](https://discuss.elastic.co/u/titan1978)\
**Post date:** [March 7, 2018, 12:59am UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/1 "2018-03-07T00:59:27Z")

</div>

Gurus,  
I've successfully managed to setup Snapshots PUT and GET when using s3-repositories in us-east-1 bucket. However, when I try to pull the snapshot or even repositories via GET when my ec2 instances are in us-west-1 : I get a repository not found exception ( **the bucket and repository names being the same** ).

All the roles are identical configured in the EC2 Instances between the EAST and WEST. Why would the WEST GET repositories call fail?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 7, 2018, 6:03am UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/2 "2018-03-07T06:03:54Z")

</div>

DId you set the endpoint?

---

<div class="post-metadata">

**Author:** ![titan1978](https://avatars.discourse-cdn.com/v4/letter/t/c37758/32.png) [@titan1978](https://discuss.elastic.co/u/titan1978)\
**Post date:** [March 7, 2018, 6:15am UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/3 "2018-03-07T06:15:47Z")

</div>

@dadoonet Sorry - I didn't follow. Could you elaborate?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 7, 2018, 7:09am UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/4 "2018-03-07T07:09:47Z")

</div>

Some settings need to be set: [https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-client.html](https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-client.html)

---

<div class="post-metadata">

**Author:** ![titan1978](https://avatars.discourse-cdn.com/v4/letter/t/c37758/32.png) [@titan1978](https://discuss.elastic.co/u/titan1978)\
**Post date:** [March 7, 2018, 3:23pm UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/5 "2018-03-07T15:23:11Z")

</div>

@dadoonet I didn't configure any client settings (in the documentation, it said it would default to "default"). Was looking at this website and they didn't seem to pass any client information either:

> **[Configuring Elasticsearch to Use S3 for Snapshots](https://www.objectrocket.com/blog/elasticsearch/elasticsearch-s3)**
>
> How to configure and use S3 for Elasticsearch snapshots via the Elasticsearch API

Could not configuring the client even with defaults still be an issue?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 7, 2018, 3:43pm UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/6 "2018-03-07T15:43:07Z")

</div>

> [@titan1978](#):
>
> Could not configuring the client even with defaults still be an issue?

This is what I meant initially.  
The default endpoint is east. If you want to use another region you need to set it.

An update will come to automatically guess it from the ec2 instance by default but it's not yet there.

---

<div class="post-metadata">

**Author:** ![titan1978](https://avatars.discourse-cdn.com/v4/letter/t/c37758/32.png) [@titan1978](https://discuss.elastic.co/u/titan1978)\
**Post date:** [March 7, 2018, 5:11pm UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/7 "2018-03-07T17:11:44Z")

</div>

@dadoonet This is my config in the west region that is not able to pull the repository. The EAST one that works fine just has the security groups and region configured for east.  
With this in mind, could you specify what in this config are you indicating by this statement? **The default endpoint is east. If you want to use another region you need to set it.**

> cluster.name: my-cluster  
> node.data: true  
> node.master: true  
> node.attr.box\_type: cold  
> node.name: ${HOSTNAME}  
> path.data: /opt/mount/var/data/mydir/elasticsearch  
> path.logs: /opt/mount/var/log/mydir/elasticsearch  
> network.host: _ec2:privateIpv4_  
> discovery.zen.hosts\_provider: ec2  
> discovery.zen.minimum\_master\_nodes: 4  
> discovery.ec2.host\_type: private\_ip  
> discovery.ec2.tag.Name: mytag  
> discovery.ec2.groups: MYSG  
> discovery.ec2.any\_group: false  
> cloud.aws.region: us-west-2  
> cloud.aws.proxy.host: [myproxy.myhost.com](http://myproxy.myhost.com)  
> cloud.aws.proxy.port: 8099  
> cloud.node.auto\_attributes: true  
> action.destructive\_requires\_name: true  
> bootstrap.memory\_lock: true

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 7, 2018, 8:31pm UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/8 "2018-03-07T20:31:03Z")

</div>

I'm starting to get a better picture now. Please use `</>` icon to format your code instead of the citation icon which introduces bad formatting.

What is the version you are using?

---

<div class="post-metadata">

**Author:** ![titan1978](https://avatars.discourse-cdn.com/v4/letter/t/c37758/32.png) [@titan1978](https://discuss.elastic.co/u/titan1978)\
**Post date:** [March 7, 2018, 8:36pm UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/9 "2018-03-07T20:36:19Z")

</div>

5.5.2

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 7, 2018, 9:43pm UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/10 "2018-03-07T21:43:09Z")

</div>

So you are trying to access a S3 repository which is located in east1 but from a machine running in west2. Am I correct?

May be try to set the region (or the endpoint) when creating the repository: [https://www.elastic.co/guide/en/elasticsearch/plugins/5.6/repository-s3-repository.html](https://www.elastic.co/guide/en/elasticsearch/plugins/5.6/repository-s3-repository.html)

```auto
PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "my_bucket_name",
    "region": "us-west-2"
  }
}

```

Check again that you have all needed privileges: [https://www.elastic.co/guide/en/elasticsearch/plugins/5.6/repository-s3-repository.html#repository-s3-permissions](https://www.elastic.co/guide/en/elasticsearch/plugins/5.6/repository-s3-repository.html#repository-s3-permissions)

---

<div class="post-metadata">

**Author:** ![titan1978](https://avatars.discourse-cdn.com/v4/letter/t/c37758/32.png) [@titan1978](https://discuss.elastic.co/u/titan1978)\
**Post date:** [March 7, 2018, 9:59pm UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/11 "2018-03-07T21:59:06Z")

</div>

So it looks like I had to "CREATE" the repository again when switching the west even though I "CREATED" it when I was in the east! After doing this in west, I was successfully able to get to the snapshots!

Is the repository a regional concept and needs to be done each time we create a new stack ? (We destroy the old stack when doing DR, so if we were in the EAST, we first spin up WEST and then destroy the EAST and vice - versa)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 4, 2018, 10:57pm UTC](https://discuss.elastic.co/t/elasticsearch-s3-snapshot-retrieval-failing-in-dr-ec2/122801/13 "2018-04-04T22:57:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
