ElasticSearch S3 Snapshot Retrieval Failing in DR- EC2

Elastic Stack Elasticsearch
1

Gurus,
I've successfully managed to setup Snapshots PUT and GET when using s3-repositories in us-east-1 bucket. However, when I try to pull the snapshot or even repositories via GET when my ec2 instances are in us-west-1 : I get a repository not found exception (the bucket and repository names being the same).

All the roles are identical configured in the EC2 Instances between the EAST and WEST. Why would the WEST GET repositories call fail?

2

DId you set the endpoint?

3

@dadoonet Sorry - I didn't follow. Could you elaborate?

4

Some settings need to be set: https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-client.html

5

@dadoonet I didn't configure any client settings (in the documentation, it said it would default to "default"). Was looking at this website and they didn't seem to pass any client information either:

Could not configuring the client even with defaults still be an issue?

6

This is what I meant initially.
The default endpoint is east. If you want to use another region you need to set it.

An update will come to automatically guess it from the ec2 instance by default but it's not yet there.

7

@dadoonet This is my config in the west region that is not able to pull the repository. The EAST one that works fine just has the security groups and region configured for east.
With this in mind, could you specify what in this config are you indicating by this statement? The default endpoint is east. If you want to use another region you need to set it.

cluster.name: my-cluster
node.data: true
node.master: true
node.attr.box_type: cold
node.name: ${HOSTNAME}
path.data: /opt/mount/var/data/mydir/elasticsearch
path.logs: /opt/mount/var/log/mydir/elasticsearch
network.host: ec2:privateIpv4
discovery.zen.hosts_provider: ec2
discovery.zen.minimum_master_nodes: 4
discovery.ec2.host_type: private_ip
discovery.ec2.tag.Name: mytag
discovery.ec2.groups: MYSG
discovery.ec2.any_group: false
cloud.aws.region: us-west-2
cloud.aws.proxy.host: myproxy.myhost.com
cloud.aws.proxy.port: 8099
cloud.node.auto_attributes: true
action.destructive_requires_name: true
bootstrap.memory_lock: true

8

I'm starting to get a better picture now. Please use </> icon to format your code instead of the citation icon which introduces bad formatting.

What is the version you are using?

9

5.5.2

10

So you are trying to access a S3 repository which is located in east1 but from a machine running in west2. Am I correct?

May be try to set the region (or the endpoint) when creating the repository: https://www.elastic.co/guide/en/elasticsearch/plugins/5.6/repository-s3-repository.html

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "my_bucket_name",
    "region": "us-west-2"
  }
}

Check again that you have all needed privileges: https://www.elastic.co/guide/en/elasticsearch/plugins/5.6/repository-s3-repository.html#repository-s3-permissions

11

So it looks like I had to "CREATE" the repository again when switching the west even though I "CREATED" it when I was in the east! After doing this in west, I was successfully able to get to the snapshots!

Is the repository a regional concept and needs to be done each time we create a new stack ? (We destroy the old stack when doing DR, so if we were in the EAST, we first spin up WEST and then destroy the EAST and vice - versa)

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.