Migrate elasticsearch data to AWS from another cloud

[trenunu]

Hey Guys,

We're currently hosting our elasticsearch cluster on azure(not azure service, we set up our own cluster), and we are migrating to AWS. Am wondering what is the best approach.

I saw the repository-s3 plugin, if i understand it correctly, it needs our cluster to be on AWS already, is that correct? Because it uses EC2 IAM to do the authorization, and we are just VMs in azure. I tried and i saw it try to query 169.254.169.254/latest/user-data for IAM info. is this correct?

if this is the case, the only way i can think of is to take a fs snapshot and transfer that snapshot to s3 and go from there? any other suggestions?

Cheers
T

[Armin_Braun]

Hi @trenunu

I saw the repository-s3 plugin, if i understand it correctly, it needs our cluster to be on AWS already, is that correct?

No, you can configure an S3 repository on a cluster that is outside of AWS as well.

Because it uses EC2 IAM to do the authorization, and we are just VMs in azure. I tried and i saw it try to query 169.254.169.254/latest/user-data for IAM info. is this correct?

If you set up credentials for an S3 client in the ES secret store as documented here it will work fine and not try to fall back to the EC2 environment for credentials I think.

if this is the case, the only way i can think of is to take a fs snapshot and transfer that snapshot to s3 and go from there? any other suggestions?

Technically this will work as well, but I think will be easier to try and get the credentials for the S3 repository right so you can write straight from our current cluster in Azure.

Hope that helps

[dadoonet (David Pilato)]

Not exactly related but did you look at https://www.elastic.co/cloud ?

It runs on AWS, Azure, GCP...

Cloud by elastic is one way to have access to all features, all managed by us. Think about what is there yet like Security, Monitoring, Reporting, SQL, Canvas, APM, Logs UI, Infra UI, SIEM, Maps UI and what is coming next ...

[trenunu]

Thanks Armin