Test and Dev Migration Best Practice

Elastic Stack Elasticsearch
#1

Hi All,

Does anyone have and good references for how people are handing the cycle of refreshes from production instances in the cloud back to Dev and Test instances on developer laptops.

We are keen to keep dev/test laptop instances in sync with the data, visualisations and dashboards etc on our cloud based production machines. Right now this is a manual process so we get mismatches all the time.

Someone must be able to tell us how to do it - can we restore from snapshots?

Thanks

John

#2

Snapshots is probably the best idea.

#3

Hi and thanks

How do I get snapshots off the cloud based elastic service for us on our laptops?

Is there a guide or reference?

Thanks

John

#4

If they're in S3, or similar, then just add that as a repo to the local Elasticsearch cluster.
Otherwise there's likely tools that can pull the data locally, but that's outside the scope of what we can assist with.

#5

Thanks for the update. That’s a shame though.

Given we are talking about the standard and automatic elastic cloud product snapshots then I was hoping there would be a way to access that data.

I understand you (elastic cloud) do store the elastic cloud snapshots in s3 but I don’t have any credentials with which to access these files

Are you certain that accessing these S3 snapshots for our developers’ use is not supported?

It seems like an essential thing for a development team to be able to use?

Thanks

John

#6

Yes it's not possible sorry. You can very easily add your own (additional) repo to snapshot and restore from.

I'd also encourage you to also create a request with the Support team as a feature request so we can pass it to our Product team.

#7

Hi @johnwood perhaps we have misunderstood.

You / your company can provide the S3 buckets for the repo the elasticsearch snapshots are stored in. Then whomever you want to grant access to can have access. Or you can copy to another of your buckets like any other S3

See here, just configure your own bucket.

1 Like