Copying and renaming a cluster

Elastic Stack Elasticsearch
#1

I've got a production cluster that I would like to mirror down to
development intact. I'd like to rename the cluster so I don't get
confused and things don't conflict. What would I need to do to do
this. My gateway is up on S3 and I have local files in EBS for the
nodes. I was going to copy the local files to the new cluster nodes
and the S3 files to the new bucket. What else do I need to do.

Thanks!

#2

You should also change under the data directory of each node the name of hte cluster. The format is: data/cluster_name/nodes/N/... .

Out of curiosity, why do you use both EBS for data location, and s3? If you use s3 gateway, then you can keep the data on the local instance disk, its already backed in s3.

If you want ot use EBS, then use local gateway, will be faster and use less resources (s3).

-shay.banon
On Wednesday, April 20, 2011 at 1:35 AM, mr.gaffo wrote:

I've got a production cluster that I would like to mirror down to
development intact. I'd like to rename the cluster so I don't get
confused and things don't conflict. What would I need to do to do
this. My gateway is up on S3 and I have local files in EBS for the
nodes. I was going to copy the local files to the new cluster nodes
and the S3 files to the new bucket. What else do I need to do.

Thanks!

#3

We were doing it as backup and super backup.

That's interesting though, thanks for the suggestion. I thought we
needed to have both.

So if the local is on ephemeral storage and we restart the node (eg
local is wiped), it'll just sync with S3, join the cluster, and go?

-Mike

On Apr 19, 4:25 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You should also change under the data directory of each node the name of hte cluster. The format is: data/cluster_name/nodes/N/... .

Out of curiosity, why do you use both EBS for data location, and s3? If you use s3 gateway, then you can keep the data on the local instance disk, its already backed in s3.

If you want ot use EBS, then use local gateway, will be faster and use less resources (s3).

-shay.banon

On Wednesday, April 20, 2011 at 1:35 AM, mr.gaffo wrote:

I've got a production cluster that I would like to mirror down to
development intact. I'd like to rename the cluster so I don't get
confused and things don't conflict. What would I need to do to do
this. My gateway is up on S3 and I have local files in EBS for the
nodes. I was going to copy the local files to the new cluster nodes
and the S3 files to the new bucket. What else do I need to do.

Thanks!

#4

Yep, it will resync with s3, and possibly need to download all those index files again. Though, if its on ebs, then not as much data will be downloaded since the node data dir can be reused.
On Wednesday, April 20, 2011 at 2:52 AM, mr.gaffo wrote:

We were doing it as backup and super backup.

That's interesting though, thanks for the suggestion. I thought we
needed to have both.

So if the local is on ephemeral storage and we restart the node (eg
local is wiped), it'll just sync with S3, join the cluster, and go?

-Mike

On Apr 19, 4:25 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You should also change under the data directory of each node the name of hte cluster. The format is: data/cluster_name/nodes/N/... .

Out of curiosity, why do you use both EBS for data location, and s3? If you use s3 gateway, then you can keep the data on the local instance disk, its already backed in s3.

If you want ot use EBS, then use local gateway, will be faster and use less resources (s3).

-shay.banon

On Wednesday, April 20, 2011 at 1:35 AM, mr.gaffo wrote:

I've got a production cluster that I would like to mirror down to
development intact. I'd like to rename the cluster so I don't get
confused and things don't conflict. What would I need to do to do
this. My gateway is up on S3 and I have local files in EBS for the
nodes. I was going to copy the local files to the new cluster nodes
and the S3 files to the new bucket. What else do I need to do.

Thanks!

#5

i want to copy kibana dashboard from local to clustrer ,that kibana
dashboard has 40 visualization plots in local. i want to copy that to
cluster .
i tried importing all dashboard and visulization from local
to cluster in setting tabs , its giving error coudn't locate index
pattern.
are there any better way of coping from local to cluster ?

#6