Duplicate an elasticsearch cluster

Elastic Stack Elasticsearch
1

I just started to use elasticsearch and created a small cluster (2
machines) filled with some index data. So far everything works well.

Now I have to duplicate the cluster on another two machines and
hopefully I do not have to index everything again.

I did one-to-one copying for the data directory from the old cluster
to the new cluster. After I restarted the cluster, my client code (in
pyes) shows error "IndexMissingException: [test-profile] missing",
where "test-profile" is the index name.

Since the data directory contains mata info, I thought restarting the
cluster with copied data should automatically work. Now it seems not
the case. Could someone point out what could be the potential problem
here?

Thanks.

2

Hey!

Are you using any gateways? I believe you can copy the gateway data to a
new cluster, and that should restore it, you'd have to confirm this though,
as I've never done it myself.

Patrick

patrick eefy net

On Thu, May 31, 2012 at 1:47 PM, aw airwaywong@gmail.com wrote:

I just started to use elasticsearch and created a small cluster (2
machines) filled with some index data. So far everything works well.

Now I have to duplicate the cluster on another two machines and
hopefully I do not have to index everything again.

I did one-to-one copying for the data directory from the old cluster
to the new cluster. After I restarted the cluster, my client code (in
pyes) shows error "IndexMissingException: [test-profile] missing",
where "test-profile" is the index name.

Since the data directory contains mata info, I thought restarting the
cluster with copied data should automatically work. Now it seems not
the case. Could someone point out what could be the potential problem
here?

Thanks.

3

Thanks for the hint.

Actually I am not sure if I am using any gateways or not. I just use
the default setup and I don't remember I have ever set any gateway. If
I don't have gateway, is there some way to copy and load the data to
another cluster?

On May 31, 12:54 pm, Patrick patr...@eefy.net wrote:

Hey!

Are you using any gateways? I believe you can copy the gateway data to a
new cluster, and that should restore it, you'd have to confirm this though,
as I've never done it myself.

Patrick
----------------------------------------Patrick Ancillotti - New York | about.me
patrick eefy net

On Thu, May 31, 2012 at 1:47 PM, aw airwayw...@gmail.com wrote:

I just started to use elasticsearch and created a small cluster (2
machines) filled with some index data. So far everything works well.

Now I have to duplicate the cluster on another two machines and
hopefully I do not have to index everything again.

I did one-to-one copying for the data directory from the old cluster
to the new cluster. After I restarted the cluster, my client code (in
pyes) shows error "IndexMissingException: [test-profile] missing",
where "test-profile" is the index name.

Since the data directory contains mata info, I thought restarting the
cluster with copied data should automatically work. Now it seems not
the case. Could someone point out what could be the potential problem
here?

Thanks.

4

Hey,

Once you setup the local gateway, you can take that directory, copy it to
the new cluster, and then setup the same configuration to load the data
(it'll automatically load data it finds in the gateway location)

Patrick

patrick eefy net

On Thu, May 31, 2012 at 2:12 PM, aw airwaywong@gmail.com wrote:

Thanks for the hint.

Actually I am not sure if I am using any gateways or not. I just use
the default setup and I don't remember I have ever set any gateway. If
I don't have gateway, is there some way to copy and load the data to
another cluster?

On May 31, 12:54 pm, Patrick patr...@eefy.net wrote:

Hey!

Are you using any gateways? I believe you can copy the gateway data to a
new cluster, and that should restore it, you'd have to confirm this
though,
as I've never done it myself.

Patrick

Patrick Ancillotti - New York | about.me

patrick eefy net

On Thu, May 31, 2012 at 1:47 PM, aw airwayw...@gmail.com wrote:

I just started to use elasticsearch and created a small cluster (2
machines) filled with some index data. So far everything works well.

Now I have to duplicate the cluster on another two machines and
hopefully I do not have to index everything again.

I did one-to-one copying for the data directory from the old cluster
to the new cluster. After I restarted the cluster, my client code (in
pyes) shows error "IndexMissingException: [test-profile] missing",
where "test-profile" is the index name.

Since the data directory contains mata info, I thought restarting the
cluster with copied data should automatically work. Now it seems not
the case. Could someone point out what could be the potential problem
here?

Thanks.

5

Using the local gateway (the default one) means that you can simply copy
over the data location. If the two clusters have a different cluster name,
make sure to rename the first directory under the data location to the
updated cluster name.

On Thu, May 31, 2012 at 9:47 PM, aw airwaywong@gmail.com wrote:

I just started to use elasticsearch and created a small cluster (2
machines) filled with some index data. So far everything works well.

Now I have to duplicate the cluster on another two machines and
hopefully I do not have to index everything again.

I did one-to-one copying for the data directory from the old cluster
to the new cluster. After I restarted the cluster, my client code (in
pyes) shows error "IndexMissingException: [test-profile] missing",
where "test-profile" is the index name.

Since the data directory contains mata info, I thought restarting the
cluster with copied data should automatically work. Now it seems not
the case. Could someone point out what could be the potential problem
here?

Thanks.