Copying indices on same cluster

Elastic Stack Elasticsearch
1

I was wondering how one would go about duplicating an index into another on
the same cluster. I know you can probably write code to scan and reindex,
but is that the only way? Is there some feature in ES that facilitates
this kind of thing? I'm on 0.19.7.

-- KJ

--

2

Hello!

Haven't tried it myself, but you can try snapshot'ing and using the backup on another cluster - http://www.elasticsearch.org/guide/reference/api/admin-indices-gateway-snapshot.html

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch

I was wondering how one would go about duplicating an index into another on the same cluster. I know you can probably write code to scan and reindex, but is that the only way? Is there some feature in ES that facilitates this kind of thing? I'm on 0.19.7.

-- KJ

--

3

Hi Rafal,
Thanks for the suggestion, but I don't want to copy to another cluster.
I want an identical index with a different name but with all the data
intact on the SAME cluster.

-- KJ

On Sunday, November 4, 2012 10:14:06 AM UTC-8, Rafał Kuć wrote:

Hello!

Haven't tried it myself, but you can try snapshot'ing and using the backup
on another cluster -
Elasticsearch Platform — Find real-time answers at scale | Elastic

*--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch -
Elasticsearch

I was wondering how one would go about duplicating an index into another
on the same cluster. I know you can probably write code to scan and
reindex, but is that the only way? Is there some feature in ES that
facilitates this kind of thing? I'm on 0.19.7.

-- KJ

--

4

Hi KJ,

In that case, the following should work for a single node:

If you have more nodes, you'd only have to "copy" metadata once for
the whole cluster, then stop all nodes, copy the shards and start all
the nodes again.

All this assumes you have and ES version of at least 0.19.8 and you
have the default setting of gateway.local.auto_import_dangled: true

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Mon, Nov 5, 2012 at 5:43 AM, krispyjala krispyjala@gmail.com wrote:

Hi Rafal,
Thanks for the suggestion, but I don't want to copy to another cluster. I want an identical index with a different name but with all the data intact on the SAME cluster.

-- KJ

On Sunday, November 4, 2012 10:14:06 AM UTC-8, Rafał Kuć wrote:

Hello!

Haven't tried it myself, but you can try snapshot'ing and using the backup on another cluster - Elasticsearch Platform — Find real-time answers at scale | Elastic

--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Elasticsearch

I was wondering how one would go about duplicating an index into another on the same cluster. I know you can probably write code to scan and reindex, but is that the only way? Is there some feature in ES that facilitates this kind of thing? I'm on 0.19.7.

-- KJ

--

--

5

Wow, great script, Radu. As I said though, I am on 0.19.7. Are there any
features I need in 0.19.8 that will prevent me from using this script on
0.19.7?

Thanks,
KJ

On Mon, Nov 5, 2012 at 1:34 AM, Radu Gheorghe radu.gheorghe@sematext.comwrote:

Hi KJ,

In that case, the following should work for a single node:
copy one Elasticesarch index to another · GitHub

If you have more nodes, you'd only have to "copy" metadata once for
the whole cluster, then stop all nodes, copy the shards and start all
the nodes again.

All this assumes you have and ES version of at least 0.19.8 and you
have the default setting of gateway.local.auto_import_dangled: true

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Mon, Nov 5, 2012 at 5:43 AM, krispyjala krispyjala@gmail.com wrote:

Hi Rafal,
Thanks for the suggestion, but I don't want to copy to another
cluster. I want an identical index with a different name but with all the
data intact on the SAME cluster.

-- KJ

On Sunday, November 4, 2012 10:14:06 AM UTC-8, Rafał Kuć wrote:

Hello!

Haven't tried it myself, but you can try snapshot'ing and using the
backup on another cluster -
Elasticsearch Platform — Find real-time answers at scale | Elastic

--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch -
Elasticsearch

I was wondering how one would go about duplicating an index into
another on the same cluster. I know you can probably write code to scan
and reindex, but is that the only way? Is there some feature in ES that
facilitates this kind of thing? I'm on 0.19.7.

-- KJ

--

--

--

6

Hello,

Sorry, I got lost in my own thoughts :(. Although I didn't test it on
0.19.7, it should work.

My initial approach would have been to copy the data directory of the
"old" index to a "new" directory and restart Elasticsearch. Then,
Elasticsearch would have seen the "new" directory as a dangled index
and try to import it. And you would need 0.19.8 to import those
dangled indices.

But I couldn't actually copy an index this way, so that's why I went
on the path of simply reposting the same metadata in the new index,
and copying the shards themselves when ES is offline. That seems to
work, and it has nothing to do with dangled indices. So it should work
for 0.19.7 as well.

I would suggest to try this option in a test environment and post here
if you have any trouble.

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Mon, Nov 5, 2012 at 3:55 PM, Kris Jirapinyo krispyjala@gmail.com wrote:

Wow, great script, Radu. As I said though, I am on 0.19.7. Are there any
features I need in 0.19.8 that will prevent me from using this script on
0.19.7?

Thanks,
KJ

On Mon, Nov 5, 2012 at 1:34 AM, Radu Gheorghe radu.gheorghe@sematext.com
wrote:

Hi KJ,

In that case, the following should work for a single node:
copy one Elasticesarch index to another · GitHub

If you have more nodes, you'd only have to "copy" metadata once for
the whole cluster, then stop all nodes, copy the shards and start all
the nodes again.

All this assumes you have and ES version of at least 0.19.8 and you
have the default setting of gateway.local.auto_import_dangled: true

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Mon, Nov 5, 2012 at 5:43 AM, krispyjala krispyjala@gmail.com wrote:

Hi Rafal,
Thanks for the suggestion, but I don't want to copy to another
cluster. I want an identical index with a different name but with all the
data intact on the SAME cluster.

-- KJ

On Sunday, November 4, 2012 10:14:06 AM UTC-8, Rafał Kuć wrote:

Hello!

Haven't tried it myself, but you can try snapshot'ing and using the
backup on another cluster -
Elasticsearch Platform — Find real-time answers at scale | Elastic

--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch -
Elasticsearch

I was wondering how one would go about duplicating an index into
another on the same cluster. I know you can probably write code to scan and
reindex, but is that the only way? Is there some feature in ES that
facilitates this kind of thing? I'm on 0.19.7.

-- KJ

--

--

--

--