Best reindex strategy to change only the number of Shards

Elastic Stack Elasticsearch
1

Hello,

I have a very large Index in ES (millions of entries over 100s of GB) with
too few Shards.
And I want only to create the same Index but with more, smaller, efficient
Shards.
And I would love it if this could happen quickly (minutes?)

My googling indicates that the elasticsearch-reindex Plugin
(https://github.com/karussell/elasticsearch-reindex) is the best approach.
Is that still the case??
Under the covers elasticsearch-reindex uses the recommended scan-and-scroll
technique.

But reading the doc/code in the elasticsearch-reindex Plugin, they
indicate that perhaps they should have implemented it as a River.
And I’ve not been able to find a River that accomplished this same task.

It seems that this might be a pretty common thing to need to do, and that
there may be a very efficient way to accomplish it that I am missing.
And it seems that it might be fastest to have some sort of very fast
streaming process that could essentially stream the entries of indexOld to
indexNew with a new number of Shards.

Is elasticsearch-reindex still the best tool for this job??
(I am on ES 1.0.2)

Thanks much,
— Chris

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/14711f9d-0848-4923-b00a-62cd85d8a337%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hey,

maybe there is no need to reindex? Cant you just index all your new data
into another index and use an alias to search on both indices? You might
want to watch https://www.youtube.com/watch?v=gBOhCNcjC7k

--Alex

On Sat, Jul 12, 2014 at 3:19 PM, Chris Berry chriswberry@gmail.com wrote:

Hello,

I have a very large Index in ES (millions of entries over 100s of GB) with
too few Shards.
And I want only to create the same Index but with more, smaller, efficient
Shards.
And I would love it if this could happen quickly (minutes?)

My googling indicates that the elasticsearch-reindex Plugin (
GitHub - karussell/elasticsearch-reindex: Simple re-indexing. To backup, apply index settings changes and more ElasticMagic) is the best approach.
Is that still the case??
Under the covers elasticsearch-reindex uses the recommended
scan-and-scroll technique.

But reading the doc/code in the elasticsearch-reindex Plugin, they
indicate that perhaps they should have implemented it as a River.
And I’ve not been able to find a River that accomplished this same task.

It seems that this might be a pretty common thing to need to do, and that
there may be a very efficient way to accomplish it that I am missing.
And it seems that it might be fastest to have some sort of very fast
streaming process that could essentially stream the entries of indexOld to
indexNew with a new number of Shards.

Is elasticsearch-reindex still the best tool for this job??
(I am on ES 1.0.2)

Thanks much,
— Chris

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/14711f9d-0848-4923-b00a-62cd85d8a337%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/14711f9d-0848-4923-b00a-62cd85d8a337%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8%3DuhT2zJkjpWjKOxMqC6nTPs8riJAdjUf-0sngKbVoMQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.