We have a 4-node cluster with 2 indexes each with 1 replica. We are
currently indexing all documents against a single index, while the other
one is empty. One of the nodes it doing very little work because it has
mostly empty shards.
I need a way to have ES distribute the shards from each index evenly across
all 4 nodes. I know by default, ES considers all shards equally. I've
looked into some of the ES config settings around shard balancing, but it's
not obvious how to make this happen.
On Wed, 2013-02-06 at 08:26 -0800, Craig Brown wrote:
We have a 4-node cluster with 2 indexes each with 1 replica. We are
currently indexing all documents against a single index, while the
other one is empty. One of the nodes it doing very little work because
it has mostly empty shards.
I need a way to have ES distribute the shards from each index evenly
across all 4 nodes. I know by default, ES considers all shards
equally. I've looked into some of the ES config settings around shard
balancing, but it's not obvious how to make this happen.
If you're using v0.19.10 or higher, then you can use the cluster reroute
API to move your shards around:
If not, and you're not using the other index, try just closing and
reopening it. If that doesn't work, you may have to delete it
The best way to do this is via
index.routing.allocation.total_shards_per_node:
I have a process that looks at the number of data nodes I have and the
number of shards for an index and automatically sets this to force a fairer
balancing. Would be a nice feature to have in the core elasticsearch,
though.
Best Regards,
Paul
On Wednesday, February 6, 2013 9:45:02 AM UTC-7, Clinton Gormley wrote:
Hi Craig
On Wed, 2013-02-06 at 08:26 -0800, Craig Brown wrote:
We have a 4-node cluster with 2 indexes each with 1 replica. We are
currently indexing all documents against a single index, while the
other one is empty. One of the nodes it doing very little work because
it has mostly empty shards.
I need a way to have ES distribute the shards from each index evenly
across all 4 nodes. I know by default, ES considers all shards
equally. I've looked into some of the ES config settings around shard
balancing, but it's not obvious how to make this happen.
If you're using v0.19.10 or higher, then you can use the cluster reroute
API to move your shards around:
If not, and you're not using the other index, try just closing and
reopening it. If that doesn't work, you may have to delete it
clint
Any pointers appreciated, thanks!
-Craig
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.
I have a process that looks at the number of data nodes I have and the
number of shards for an index and automatically sets this to force a fairer
balancing. Would be a nice feature to have in the core elasticsearch,
though.
Best Regards,
Paul
On Wednesday, February 6, 2013 9:45:02 AM UTC-7, Clinton Gormley wrote:
Hi Craig
On Wed, 2013-02-06 at 08:26 -0800, Craig Brown wrote:
We have a 4-node cluster with 2 indexes each with 1 replica. We are
currently indexing all documents against a single index, while the
other one is empty. One of the nodes it doing very little work because
it has mostly empty shards.
I need a way to have ES distribute the shards from each index evenly
across all 4 nodes. I know by default, ES considers all shards
equally. I've looked into some of the ES config settings around shard
balancing, but it's not obvious how to make this happen.
If you're using v0.19.10 or higher, then you can use the cluster reroute
API to move your shards around:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.